Java Virtual Threads vs Platform Threads Performance Comparison under high load.

Platform Thread vs Java Thread.

Before we begin comparing the performance of virtual threads to platform threads we need to understand the key differences between them. Java’s platform thread is just a wrapper over the OS thread. Since OS threads are managed via the underlying operating system, their scheduling and optimisations are practically inaccessible to JVM. This is where virtual threads comes into action, part of project loom JEP425 virtual threads provide a one-to-one mapping over the underlying OS threads. In this case JVM is responsible to mapping virtual threads to OS threads, and in scenarios where Java App is busy in a non CPU bound task such as Network call or a DB call. JVM can just un-map that thread corresponding to non CPU bound task and free up that OS thread as well, there by allowing full control over scheduling optimisations.

Setting up Spring Application.

For our case we will consider a Spring Boot application (**version 3) which will be running on a TomCat Server, and performing an intensive non CPU bound process. Java Virtual Threads is available on version 19 and 20 in preview mode currently to enable them we need to set preview flags in our build.gradle file. We will be using Java 19 here.

[** Prior versions to 3 may use TomCat 9 which is not optimised to virtual threads.]

compileJava {
 options.compilerArgs += ['--enable-preview']
}

tasks.withType(JavaExec) {
 jvmArgs += '--enable-preview'
}

tasks.withType(Test).all {
 jvmArgs += '--enable-preview'
}

We will have a standard GET Endpoint running on our application that will invoke a non CPU intensive task.

package com.application.virtualthread.controller;

import com.application.virtualthread.service.NonCPUBoundService;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.http.HttpStatus;
import org.springframework.http.ResponseEntity;
import org.springframework.web.bind.annotation.GetMapping;
import org.springframework.web.bind.annotation.RequestMapping;
import org.springframework.web.bind.annotation.RestController;

@RestController
@RequestMapping("/thread-application")
public class AppController {

    @Autowired
    private NonCPUBoundService nonCPUBoundService;

    @GetMapping("/run")
    public ResponseEntity<String> invokeVirtualThreadApplication() {
        nonCPUBoundService.executeNonCPUBoundTask();
        return new ResponseEntity<>(HttpStatus.OK);
    }
}

To mimic a non CPU intensive task in which the application might be busy waiting for some other resource over network, we will use Thread.sleep here for 5 secs.

package com.application.virtualthread.service;

import org.springframework.stereotype.Service;

@Service
public class NonCPUBoundService {
    public void executeNonCPUBoundTask() {
        try {
            Thread.sleep(5000);
        } catch (InterruptedException e) {
            throw new RuntimeException(e);
        }
    }

}

For this 5 sec non CPU bound task, If we consider platform threads; JVM is quite aware that CPU is not being utilised but the OS is clueless over it, so scheduling optimisations can’t be done. Its here where virtual threads beat it, since JVM has a one-to-one mapping it can simply un-map the virtual thread freeing the OS thread for other requests. Spring’s dispatcher servlet maps request onto Tomcat’s thread pool. To use virtual threads as tomcat’s thread pool, we must make some configuration edits:

package com.application.virtualthread.configuration;

import lombok.extern.slf4j.Slf4j;
import org.springframework.beans.factory.annotation.Value;
import org.springframework.boot.autoconfigure.condition.ConditionalOnProperty;
import org.springframework.boot.web.embedded.tomcat.TomcatProtocolHandlerCustomizer;
import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.Configuration;

import java.util.concurrent.Executors;

@Slf4j
@Configuration
public class AppConfig {

    // Enables tomcat virtual pool thread if flag is set true in application.properties.
    @Bean
    @ConditionalOnProperty(name = "virtual-thread.flag", havingValue = "true")
    public TomcatProtocolHandlerCustomizer<?> protocolHandlerVirtualThreadCustomizer() {
        log.info("Using Virtual Threads!!!");
        return protocolHandler -> protocolHandler.setExecutor(Executors.newVirtualThreadPerTaskExecutor());
    }
}

Creating a high load traffic.

Tomcat has default max-thread configured to 200. This means it can serve upto 200 parallel request without any performance degradation. However, if we consider a high-load traffic environment of 1000 parallel requests, at an instant 200 requests will be served instantly and the other 800 requests will have to wait till the threads become free(in our case 5 sec per thread). With virtual threads this issue is resolved as for those 5 sec of non cpu bound task, the thread is freed. For our load-testing we will be using the Apache Benchmark tool to simulate a concurrent traffic of 1000 requests.

// paste this in terminal to hit the endpoint
ab -n 1000 -c 1000 http://localhost:8080/thread-application/run

Performance Reports

Platform Thread

Virtual Thread

The above Benchmark reports suggest Platform thread bound requests took a median latency of 15 sec while those of virtual threads took around 5 secs only. You can visit this repository for implementation details: GitHub

Thanks for reading!