The 30 second answer
For a server, the number of requests in progress equals throughput times latency; that's Little's Law, and JEP 444 uses it. With one blocking platform thread per request, the OS thread count caps how many requests can be in progress, so it caps throughput. Virtual threads remove that cap. I keep simple blocking code but can handle far more waiting requests, without rewriting everything in reactive style.
Example: if each request takes 200 ms, mostly waiting on a database or API, and you want 10,000 requests per second, Little's Law says about 2,000 requests must be in flight. With a pool of 200 platform threads you top out near 1,000 requests per second, however idle the CPU is.
Before Java 21 the options were bigger thread pools (memory and context-switch cost) or asynchronous/reactive code (harder to write, debug and profile). Virtual threads keep the thread-per-request model and readable stack traces, while making threads cheap.
What becomes the new limit:
- Downstream resources: DB connection pools, rate limits, other services. Bound them with a pool or
Semaphore. - Memory: each blocked task keeps its stack and objects on the heap.
- CPU: virtual threads give nothing for compute-bound work.
[Opinion] If an application is already efficient reactive code, switching to virtual threads mainly improves readability, not raw throughput.
Sources: JEP 444: Virtual Threads (Motivation)