The 30 second answer
For mostly I/O-bound requests on Java 21, I'd use virtual threads, one per request, so threads stop being the bottleneck. Then I protect the real limits: bounded database connections, timeouts on every remote call, and a cap on concurrent calls to each downstream service. When overloaded, I reject or queue extra requests instead of falling over. Caching, batching and scaling out behind a load balancer cover the rest.
- Concurrency model. On Java 21, virtual threads keep blocking code simple while allowing many concurrent requests. In Spring Boot 3.2 and later,
spring.threads.virtual.enabled=truemakes the embedded web server and Spring Boot's task executors use virtual threads. - Protect bottlenecks. The DB connection pool becomes the real limit; put a
Semaphoreor bulkhead in front of each downstream service. - Timeouts everywhere. Connect and read timeouts on HTTP clients and DB queries, so slow dependencies don't pile up waiting requests.
- Backpressure and load shedding. Rate limiting, returning 429 or 503 early, bounded queues, circuit breakers.
- Do less work. Caching, batching, and moving work that doesn't need an immediate answer to a queue.
- Scale out. Stateless instances behind a load balancer, with autoscaling.
Semaphore paymentsLimit = new Semaphore(50);
Response callPayments() throws InterruptedException {
if (!paymentsLimit.tryAcquire(100, TimeUnit.MILLISECONDS)) throw new TooBusyException();
try { return client.call(); } finally { paymentsLimit.release(); }
}[Opinion] Points 2 to 6 are standard engineering practice, not from the JEP.
Sources: JEP 444: Virtual Threads, Spring Boot reference: virtual threads, Semaphore Javadoc
Sources (0)
See inline citations in the answer above.