Virtual Threads and High Performance IOs

This section is the culmination of everything we’ve learned about IO. It explains how Virtual Threads allow us to write code that looks like the old “Thread-per-Request” model but performs like the “Asynchronous/NIO” model. 1. The “Why” In the previous NIO (Non-Blocking) models, we had to break our logic into “callbacks” or “promises.” This made error handling and stack traces a nightmare. Virtual Threads change the game: The Magic: When a Virtual Thread performs a blocking IO operation (like socket.read() or jdbc.executeQuery()), the underlying JVM unmounts the virtual thread from the physical OS thread (Carrier Thread). The Result: The physical OS thread is now free to run another Virtual Thread while the first one waits for the network. No OS-level context switching occurs. 2. Comparison: NIO (Selectors) vs. Virtual Threads Feature NIO / Netty / Event Loop Virtual Threads (Loom) Code Style Asynchronous / Reactive. Synchronous / Procedural. Stack Traces Often fragmented and useless. Complete and readable. Debugging Difficult (Step-through is hard). Easy (Standard debugger works). Scalability High (Millions of connections). High (Millions of connections). Resource Usage Low. Low. 3. The “Golden” Snippet: The “Simple” High-Scale Server This code looks exactly like a basic blocking server from 20 years ago, but it can handle 1,000,000 concurrent connections on a standard server. ...

1. The “Why” Until now, we had two choices: Platform Threads (BIO): Easy to write, but expensive. 1,000 threads = 1GB RAM. You run out of memory quickly. Asynchronous (NIO): Scales to millions, but “Callback Hell” makes it incredibly hard to write, debug, and read. Virtual Threads give you the best of both worlds: You write simple, blocking code (like in the 90s), but the JVM magically handles it like a high-performance Asynchronous system under the hood. ...

1. Don’t Pool Virtual Threads The Old Rule: Threads are expensive; create a FixedThreadPool and reuse them. The New Rule: Virtual threads are disposable objects. Create a new one for every single task. Why? A pool’s purpose is to limit resource usage and avoid the cost of thread creation. Since Virtual Threads cost almost nothing to create and take up very little memory, pooling them just adds unnecessary complexity and contention. // DO THIS (New for every task) try (var executor = Executors.newVirtualThreadPerTaskExecutor()) { executor.submit(() -> doWork()); } // DON'T DO THIS (Pooling virtual threads is an anti-pattern) // ExecutorService pool = Executors.newFixedThreadPool(100); 2. Avoid “Pinning” (Replace synchronized) As we discussed, Pinning happens when a Virtual Thread cannot be “unmounted” from its Carrier Thread during a blocking operation. This usually happens inside synchronized blocks. ...

To understand what Virtual Threads do under the hood, we have to look at the “Magic” that happens between the Java code you write and the Operating System (OS) that actually executes it. Before Java 21, a Java Thread was just a thin wrapper around an OS Thread. If you created 1,000 Java threads, the OS had to manage 1,000 stacks, which is why your memory would spike. 1. The Architecture: M:N Scheduling Virtual Threads introduce a “layered” approach. Instead of a 1:1 relationship with the OS, we now have an M:N relationship. ...