From Threads to Virtual Threads: Innovations in Java Concurrency#

1. Birth of Java Threads#

Java has provided built-in support for threads since its earliest days. The Thread class and Runnable interface represent the core constructs for multithreading. Simply put:

• A thread is a lightweight process managed by the Java Virtual Machine (JVM).
• A runnable is a task object that tells the thread what to do when it starts executing.

A basic example of creating and using a thread:

1
public class SimpleThreadDemo {
2
    public static void main(String[] args) {
3
        Thread thread = new Thread(() -> {
4
            System.out.println("Hello from a thread!");
5
        });
6
        thread.start();
7

8
        System.out.println("Hello from the main method!");
9
    }
10
}

In this snippet, a Thread object is instantiated and passed a Runnable (in lambda form). When thread.start() is invoked, the code inside the lambda runs in a separate thread, concurrent with the main thread.

2. Lifecycle of a Thread#

A thread goes through several states:

1. New: Created but not yet started.
2. Runnable: Eligible to run, waiting its turn for CPU time.
3. Running: Actively executing on a CPU core.
4. Blocked / Waiting: Paused due to an IO operation, synchronization lock, or a wait call.
5. Terminated: Completed execution.

Understanding these states helps in debugging and optimizing thread usage, as you’d want to avoid or reduce contention and blocking states where possible.

3. Risks of Low-Level Thread Management#

While working with raw Thread objects, problems can arise:

• Complex synchronization: Multiple threads might need to share data, leading to potential race conditions or java.util.ConcurrentModificationException if you don’t manage access properly.
• Limited control: Thread creation and destruction overhead can be large. Poor management can lead to resource exhaustion.
• Difficult to scale: Handling large numbers of threads manually in large-scale apps becomes cumbersome and less efficient.

Hence, more advanced concurrency frameworks were introduced in Java.

1. The Executor Concept#

To address the scaling and resource-management challenges, Java introduced the Executors framework in Java 5. Instead of creating threads manually, you submit tasks to an executor service, which then assigns these tasks to a pool of reusable threads. This standardization made concurrency code more maintainable and robust.

1
import java.util.concurrent.ExecutorService;
2
import java.util.concurrent.Executors;
3

4
public class ThreadPoolExample {
5
    public static void main(String[] args) {
6
        ExecutorService executor = Executors.newFixedThreadPool(4);
7

8
        for (int i = 0; i < 10; i++) {
9
            int taskId = i;
10
            executor.submit(() -> {
11
                System.out.println("Task " + taskId + " executed by " + Thread.currentThread().getName());
12
            });
13
        }
14

15
        executor.shutdown();
16
    }
17
}

2. Types of Executor Services#

3. Fork/Join Framework#

Another major addition was the Fork/Join framework, which is targeted primarily at parallelizing CPU-intensive tasks by recursively breaking them down into smaller tasks. While not as commonly used in everyday tasks as ExecutorService, it’s vital in compute-bound scenarios.

1
import java.util.concurrent.RecursiveTask;
2

3
public class SumArrayTask extends RecursiveTask<Long> {
4
    private static final int THRESHOLD = 10_000;
5
    private final long[] arr;
6
    private final int start;
7
    private final int end;
8

9
    public SumArrayTask(long[] arr, int start, int end) {
10
        this.arr = arr;
11
        this.start = start;
12
        this.end = end;
13
    }
14

15
    @Override
16
    protected Long compute() {
17
        int length = end - start;
18
        if (length < THRESHOLD) {
19
            long sum = 0;
20
            for (int i = start; i < end; i++) {
21
                sum += arr[i];
22
            }
23
            return sum;
24
        } else {
25
            int midpoint = start + length / 2;
26
            SumArrayTask leftTask = new SumArrayTask(arr, start, midpoint);
27
            SumArrayTask rightTask = new SumArrayTask(arr, midpoint, end);
28

29
            leftTask.fork();
30
            Long rightResult = rightTask.compute();
31
            Long leftResult = leftTask.join();
32

33
            return leftResult + rightResult;
34
        }
35
    }
36
}

1. Synchronous Programming#

With synchronous programming, each operation in a thread blocks until its job is complete. If the thread is waiting on I/O, it remains blocked before proceeding to the next line of code. Synchronous programming is simplicity-friendly but not always efficiency-friendly when dealing with high-latency operations.

2. Asynchronous / Non-blocking Programming#

Asynchronous programming doesn’t require a caller to wait for the entire task to complete. Instead, a callback or future result is returned. This approach often uses fewer threads to handle more tasks concurrently, especially useful for high-level concurrency, like managing millions of client connections in a web server scenario.

Example: Using CompletableFuture

1
import java.util.concurrent.CompletableFuture;
2

3
public class AsyncDemo {
4
    public static void main(String[] args) {
5
        CompletableFuture.supplyAsync(() -> {
6
            // Simulate a long-running operation
7
            sleep(2000);
8
            return "Result of the async computation";
9
        }).thenAccept(result -> {
10
            System.out.println("Callback received: " + result);
11
        });
12

13
        System.out.println("Main thread continues...");
14
        sleep(3000); // Wait to see the async result before shutting down
15
    }
16

17
    private static void sleep(long millis) {
18
        try {
19
            Thread.sleep(millis);
20
        } catch (InterruptedException e) {
21
            e.printStackTrace();
22
        }
23
    }
24
}

As powerful as asynchronous programming is, it also introduces callback chains, more complex error handling, and debugging difficulties. This is where the new concept of virtual threads can potentially bridge the gap by offering a simpler synchronous style with high concurrency.

Key Advantages of Virtual Threads#

1. Lightweight: Creating and maintaining a virtual thread is far cheaper than a platform thread.
2. Ease of Use: You can still use a synchronous approach in code (e.g., plain blocking I/O calls), but the underlying scheduling is managed in a non-blocking manner by the JVM.
3. Scalability: Virtual threads enable you to handle countless concurrent tasks without ballooning resource usage.

How to Get Started with Virtual Threads#

Virtual threads are available in modern Java releases (starting with experimental or preview in Java 19/20 onward). When properly enabled, you can create virtual threads using a new factory method on Thread or through dedicated executors.

1. Creating a Single Virtual Thread#

1
public class VirtualThreadExample {
2
    public static void main(String[] args) {
3
        Thread vThread = Thread.ofVirtual().unstarted(() -> {
4
            System.out.println("Hello from a virtual thread!");
5
        });
6

7
        vThread.start();
8
        System.out.println("Main thread ends here.");
9
    }
10
}

In this snippet:

• Thread.ofVirtual() is the factory for creating a virtual thread.
• unstarted(...) creates a new instance that does not start automatically. We then call start().

2. Virtual Thread Executor#

Virtual threads can also be managed by an executor service, simplifying large-scale concurrency:

1
import java.util.concurrent.ExecutorService;
2
import java.util.concurrent.Executors;
3

4
public class VirtualThreadExecutor {
5
    public static void main(String[] args) {
6
        ExecutorService executor = Executors.newVirtualThreadPerTaskExecutor();
7

8
        for (int i = 0; i < 50; i++) {
9
            int taskId = i;
10
            executor.submit(() -> {
11
                System.out.println("Task " + taskId + " running on " + Thread.currentThread());
12
            });
13
        }
14

15
        executor.close(); // Closes the executor
16
    }
17
}

• Executors.newVirtualThreadPerTaskExecutor() returns an executor that creates a new virtual thread for each task submitted.
• Note that shutdown is replaced with close() in these new classes to handle the structured concurrency model.

3. Real-World Scenarios#

1. Server-Side Requests: Imagine a web server where each incoming request is handled by a new virtual thread. The server can scale to thousands or millions of concurrent connections without running out of threads and memory.
2. Microservices: In microservices that perform numerous RPC calls, you can spin up a virtual thread for each external call, improving concurrency while maintaining a straightforward synchronous style.
3. High-Frequency Trading: Systems requiring extremely fast and concurrent data processing can benefit from the low overhead of virtual threads, seamlessly balancing CPU use among active tasks.

1. Reactor / Reactive Streams Integration#

While virtual threads can reduce the need for fully reactive approaches (like Reactor or RxJava) for many I/O-bound tasks, reactive libraries still offer backpressure and streaming capabilities that are beneficial in real-time data streams. One might combine the simplicity of virtual threads for request handling with reactive streams for efficient data processing.

2. Thread Locals and Scoping#

Traditional thread-local variables can also be used in virtual threads, but caution is advised. Virtual threads may be reused or parked/resumed in ways you might not anticipate, so thoroughly evaluate any reliance on ThreadLocal for storing critical data.

Example:

1
public class VirtualThreadLocalExample {
2
    private static final ThreadLocal<String> context = ThreadLocal.withInitial(() -> "Default context");
3

4
    public static void main(String[] args) {
5
        Thread thread = Thread.ofVirtual().unstarted(() -> {
6
            context.set("VirtualThread Context");
7
            System.out.println("Context: " + context.get());
8
        });
9
        thread.start();
10
    }
11
}

3. Loom and Continuations#

Under the hood, Project Loom uses something akin to bytecode manipulation to represent blocking calls as continuations that can suspend and resume. This means that a blocking call within a virtual thread no longer necessarily pins an OS thread; it simply triggers a continuation pause.

4. Monitoring and Debugging Virtual Threads#

Running hundreds of thousands of virtual threads might raise concerns about how to debug or profile them:

• Monitoring Tools: The same JDK tools (e.g., jcmd, jstack) are evolving to display virtual threads. However, jstack output can be massive if you have hundreds of thousands of threads.
• Better Observability: Profiling tools and logging frameworks will likely evolve to handle the scale of virtual threads, possibly sampling only active threads.

5. High-Throughput Systems#

If your application primarily does CPU-bound work, virtual threads benefit you insofar as you can have more concurrent tasks waiting, but pure CPU-bound tasks are still limited by the number of available cores. If tasks frequently block on I/O, you can handle significantly more concurrency, which is ideal for high-throughput systems that juggle many simultaneous connections.

6. Potential Pitfalls#

• Memory Leaks: Even if creating a virtual thread is cheap, a large number of them storing references to objects can still cause memory issues if they are not properly managed.
• Third-Party Libraries: Some libraries may not yet be fully compatible with the assumptions in virtual threads (especially for advanced scheduling or thread-local usage). Always validate dependencies.