I've seen this pattern in countless code reviews over the years. A developer needs to do something in the background—maybe send an email or process a file—and their first instinct is to just wrap the logic in a new Thread() and call .start(). It seems intuitive. It works fine during local testing with two or three items. Then, it hits production, the load spikes, and the entire JVM crashes with an OutOfMemoryError: unable to create new native thread.

The "New Thread Per Task" Trap

Take a look at this snippet. Imagine we're building a system that processes a batch of log files from a directory to find error patterns.

public class LogAnalyzer {
    public void processLogs(List<File> logFiles) {
        for (File file : logFiles) {
            // I'll just spin up a thread for each file to make it fast!
            new Thread(() -> {
                analyzeFile(file);
            }).start();
        }
    }

    private void analyzeFile(File file) {
        // Simulate heavy I/O and processing
        System.out.println("Analyzing " + file.getName() + " on " + Thread.currentThread().getName());
        try { Thread.sleep(1000); } catch (InterruptedException e) { }
    }
}

On the surface, this looks "concurrent." But here is the problem: if logFiles contains 5,000 files, you just told the OS to create 5,000 platform threads. Each thread in Java has its own stack memory (usually 1MB by default). You've just attempted to allocate 5GB of RAM just for the thread stacks, not even counting the actual data processing. Even if you have the RAM, the CPU will spend more time "context switching"—swapping between these thousands of threads—than actually doing the work.

Managing Resources with a Fixed Thread Pool

Instead of manually managing the lifecycle of every single thread, we use the Executor Framework. The core idea is to decouple task submission from task execution. You define a pool of worker threads that stay alive and pull tasks from a queue.

Here is how I would rewrite that LogAnalyzer to be production-ready:

import java.util.concurrent.ExecutorService;
import java.util.concurrent.Executors;
import java.util.concurrent.TimeUnit;

public class LogAnalyzer {
    public void processLogs(List<File> logFiles) {
        // Create a pool with a fixed number of threads (e.g., based on CPU cores)
        int cores = Runtime.getRuntime().availableProcessors();
        ExecutorService executor = Executors.newFixedThreadPool(cores);

        for (File file : logFiles) {
            // We submit a Runnable to the executor instead of creating a new Thread
            executor.submit(() -> {
                analyzeFile(file);
            });
        }

        // Crucial: Tell the executor to stop accepting new tasks and shut down
        executor.shutdown();
        try {
            // Wait for existing tasks to finish so the program doesn't exit early
            if (!executor.awaitTermination(60, TimeUnit.SECONDS)) {
                executor.shutdownNow();
            }
        } catch (InterruptedException e) {
            executor.shutdownNow();
        }
    }

    private void analyzeFile(File file) {
        System.out.println("Analyzing " + file.getName() + " on " + Thread.currentThread().getName());
        try { Thread.sleep(1000); } catch (InterruptedException e) { }
    }
}

Now, no matter if you have 10 files or 10 million, you will only ever have a handful of threads running (equal to your CPU core count). The ExecutorService manages an internal LinkedBlockingQueue; tasks wait there patiently until a worker thread becomes available. This keeps your memory usage predictable and your CPU efficient.

The Danger of the "Forgotten" Shutdown

One thing that trips up a lot of developers is forgetting to call shutdown(). Unlike a local variable that gets garbage collected, the threads in an ExecutorService are often non-daemon threads. If you don't shut down the executor, the JVM will keep running even after your main method finishes because those worker threads are still idling, waiting for more work that will never come.

I generally recommend the shutdown() followed by awaitTermination() pattern. shutdown() is a polite request: "Finish what you're doing, but don't take any new orders." shutdownNow() is the emergency brake: "Stop everything immediately and tell me what tasks didn't get finished."

📋 Practical Task

Exercise: Building a Parallel Image Metadata Extractor

You are tasked with building a tool that simulates extracting metadata from a large folder of images. Since reading file headers is an I/O-bound task, you want to process multiple images concurrently without crashing the system.

Requirements:

• Create a class ImageProcessor with a method processImages(List<String> imageNames).
• Use a FixedThreadPool with exactly 4 threads.
• For each image name in the list, submit a task that prints: "Processing [imageName] on [threadName]".
• Simulate a delay of 200ms per image using Thread.sleep() to mimic disk I/O.
• Ensure the ExecutorService is shut down correctly and the main thread waits for all tasks to complete before printing "All images processed."
• Test your code with a list of at least 20 image filenames to verify that only 4 unique thread names are ever used in the console output.