I was working on a project the other day where I needed to do two things: scrape some data from a dozen different APIs and then run a heavy mathematical simulation on that data. My first instinct was just to write a loop. Simple, right? But as soon as I ran it, I realized I was staring at a progress bar that moved with the speed of a tectonic plate.

The Wall of Sequential Execution

Here is how I started. I wrote a function to simulate a network request (which just sleeps) and a function to simulate a CPU-heavy calculation (which just runs a big loop). I ran them one after another.

import time

def fetch_data(id):
    print(f"Fetching {id}...")
    time.sleep(1)  # Simulating network lag
    return f"Data {id}"

def heavy_calc(id):
    print(f"Calculating {id}...")
    count = 0
    for i in range(10**7):  # Simulating CPU work
        count += i
    return count

# Doing this 4 times sequentially
start = time.time()
for i in range(4):
    fetch_data(i)
    heavy_calc(i)
print(f"Total time: {time.time() - start:.2f}s")

It took forever. Why? Because the CPU was sitting idle while waiting for the "network" (the sleep), and then the network was idle while the CPU was crunching numbers. I'm wasting time in both directions. I figured, "I'll just use threads. That's what threads are for, right?"

The GIL Letdown

I wrapped the calls in threading.Thread. I expected a massive speedup because I have a multi-core processor. I thought the CPU work would be split across cores and the I/O work would happen in the background.

import threading

threads = []
start = time.time()
for i in range(4):
    t = threading.Thread(target=lambda: (fetch_data(i), heavy_calc(i)))
    threads.append(t)
    t.start()

for t in threads:
    t.join()
print(f"Threaded time: {time.time() - start:.2f}s")

Here is where I hit the wall. The fetch_data parts happened almost simultaneously—great! But the heavy_calc parts? They didn't actually run in parallel. They took roughly the same amount of time as the sequential version.

This is because of the Global Interpreter Lock (GIL). In Python, the GIL ensures that only one thread executes Python bytecode at a time. For I/O (like time.sleep or reading a socket), the thread releases the GIL, which is why the fetching felt fast. But for raw CPU math, the threads were just fighting over the same lock, essentially taking turns. It's like having four chefs in a kitchen but only one knife.

Breaking the GIL with Processes

If I want to actually use my other CPU cores, I need a separate instance of the Python interpreter for each task. That means multiprocessing. I swapped threading.Thread for multiprocessing.Process and suddenly, the math started happening in parallel.

import multiprocessing

processes = []
start = time.time()
for i in range(4):
    p = multiprocessing.Process(target=lambda: (fetch_data(i), heavy_calc(i)))
    processes.append(p)
    p.start()

for p in processes:
    p.join()
print(f"Process time: {time.time() - start:.2f}s")

This was significantly faster for the calculations. But there's a catch: processes are "heavy." Each one has its own memory space. If I had to fetch 1,000 URLs, spinning up 1,000 processes would likely crash my RAM or spend more time managing processes than actually doing work. I need something lighter for the I/O part.

Scaling to Thousands of Requests

This is where coroutines come in via asyncio. Unlike threads (which the OS swaps in and out) or processes (which are separate programs), coroutines are "cooperative." They voluntarily yield control back to an event loop when they hit a waiting point.

I rewrote the I/O part using async and await. Notice that I removed the heavy_calc from here; if I put a CPU-heavy loop inside an async function, it would block the entire event loop, and everything would freeze.

import asyncio

async def fetch_data_async(id):
    print(f"Fetching {id}...")
    await asyncio.sleep(1)  # Non-blocking sleep
    return f"Data {id}"

async def main():
    tasks = [fetch_data_async(i) for i in range(100)] # 100 requests!
    await asyncio.gather(*tasks)

start = time.time()
asyncio.run(main())
print(f"Async time: {time.time() - start:.2f}s")

I just handled 100 "network" requests in about one second. If I had used processes or threads for 100 requests, the overhead would have been noticeable. With coroutines, it's almost free.

So, how do I choose? I've narrowed it down to this rule of thumb:

• Coroutines (asyncio): Use these for "waiting" tasks. Network requests, database queries, or reading files. High volume, low CPU.
• Processes (multiprocessing): Use these for "crunching" tasks. Data analysis, image processing, or heavy math. Low volume, high CPU.
• Threads (threading): Use these when you have I/O tasks but are using libraries that aren't async compatible. They are a middle ground, but the GIL makes them useless for CPU-heavy work.

📋 Practical Task

The Hybrid Performance Optimizer

You are tasked with building a script that mimics a real-world data pipeline. You need to handle two distinct types of workloads efficiently:

1. The I/O Bound Stage: Simulate fetching data from 50 different sources. Use asyncio` and await asyncio.sleep(0.1) to simulate this.
2. The CPU Bound Stage: Simulate processing that data by calculating the sum of squares for numbers up to 1,000,000 for each of the 50 results. This must be handled using multiprocessing to avoid the GIL.

Requirements:

• Your script should use asyncio.gather to fetch the "data" concurrently.
• Once the fetching is complete, use a multiprocessing.Pool to distribute the calculations across your CPU cores.
• Measure and print the total execution time for the entire pipeline.