Skip to Content
Course content

116: mpsc Channels In Depth

Click on the "Edit" button in the top corner of the screen to edit your slide content.

By now, you know that Rust's mpsc (multi-producer, single-consumer) channels are the go-to for sending data between threads. On the surface, it looks simple: you create a channel, clone the sender, and start pushing data. But there's a trap here that I've seen trip up plenty of experienced devs. The trap is the difference between an unbounded channel and a bounded one.

The danger of the infinite buffer

When you use mpsc::channel(), you're creating an unbounded channel. I like to think of this as a conveyor belt that can stretch infinitely. If your producer threads are faster than your consumer thread, the conveyor belt just keeps growing. In a small demo, this is invisible. In a production system—say, a log aggregator that processes thousands of events per second—it's a ticking time bomb.

use std::sync::mpsc;
use std::thread;
use std::time::Duration;

fn main() {
    // Naive: Unbounded channel
    let (tx, rx) = mpsc::channel();

    for i in 0..10 {
        let tx_clone = tx.clone();
        thread::spawn(move || {
            loop {
                tx_clone.send(format!("Log entry from thread {}", i)).unwrap();
                // We're hammering the channel as fast as possible
            }
        });
    }

    // The consumer is slow
    for received in rx {
        println!("Processing: {}", received);
        thread::sleep(Duration::from_millis(100)); 
    }
}

In this snippet, the producers are sprinting, but the consumer is strolling. Because mpsc::channel() doesn't have a limit, the internal buffer will grow until your OS decides it's had enough and kills the process with an Out-of-Memory (OOM) error. I've spent a few late nights debugging "random" crashes in distributed systems only to realize it was an unbounded channel eating all the RAM because a downstream service slowed down.

Introducing backpressure with sync_channel

The professional way to handle this is with mpsc::sync_channel(bound). This creates a bounded channel. Once the buffer reaches the bound limit, any thread attempting to send() will block (put to sleep) until the consumer removes an item from the queue. This is what we call "backpressure." It forces the producer to slow down to the speed of the consumer.

use std::sync::mpsc;
use std::thread;
use std::time::Duration;

fn main() {
    // Better: Bounded channel with a capacity of 10
    let (tx, rx) = mpsc::sync_channel(10);

    for i in 0..10 {
        let tx_clone = tx.clone();
        thread::spawn(move || {
            loop {
                // This will now block if the buffer is full
                if let Err(_) = tx_clone.send(format!("Log entry {}", i)) {
                    break; // Receiver is gone, stop producing
                }
            }
        });
    }

    // Drop the original sender so the receiver knows when all producers are done
    drop(tx);

    for received in rx {
        println!("Processing: {}", received);
        thread::sleep(Duration::from_millis(100));
    }
}

When the producer needs to keep moving

Now, you might ask: "What if I can't afford to block my producer threads?" In some UI threads or high-frequency networking code, blocking is a sin. In those cases, you don't use send(); you use try_send(). This method returns immediately. If the buffer is full, it returns a Full error instead of sleeping.

This shifts the responsibility to you. You have to decide: do you drop the data? Do you log a warning? Or do you retry after a brief pause? I usually prefer try_send() when I'm dealing with telemetry data—if the buffer is full, it's better to lose a few metrics points than to freeze the entire application.

The "Hanging Receiver" problem

One last thing I want to point out is the importance of dropping your senders. The rx loop (the for received in rx syntax) only terminates when all Sender handles are dropped. In my second example, I explicitly called drop(tx). If I hadn't, the main thread would still hold one copy of the sender, and the loop would hang forever, waiting for a message from itself that will never come. It's a subtle bug, but it's a classic in Rust concurrency.




📋 Practical Task

Exercise: Concurrent Image Metadata Extractor

You are building a tool that scans a directory for images and extracts their metadata. Because disk I/O and metadata parsing can be slow, you need to parallelize the work.

Requirements:

  • Create a struct ImageMetadata { filename: String, size: u64 }.
  • Use a sync_channel with a bound of 5 to send ImageMetadata from worker threads to a single aggregator thread.
  • Spawn 4 worker threads. Each worker should simulate "processing" 3 images by sending dummy ImageMetadata objects into the channel.
  • The aggregator thread must collect all metadata and print the total number of images processed.
  • Crucial: Ensure the program terminates gracefully. The aggregator loop must end once all workers have finished and all senders are dropped.
Rating
0 0

There are no comments for now.

to be the first to leave a comment.