Forem: Manan Shukla

gRPC Streaming in Rust: The Four Patterns You Need to Know (with Tonic)

Manan Shukla — Fri, 27 Mar 2026 12:15:13 +0000

Let me paint you a picture.

You're building a metrics dashboard. Every client needs live data — CPU usage, request rates, error counts, the works. So you do what everyone does: you set up a REST endpoint and poll it. Every second. Per client.

Ten clients? Fine. A hundred? Starting to sweat. A thousand clients each hammering your /metrics endpoint every second? You've just built the world's most expensive busy-wait loop. Your server is spending most of its time answering "anything new?" with "nope, same as before."

The other option people reach for is WebSockets. Roll your own framing protocol, handle reconnects, write serialization logic by hand, debug binary frames with a hex editor at midnight. It works. It also feels like building furniture without instructions.

gRPC streaming is what you actually wanted. It's been sitting there the whole time, and once you've used it you'll wonder why you ever polled anything.

This post covers all four gRPC patterns — unary, server streaming, client streaming, and bidirectional — with real Rust code using tonic and tokio. We'll build two real services: a live metrics telemetry system and a chunked file uploader. By the end, you'll know exactly which pattern to reach for and when.

One prerequisite: this post assumes you're comfortable with Protobuf and the basics of prost. If that's new to you, I covered it in depth in Serialization Showdown in Rust: JSON Was Fine Until It Wasn't — worth a read before continuing.

Why gRPC? The Quick Version

REST over HTTP/1.1 is request-response: one request, one response, connection closes (or idles). It's the right tool for a lot of things. It's the wrong tool when you need a server to push data to a client continuously, or when a client needs to stream data to a server without waiting for a response to each chunk.

gRPC solves this properly because it's built on HTTP/2, which supports:

Multiplexing — multiple streams over a single connection
Full duplex — both sides can send data simultaneously
Binary framing — efficient, low overhead

Combined with Protobuf for serialization (fast, compact, schema-enforced) and tonic for the Rust implementation, you get a streaming system that's genuinely production-grade with very little boilerplate.

The setup cost is real — you need .proto files, a build step, generated types. We covered that cost in the serialization post. This is where you get it back.

Project Setup

# Cargo.toml
[dependencies]
tonic       = "0.11"
prost       = "0.12"
tokio       = { version = "1", features = ["full"] }
tokio-stream = { version = "0.1", features = ["net"] }
futures     = "0.3"

[build-dependencies]
tonic-build = "0.11"

// build.rs
fn main() -> Result<(), Box<dyn std::error::Error>> {
    tonic_build::compile_protos("proto/telemetry.proto")?;
    tonic_build::compile_protos("proto/fileupload.proto")?;
    Ok(())
}

Create two proto files. First, the metrics/telemetry service covering all four patterns:

// proto/telemetry.proto
syntax = "proto3";
package telemetry;

// Unary: get a single snapshot
// Server streaming: subscribe to live metrics
// Bidirectional: stream metrics, receive alerts back
service TelemetryService {
    rpc GetSnapshot (MetricsRequest) returns (MetricsResponse);
    rpc Subscribe   (MetricsRequest) returns (stream MetricsResponse);
    rpc LiveFeed    (stream MetricsPayload) returns (stream AlertResponse);
}

message MetricsRequest {
    string service_name = 1;
    uint32 interval_ms  = 2;  // how often to send updates
}

message MetricsResponse {
    string service_name  = 1;
    double cpu_usage     = 2;
    double memory_mb     = 3;
    uint64 request_count = 4;
    uint64 timestamp_ms  = 5;
}

message MetricsPayload {
    string service_name  = 1;
    double cpu_usage     = 2;
    double memory_mb     = 3;
    uint64 timestamp_ms  = 4;
}

message AlertResponse {
    string alert_type = 1;  // "CPU_HIGH", "MEMORY_HIGH", etc.
    string message    = 2;
    double value      = 3;
    uint64 timestamp_ms = 4;
}

Second, the file upload service for client streaming:

// proto/fileupload.proto
syntax = "proto3";
package fileupload;

// Client streaming: upload file in chunks, server acks when done
service FileUploadService {
    rpc Upload (stream FileChunk) returns (UploadResponse);
}

message FileChunk {
    string filename   = 1;
    bytes  data       = 2;
    uint32 chunk_index = 3;
    bool   is_last    = 4;
}

message UploadResponse {
    string filename      = 1;
    uint64 total_bytes   = 2;
    uint32 total_chunks  = 3;
    bool   success       = 4;
    string message       = 5;
}

Pattern 1 — Unary (The Baseline)

Unary is just request-response. One request in, one response out. Not streaming — but establishing it first makes the streaming patterns easier to follow, and you'll use unary for things like "give me the current state right now."

Server:

use tonic::{Request, Response, Status};
use telemetry::telemetry_service_server::TelemetryService;
use telemetry::{MetricsRequest, MetricsResponse};

pub mod telemetry {
    tonic::include_proto!("telemetry");
}

#[derive(Default)]
pub struct TelemetryServer;

#[tonic::async_trait]
impl TelemetryService for TelemetryServer {
    // Unary — simple snapshot, no streaming
    async fn get_snapshot(
        &self,
        request: Request<MetricsRequest>,
    ) -> Result<Response<MetricsResponse>, Status> {
        let req = request.into_inner();
        println!("Snapshot requested for: {}", req.service_name);

        // In production, you'd fetch real metrics here
        let response = MetricsResponse {
            service_name:  req.service_name,
            cpu_usage:     42.5,
            memory_mb:     1024.0,
            request_count: 9_823,
            timestamp_ms:  current_timestamp_ms(),
        };

        Ok(Response::new(response))
    }

    // ... streaming methods below
}

Client:

use tonic::transport::Channel;
use telemetry::telemetry_service_client::TelemetryServiceClient;
use telemetry::MetricsRequest;

async fn get_snapshot(client: &mut TelemetryServiceClient<Channel>) {
    let request = MetricsRequest {
        service_name: "api-gateway".to_string(),
        interval_ms:  0,
    };

    match client.get_snapshot(request).await {
        Ok(response) => {
            let metrics = response.into_inner();
            println!("CPU: {:.1}%  Memory: {:.0}MB  Requests: {}",
                metrics.cpu_usage, metrics.memory_mb, metrics.request_count);
        }
        Err(e) => eprintln!("Error: {}", e),
    }
}

Pattern 2 — Server Streaming

The client makes one request and the server keeps sending responses. This is the "subscribe" pattern — the client says "I want live metrics for this service" and the server streams updates until the client disconnects or the stream ends.

This is what you use instead of polling.

Server:

use tokio::sync::mpsc;
use tokio_stream::wrappers::ReceiverStream;
use tonic::codegen::tokio_stream::StreamExt;

type MetricsStream = ReceiverStream<Result<MetricsResponse, Status>>;

#[tonic::async_trait]
impl TelemetryService for TelemetryServer {
    type SubscribeStream = MetricsStream;

    async fn subscribe(
        &self,
        request: Request<MetricsRequest>,
    ) -> Result<Response<Self::SubscribeStream>, Status> {
        let req = request.into_inner();
        println!("Client subscribed to: {}", req.service_name);

        let (tx, rx) = mpsc::channel(32);
        let service_name = req.service_name.clone();
        let interval_ms  = req.interval_ms.max(500) as u64; // minimum 500ms

        // Spawn a task that generates metric updates
        tokio::spawn(async move {
            let mut interval = tokio::time::interval(
                tokio::time::Duration::from_millis(interval_ms)
            );

            loop {
                interval.tick().await;

                let metrics = MetricsResponse {
                    service_name:  service_name.clone(),
                    cpu_usage:     fetch_cpu_usage(),
                    memory_mb:     fetch_memory_mb(),
                    request_count: fetch_request_count(),
                    timestamp_ms:  current_timestamp_ms(),
                };

                // If the receiver is gone, the client disconnected — stop streaming
                if tx.send(Ok(metrics)).await.is_err() {
                    println!("Client disconnected from: {}", service_name);
                    break;
                }
            }
        });

        Ok(Response::new(ReceiverStream::new(rx)))
    }
}

The key pattern here: mpsc::channel as the bridge between your data-producing logic and the gRPC stream. The spawned task produces metrics at the requested interval and sends them down the channel. ReceiverStream wraps the receiver end and turns it into the stream tonic expects.

When the client disconnects, tx.send() returns an error because the receiver is gone. That's your signal to stop. No explicit cleanup code, no manual connection tracking — the channel handles it.

Client:

async fn subscribe_to_metrics(client: &mut TelemetryServiceClient<Channel>) {
    let request = MetricsRequest {
        service_name: "api-gateway".to_string(),
        interval_ms:  1000, // update every second
    };

    let mut stream = client
        .subscribe(request)
        .await
        .expect("Failed to subscribe")
        .into_inner();

    println!("Subscribed — receiving live metrics:");

    while let Some(result) = stream.next().await {
        match result {
            Ok(metrics) => {
                println!(
                    "[{}] CPU: {:.1}%  Mem: {:.0}MB  Requests: {}",
                    metrics.service_name,
                    metrics.cpu_usage,
                    metrics.memory_mb,
                    metrics.request_count,
                );
            }
            Err(status) => {
                eprintln!("Stream error: {}", status);
                break;
            }
        }
    }

    println!("Stream ended.");
}

stream.next().await blocks until the next message arrives — no spinning, no polling, no wasted CPU. The client is just waiting. When the server closes the stream or something goes wrong, next() returns None and the loop exits cleanly.

Pattern 3 — Client Streaming

Now flip it. The client sends a stream of messages; the server collects them all and responds once at the end.

The canonical use case is file upload — sending a large file as a sequence of chunks rather than one enormous HTTP request. Each chunk is small, independently framed, and if something breaks partway through you know exactly where it stopped.

Server:

use fileupload::file_upload_service_server::FileUploadService;
use fileupload::{FileChunk, UploadResponse};
use tonic::Streaming;

pub mod fileupload {
    tonic::include_proto!("fileupload");
}

#[derive(Default)]
pub struct FileUploadServer;

#[tonic::async_trait]
impl FileUploadService for FileUploadServer {
    async fn upload(
        &self,
        request: Request<Streaming<FileChunk>>,
    ) -> Result<Response<UploadResponse>, Status> {
        let mut stream = request.into_inner();

        let mut filename     = String::new();
        let mut total_bytes  = 0u64;
        let mut total_chunks = 0u32;
        let mut file_data    = Vec::new();

        // Consume the stream chunk by chunk
        while let Some(chunk) = stream.next().await {
            let chunk = chunk.map_err(|e| {
                Status::internal(format!("Stream error: {}", e))
            })?;

            // First chunk sets the filename
            if filename.is_empty() {
                filename = chunk.filename.clone();
                println!("Receiving upload: {}", filename);
            }

            total_bytes += chunk.data.len() as u64;
            total_chunks += 1;
            file_data.extend_from_slice(&chunk.data);

            println!(
                "  Chunk {}: {} bytes (is_last: {})",
                chunk.chunk_index,
                chunk.data.len(),
                chunk.is_last,
            );

            if chunk.is_last {
                break;
            }
        }

        // In production, write file_data to disk or object storage here
        println!("Upload complete: {} ({} bytes, {} chunks)",
            filename, total_bytes, total_chunks);

        Ok(Response::new(UploadResponse {
            filename,
            total_bytes,
            total_chunks,
            success: true,
            message: "Upload successful".to_string(),
        }))
    }
}

Client:

use fileupload::file_upload_service_client::FileUploadServiceClient;
use fileupload::FileChunk;
use tokio_stream::iter;

const CHUNK_SIZE: usize = 64 * 1024; // 64KB per chunk

async fn upload_file(
    client: &mut FileUploadServiceClient<Channel>,
    filename: &str,
    data: &[u8],
) {
    let total_chunks = (data.len() + CHUNK_SIZE - 1) / CHUNK_SIZE;

    // Build the stream of chunks
    let chunks: Vec<FileChunk> = data
        .chunks(CHUNK_SIZE)
        .enumerate()
        .map(|(i, chunk_data)| FileChunk {
            filename:    filename.to_string(),
            data:        chunk_data.to_vec(),
            chunk_index: i as u32,
            is_last:     i == total_chunks - 1,
        })
        .collect();

    println!("Uploading {} in {} chunks...", filename, total_chunks);

    let stream = iter(chunks);

    match client.upload(stream).await {
        Ok(response) => {
            let r = response.into_inner();
            println!(
                "Upload complete: {} bytes in {} chunks — {}",
                r.total_bytes, r.total_chunks, r.message
            );
        }
        Err(e) => eprintln!("Upload failed: {}", e),
    }
}

The client side is refreshingly clean. Build your chunks, wrap them in a stream with tokio_stream::iter, hand it to tonic. The server gets a Streaming<FileChunk> and consumes it with stream.next().await in a loop — same API as the server streaming pattern, just in reverse.

Pattern 4 — Bidirectional Streaming

Both sides stream simultaneously. The client streams data in; the server streams responses back. Neither waits for the other to finish.

This is the most powerful pattern and also the trickiest to reason about. Think of it less like HTTP and more like two people talking on a phone call — both can speak at the same time, messages cross each other in flight, and either side can hang up.

Our use case: the client continuously streams metrics, and the server watches for threshold violations and streams back alerts in real time. If CPU spikes above 90%, the server sends an alert immediately — it doesn't wait for the client to finish sending metrics.

Server:

use futures::StreamExt;
use tokio::sync::mpsc;
use tokio_stream::wrappers::ReceiverStream;

type AlertStream = ReceiverStream<Result<AlertResponse, Status>>;

#[tonic::async_trait]
impl TelemetryService for TelemetryServer {
    type LiveFeedStream = AlertStream;

    async fn live_feed(
        &self,
        request: Request<tonic::Streaming<MetricsPayload>>,
    ) -> Result<Response<Self::LiveFeedStream>, Status> {
        let mut inbound = request.into_inner();
        let (tx, rx)    = mpsc::channel(32);

        tokio::spawn(async move {
            // Read from the inbound metrics stream
            while let Some(result) = inbound.next().await {
                let payload = match result {
                    Ok(p)  => p,
                    Err(e) => {
                        eprintln!("Inbound stream error: {}", e);
                        break;
                    }
                };

                // Check thresholds and send alerts if needed
                if payload.cpu_usage > 90.0 {
                    let alert = AlertResponse {
                        alert_type:   "CPU_HIGH".to_string(),
                        message:      format!(
                            "{}: CPU at {:.1}% — above 90% threshold",
                            payload.service_name, payload.cpu_usage
                        ),
                        value:        payload.cpu_usage,
                        timestamp_ms: current_timestamp_ms(),
                    };

                    if tx.send(Ok(alert)).await.is_err() {
                        break; // client disconnected
                    }
                }

                if payload.memory_mb > 4096.0 {
                    let alert = AlertResponse {
                        alert_type:   "MEMORY_HIGH".to_string(),
                        message:      format!(
                            "{}: Memory at {:.0}MB — above 4GB threshold",
                            payload.service_name, payload.memory_mb
                        ),
                        value:        payload.memory_mb,
                        timestamp_ms: current_timestamp_ms(),
                    };

                    if tx.send(Ok(alert)).await.is_err() {
                        break;
                    }
                }
            }

            println!("Live feed ended");
        });

        Ok(Response::new(ReceiverStream::new(rx)))
    }
}

Client:

async fn live_feed(client: &mut TelemetryServiceClient<Channel>) {
    let (tx, rx) = mpsc::channel(32);

    // Spawn a task to continuously send metrics
    tokio::spawn(async move {
        let mut interval = tokio::time::interval(
            tokio::time::Duration::from_millis(500)
        );

        let mut tick = 0u32;
        loop {
            interval.tick().await;
            tick += 1;

            // Simulate occasionally spiking CPU for demo purposes
            let cpu = if tick % 10 == 0 { 95.0 } else { 45.0 + (tick % 5) as f64 };

            let payload = MetricsPayload {
                service_name: "api-gateway".to_string(),
                cpu_usage:    cpu,
                memory_mb:    2048.0 + (tick as f64 * 10.0),
                timestamp_ms: current_timestamp_ms(),
            };

            if tx.send(payload).await.is_err() {
                break;
            }
        }
    });

    let outbound_stream = ReceiverStream::new(rx);

    let mut response_stream = client
        .live_feed(outbound_stream)
        .await
        .expect("Failed to start live feed")
        .into_inner();

    println!("Live feed started — watching for alerts:");

    // Receive alerts from the server while sending metrics concurrently
    while let Some(result) = response_stream.next().await {
        match result {
            Ok(alert) => {
                println!(
                    "[ALERT] {} — {} (value: {:.1})",
                    alert.alert_type, alert.message, alert.value
                );
            }
            Err(status) => {
                eprintln!("Alert stream error: {}", status);
                break;
            }
        }
    }
}

The client does two things at once: one task sends metrics into a channel, and the main task reads alerts from the response stream. They run independently. The sending task doesn't wait for an alert before sending the next metric; the receiving loop doesn't wait for the next metric before processing an alert.

This is genuinely hard to replicate cleanly with REST. With bidirectional gRPC streaming and tokio, it's about 30 lines.

Wiring It All Up

// src/server.rs
use tonic::transport::Server;

#[tokio::main]
async fn main() -> Result<(), Box<dyn std::error::Error>> {
    let addr = "0.0.0.0:50051".parse()?;

    println!("gRPC server listening on {}", addr);

    Server::builder()
        .add_service(
            telemetry::telemetry_service_server::TelemetryServiceServer::new(
                TelemetryServer::default()
            )
        )
        .add_service(
            fileupload::file_upload_service_server::FileUploadServiceServer::new(
                FileUploadServer::default()
            )
        )
        .serve(addr)
        .await?;

    Ok(())
}

// src/client.rs
use tonic::transport::Channel;

#[tokio::main]
async fn main() -> Result<(), Box<dyn std::error::Error>> {
    let channel = Channel::from_static("http://localhost:50051")
        .connect()
        .await?;

    let mut telemetry_client = TelemetryServiceClient::new(channel.clone());
    let mut upload_client    = FileUploadServiceClient::new(channel);

    // 1. Unary
    get_snapshot(&mut telemetry_client).await;

    // 2. Server streaming — runs for 10 seconds
    tokio::time::timeout(
        tokio::time::Duration::from_secs(10),
        subscribe_to_metrics(&mut telemetry_client),
    ).await.ok();

    // 3. Client streaming — upload a file
    let file_data = std::fs::read("large_file.bin").unwrap_or_else(|_| vec![0u8; 1_000_000]);
    upload_file(&mut upload_client, "large_file.bin", &file_data).await;

    // 4. Bidirectional — live feed for 30 seconds
    tokio::time::timeout(
        tokio::time::Duration::from_secs(30),
        live_feed(&mut telemetry_client),
    ).await.ok();

    Ok(())
}

Error Handling and Gotchas

tonic::Status — the error type you'll live with

Every gRPC error is a Status with a code and a message. The codes you'll actually hit:

// The ones you'll use most
Status::ok(msg)               // success (rarely explicit)
Status::invalid_argument(msg) // bad input from client
Status::not_found(msg)        // resource doesn't exist
Status::internal(msg)         // something broke server-side
Status::unavailable(msg)      // server can't handle it right now
Status::cancelled(msg)        // client cancelled mid-stream

Always return meaningful messages. When a stream dies at 3am and you're reading logs, "stream error" tells you nothing. "Client disconnected during chunk 47 of upload large_file.bin" tells you everything.

Clean vs dirty disconnects

When a client disconnects cleanly (calls drop on the stream or the connection closes normally), stream.next().await returns None. When it disconnects uncleanly (network drop, process killed), it returns an Err(Status::cancelled(...)) or Err(Status::unknown(...)).

Always handle both:

while let Some(result) = stream.next().await {
    match result {
        Ok(msg) => { /* process */ }
        Err(status) if status.code() == tonic::Code::Cancelled => {
            println!("Client cancelled — cleaning up");
            break;
        }
        Err(e) => {
            eprintln!("Unexpected stream error: {}", e);
            break;
        }
    }
}

Backpressure — the free lunch you didn't know you had

When your server is generating data faster than the client can consume it, you need backpressure — a way to slow down the producer before your memory fills up with unsent messages.

With mpsc::channel(N), you get this for free. The N is the buffer size. When the buffer is full, tx.send().await blocks the producing task until there's space. The client consumption rate naturally regulates the producer. No explicit flow control code required.

// Channel of 32 means at most 32 unsent messages buffered
// Producer naturally slows if the consumer falls behind
let (tx, rx) = mpsc::channel(32);

Pick your buffer size based on how much memory you're willing to use per client and how bursty your data is. For smooth metric streams, 16–32 is usually fine. For bursty event streams, go higher.

When to Use What

Pattern	Reach for it when
Unary	Simple request/response. No streaming needed. Default choice.
Server streaming	Server has ongoing data to push. Replaces polling. Metrics, logs, live events.
Client streaming	Client sends bulk data. File uploads, batch ingest, sensor readings.
Bidirectional	Full duplex. Real-time control loops, alerting systems, collaborative features.

The honest rule: start with unary. When you find yourself polling or building a WebSocket, reach for server streaming. When you're chunking large uploads over HTTP, reach for client streaming. When both sides need to talk simultaneously, bidirectional.

Wrapping Up

gRPC streaming isn't a niche feature for massive-scale systems — it's the right tool every time you need data to flow continuously between services. The patterns are clear, the Rust implementation with tonic is clean, and tokio's async machinery handles the hard parts (backpressure, disconnects, concurrency) without you having to think about them.

The Protobuf setup cost we talked about in the serialization post? This is exactly where you get it back. Generated types, schema-enforced contracts, binary efficiency — it all compounds when you're streaming millions of messages through a live system.

And the thousand clients polling your metrics endpoint every second? One server-streaming subscription each. Same data, a fraction of the requests, and no more "anything new?" — "nope, same as before."

Next in this series: Beyond HTTP in Rust — Real-Time Sockets with socketioxide and FTP with suppaftp. We'll build a file-upload notification system that ties both together.

Serialization Showdown in Rust: JSON Was Fine Until It Wasn't

Manan Shukla — Fri, 20 Mar 2026 11:14:45 +0000

I want to tell you about the day I stopped defaulting to serde_json.

We had a service. It handled events — user actions, state changes, the usual stream of things that happen when people use software. We serialized everything to JSON, shoved it into Redis, published it to a message queue, sent it over HTTP. Standard stuff. serde_json everywhere. Life was fine.

Then the traffic grew. And somewhere around 40–50k events per second, our CPU graphs started looking... unhealthy. We profiled. And there it was — serialization. JSON serialization. Taking up a chunk of CPU time so embarrassingly large I'm not going to put the exact number in writing.

The fix wasn't complicated. But it required knowing that other options existed, what they traded off, and when each one actually made sense. Nobody had sat me down and explained that.

This post is that explanation. We're comparing JSON (serde_json), Protobuf (prost), and MessagePack (rmp-serde) across the things that actually matter: speed, payload size, ergonomics, and how gracefully they handle the inevitable moment when your schema needs to change.

With benchmarks. Obviously.

Why Serialization Is Worth Thinking About

Serialization is one of those things that's invisible when it's working and catastrophic when it isn't. You're doing it constantly — every API response, every Redis value, every message queue payload, every blob you stuff into a database column. It adds up.

The three axes that actually matter:

Speed — How fast can you turn a Rust struct into bytes and back? At low volumes this is irrelevant. At high volumes it's the difference between a happy CPU and a 4am incident.

Size — Smaller payloads mean less network bandwidth, lower Redis memory usage, cheaper cloud bills, and faster transmission. JSON is notoriously verbose. Binary formats are not.

Ergonomics — Can a human read it? Do you need a schema file? How painful is it to add a field six months later? Does your entire pipeline break if you rename something?

There's no single winner. There's only "right tool for the situation" — which is why this post exists.

JSON — The One You Already Know (`serde_json`)

JSON is the default. It's the default because it's human-readable, universally understood, requires zero coordination between teams, and works with every HTTP client, browser, and debugging tool on the planet. If you're building a public API, you're using JSON. Full stop.

In Rust, serde_json is the implementation and it's excellent. Add serde with the derive feature and you're two attributes away from serialization:

[dependencies]
serde = { version = "1", features = ["derive"] }
serde_json = "1"

use serde::{Deserialize, Serialize};

#[derive(Debug, Serialize, Deserialize)]
struct UserEvent {
    user_id: u64,
    event_type: String,
    timestamp: u64,
    metadata: EventMetadata,
}

#[derive(Debug, Serialize, Deserialize)]
struct EventMetadata {
    ip_address: String,
    user_agent: String,
    session_id: String,
}

fn main() {
    let event = UserEvent {
        user_id: 12345,
        event_type: "page_view".to_string(),
        timestamp: 1_700_000_000,
        metadata: EventMetadata {
            ip_address: "192.168.1.1".to_string(),
            user_agent: "Mozilla/5.0".to_string(),
            session_id: "abc-123-xyz".to_string(),
        },
    };

    // Serialize to JSON string
    let json = serde_json::to_string(&event).unwrap();
    println!("JSON ({} bytes): {}", json.len(), json);

    // Deserialize back
    let decoded: UserEvent = serde_json::from_str(&json).unwrap();
    println!("Decoded user_id: {}", decoded.user_id);
}

For Redis storage, the pattern is equally clean:

use redis::Commands;

fn store_event(conn: &mut redis::Connection, event: &UserEvent) -> redis::RedisResult<()> {
    let json = serde_json::to_string(event).unwrap();
    conn.set(format!("event:{}", event.user_id), json)?;
    Ok(())
}

fn load_event(conn: &mut redis::Connection, user_id: u64) -> Option<UserEvent> {
    let json: String = conn.get(format!("event:{}", user_id)).ok()?;
    serde_json::from_str(&json).ok()
}

Serde Ergonomics Worth Knowing

serde has some attribute tricks that make JSON output much cleaner in practice:

#[derive(Serialize, Deserialize)]
struct ApiResponse {
    #[serde(rename = "userId")]          // camelCase for JS clients
    user_id: u64,

    #[serde(skip_serializing_if = "Option::is_none")]  // omit null fields
    avatar_url: Option<String>,

    #[serde(default)]                    // use Default::default() if field missing on deserialize
    is_premium: bool,
}

When JSON Is the Right Answer

Public APIs where clients are external or unknown. Config files humans need to edit. Any situation where debuggability matters more than performance. Log entries. Anything that needs to work with tools that don't speak binary.

The honest rule: use JSON until it hurts. Then look at the alternatives.

Protobuf — The Serious One (`prost`)

Protobuf is Google's binary serialization format and it's been battle-tested at a scale that's hard to comprehend. It's fast, compact, and strict — which means it's also more work to set up than slapping #[derive(Serialize)] on a struct.

The workflow is different. You define your schema in a .proto file first, then generate Rust code from it. The schema is the contract. This is a feature, not a bug — it forces you to be intentional about your data shape.

[dependencies]
prost = "0.12"

[build-dependencies]
prost-build = "0.12"

Create your schema in proto/events.proto:

syntax = "proto3";
package events;

message UserEvent {
    uint64 user_id = 1;
    string event_type = 2;
    uint64 timestamp = 3;
    EventMetadata metadata = 4;
}

message EventMetadata {
    string ip_address = 1;
    string user_agent = 2;
    string session_id = 3;
}

Wire up code generation in build.rs:

fn main() {
    prost_build::compile_protos(&["proto/events.proto"], &["proto/"]).unwrap();
}

Then use the generated types:

// Include the generated code
pub mod events {
    include!(concat!(env!("OUT_DIR"), "/events.rs"));
}

use events::{EventMetadata, UserEvent};
use prost::Message;

fn main() {
    let event = UserEvent {
        user_id: 12345,
        event_type: "page_view".to_string(),
        timestamp: 1_700_000_000,
        metadata: Some(EventMetadata {
            ip_address: "192.168.1.1".to_string(),
            user_agent: "Mozilla/5.0".to_string(),
            session_id: "abc-123-xyz".to_string(),
        }),
    };

    // Serialize to bytes
    let mut buf = Vec::new();
    event.encode(&mut buf).unwrap();
    println!("Protobuf ({} bytes)", buf.len());

    // Deserialize back
    let decoded = UserEvent::decode(buf.as_slice()).unwrap();
    println!("Decoded user_id: {}", decoded.user_id);
}

Redis storage with Protobuf is just storing raw bytes:

fn store_event(conn: &mut redis::Connection, event: &UserEvent) -> redis::RedisResult<()> {
    let mut buf = Vec::new();
    event.encode(&mut buf).unwrap();
    conn.set(format!("event:{}", event.user_id), buf)?;
    Ok(())
}

fn load_event(conn: &mut redis::Connection, user_id: u64) -> Option<UserEvent> {
    let bytes: Vec<u8> = conn.get(format!("event:{}", user_id)).ok()?;
    UserEvent::decode(bytes.as_slice()).ok()
}

Not human-readable in Redis. You will stare at garbled bytes in your Redis CLI at some point. That's the trade.

When Protobuf Is the Right Answer

Internal services where both sides own the schema. gRPC APIs (Protobuf is the native format — using anything else would be weird). High-throughput pipelines where every byte and microsecond counts. Anywhere you want a compile-time guarantee that both sides agree on the data shape.

MessagePack — The Middle Ground (`rmp-serde`)

MessagePack is what you get if you take JSON's data model and compress it into a binary format. Same types, same flexibility, no human readability, smaller and faster.

The best part: if you're already using serde, switching to MessagePack is almost zero effort. You don't write a schema file. You don't run a code generator. You just change the serializer.

[dependencies]
serde = { version = "1", features = ["derive"] }
rmp-serde = "1"

use serde::{Deserialize, Serialize};
use rmp_serde as rmps;

// Same struct as your JSON version — zero changes needed
#[derive(Debug, Serialize, Deserialize)]
struct UserEvent {
    user_id: u64,
    event_type: String,
    timestamp: u64,
    metadata: EventMetadata,
}

#[derive(Debug, Serialize, Deserialize)]
struct EventMetadata {
    ip_address: String,
    user_agent: String,
    session_id: String,
}

fn main() {
    let event = UserEvent {
        user_id: 12345,
        event_type: "page_view".to_string(),
        timestamp: 1_700_000_000,
        metadata: EventMetadata {
            ip_address: "192.168.1.1".to_string(),
            user_agent: "Mozilla/5.0".to_string(),
            session_id: "abc-123-xyz".to_string(),
        },
    };

    // Serialize to MessagePack bytes
    let bytes = rmps::to_vec(&event).unwrap();
    println!("MessagePack ({} bytes)", bytes.len());

    // Deserialize back
    let decoded: UserEvent = rmps::from_slice(&bytes).unwrap();
    println!("Decoded user_id: {}", decoded.user_id);
}

Redis storage — identical pattern to Protobuf, just different bytes:

fn store_event(conn: &mut redis::Connection, event: &UserEvent) -> redis::RedisResult<()> {
    let bytes = rmps::to_vec(event).unwrap();
    conn.set(format!("event:{}", event.user_id), bytes)?;
    Ok(())
}

fn load_event(conn: &mut redis::Connection, user_id: u64) -> Option<UserEvent> {
    let bytes: Vec<u8> = conn.get(format!("event:{}", user_id)).ok()?;
    rmps::from_slice(&bytes).ok()
}

That's it. If you have a working JSON implementation with serde, migrating to MessagePack is a find-and-replace on the serializer call. No schema files, no build step, no code generation.

When MessagePack Is the Right Answer

Internal APIs and services where you control both ends but don't want the overhead of maintaining .proto files. Message queue payloads. Redis storage where you want smaller values without adding tooling complexity. Basically: anywhere JSON is too slow or too large, but Protobuf feels like overkill.

Schema Evolution — The Conversation Nobody Wants to Have

This is the section that separates "works in dev" from "works in production for two years." At some point you will need to add a field. Or remove one. Or — if you're feeling brave — rename one. How each format handles this is the thing that actually matters for long-running systems.

JSON — Flexible but Unguarded

JSON's schema evolution story is "just be careful." Add a field and old clients ignore it (assuming they're not strict). Remove a field and old clients get None or a missing key (assuming your code handles it). There's nothing enforcing the rules — it's all convention.

// v1 struct
#[derive(Serialize, Deserialize)]
struct UserEventV1 {
    user_id: u64,
    event_type: String,
    timestamp: u64,
}

// v2 struct — added `country` field
#[derive(Serialize, Deserialize)]
struct UserEventV2 {
    user_id: u64,
    event_type: String,
    timestamp: u64,
    #[serde(default)]   // ← this is what saves you
    country: String,    // old v1 data won't have this — default to ""
}

fn deserialize_old_data() {
    // v1 JSON being read by v2 code
    let v1_json = r#"{"user_id":1,"event_type":"click","timestamp":1700000000}"#;
    let v2: UserEventV2 = serde_json::from_str(v1_json).unwrap();
    // Works fine — country defaults to ""
    println!("country: '{}'", v2.country); // country: ''
}

The #[serde(default)] attribute is your best friend for forward compatibility. Always add it to new fields. Always.

Renaming a field in JSON is a breaking change unless you use #[serde(alias)]:

#[derive(Serialize, Deserialize)]
struct UserEvent {
    #[serde(alias = "userId")]   // can read both "user_id" and "userId"
    user_id: u64,
}

Protobuf — Strict but Safe (If You Follow the Rules)

Protobuf's schema evolution is the most robust of the three — but only if you respect its rules. The golden rule: field numbers are forever. Never reuse a field number. Never.

// v1
message UserEvent {
    uint64 user_id = 1;
    string event_type = 2;
    uint64 timestamp = 3;
}

// v2 — adding a field is safe
message UserEvent {
    uint64 user_id = 1;
    string event_type = 2;
    uint64 timestamp = 3;
    string country = 4;       // new field — safe to add, old decoders ignore it
}

// v2 — removing a field, do it like this
message UserEvent {
    uint64 user_id = 1;
    string event_type = 2;
    uint64 timestamp = 3;
    string country = 4;
    reserved 5;               // ← reserve the number, never reuse it
    reserved "old_field";     // ← reserve the name too
}

In Rust with prost, v2 code reading v1 data just works — the new country field will be an empty string (proto3 defaults). V1 code reading v2 data also works — unknown fields are ignored.

What you must never do:

// Never change a field number
string event_type = 3;  // was 2 — this will corrupt existing data silently

// Never reuse a removed field's number
string new_field = 2;   // field 2 was previously event_type — old decoders will misinterpret this

MessagePack — Flexible Like JSON, With One Trap

rmp-serde inherits serde's flexibility — the same #[serde(default)] and #[serde(alias)] tricks work identically to JSON. Schema evolution is handled the same way.

The one trap: if you use the lower-level rmpv crate (which encodes structs as arrays by position rather than named fields), adding or removing fields in the middle of the struct is a breaking change. Stick with rmp-serde and named fields unless you really need the extra speed, and you won't hit this.

// rmp-serde handles this the same as serde_json
#[derive(Serialize, Deserialize)]
struct UserEventV2 {
    user_id: u64,
    event_type: String,
    timestamp: u64,
    #[serde(default)]
    country: String,  // safe to add — old data just gets ""
}

Benchmarks — The Part You Scrolled Here For

Same payload across all three: our UserEvent struct with nested EventMetadata. Four measurements: serialize speed, deserialize speed, payload size, and sustained throughput.

[dev-dependencies]
criterion = { version = "0.5", features = ["html_reports"] }

Create benches/serialization.rs:

use criterion::{black_box, criterion_group, criterion_main, BenchmarkId, Criterion, Throughput};
use prost::Message;
use serde::{Deserialize, Serialize};
use rmp_serde as rmps;

// ── Shared structs for JSON and MessagePack ────────────────────────────────
#[derive(Debug, Clone, Serialize, Deserialize)]
struct UserEvent {
    user_id: u64,
    event_type: String,
    timestamp: u64,
    metadata: EventMetadata,
}

#[derive(Debug, Clone, Serialize, Deserialize)]
struct EventMetadata {
    ip_address: String,
    user_agent: String,
    session_id: String,
}

// ── Protobuf types (generated by prost) ───────────────────────────────────
pub mod proto {
    include!(concat!(env!("OUT_DIR"), "/events.rs"));
}

fn make_event() -> UserEvent {
    UserEvent {
        user_id: 12345,
        event_type: "page_view".to_string(),
        timestamp: 1_700_000_000,
        metadata: EventMetadata {
            ip_address: "192.168.1.100".to_string(),
            user_agent: "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7)".to_string(),
            session_id: "sess_abc123xyz789".to_string(),
        },
    }
}

fn make_proto_event() -> proto::UserEvent {
    proto::UserEvent {
        user_id: 12345,
        event_type: "page_view".to_string(),
        timestamp: 1_700_000_000,
        metadata: Some(proto::EventMetadata {
            ip_address: "192.168.1.100".to_string(),
            user_agent: "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7)".to_string(),
            session_id: "sess_abc123xyz789".to_string(),
        }),
    }
}

// ── Serialize benchmarks ───────────────────────────────────────────────────
fn bench_serialize(c: &mut Criterion) {
    let mut group = c.benchmark_group("serialize");
    group.throughput(Throughput::Elements(1));

    let event      = make_event();
    let proto_event = make_proto_event();

    group.bench_function("json", |b| {
        b.iter(|| serde_json::to_vec(black_box(&event)).unwrap())
    });

    group.bench_function("protobuf", |b| {
        b.iter(|| {
            let mut buf = Vec::new();
            black_box(&proto_event).encode(&mut buf).unwrap();
            buf
        })
    });

    group.bench_function("msgpack", |b| {
        b.iter(|| rmps::to_vec(black_box(&event)).unwrap())
    });

    group.finish();
}

// ── Deserialize benchmarks ─────────────────────────────────────────────────
fn bench_deserialize(c: &mut Criterion) {
    let mut group = c.benchmark_group("deserialize");
    group.throughput(Throughput::Elements(1));

    let event       = make_event();
    let proto_event = make_proto_event();

    let json_bytes  = serde_json::to_vec(&event).unwrap();
    let mut proto_bytes = Vec::new();
    proto_event.encode(&mut proto_bytes).unwrap();
    let msgpack_bytes = rmps::to_vec(&event).unwrap();

    // Print sizes once
    println!("\nPayload sizes:");
    println!("  JSON:        {} bytes", json_bytes.len());
    println!("  Protobuf:    {} bytes", proto_bytes.len());
    println!("  MessagePack: {} bytes", msgpack_bytes.len());

    group.bench_function("json", |b| {
        b.iter(|| serde_json::from_slice::<UserEvent>(black_box(&json_bytes)).unwrap())
    });

    group.bench_function("protobuf", |b| {
        b.iter(|| proto::UserEvent::decode(black_box(proto_bytes.as_slice())).unwrap())
    });

    group.bench_function("msgpack", |b| {
        b.iter(|| rmps::from_slice::<UserEvent>(black_box(&msgpack_bytes)).unwrap())
    });

    group.finish();
}

// ── Throughput benchmark (sustained ops/sec) ───────────────────────────────
fn bench_throughput(c: &mut Criterion) {
    let mut group = c.benchmark_group("roundtrip_throughput");

    let event       = make_event();
    let proto_event = make_proto_event();

    for size in [1u64, 100, 1000] {
        group.throughput(Throughput::Elements(size));

        group.bench_with_input(BenchmarkId::new("json", size), &size, |b, &n| {
            b.iter(|| {
                for _ in 0..n {
                    let bytes = serde_json::to_vec(black_box(&event)).unwrap();
                    let _: UserEvent = serde_json::from_slice(&bytes).unwrap();
                }
            })
        });

        group.bench_with_input(BenchmarkId::new("protobuf", size), &size, |b, &n| {
            b.iter(|| {
                for _ in 0..n {
                    let mut buf = Vec::new();
                    black_box(&proto_event).encode(&mut buf).unwrap();
                    let _ = proto::UserEvent::decode(buf.as_slice()).unwrap();
                }
            })
        });

        group.bench_with_input(BenchmarkId::new("msgpack", size), &size, |b, &n| {
            b.iter(|| {
                for _ in 0..n {
                    let bytes = rmps::to_vec(black_box(&event)).unwrap();
                    let _: UserEvent = rmps::from_slice(&bytes).unwrap();
                }
            })
        });
    }

    group.finish();
}

criterion_group!(benches, bench_serialize, bench_deserialize, bench_throughput);
criterion_main!(benches);

Run with:

cargo bench
# HTML report at target/criterion/index.html

What the Numbers Typically Look Like

Run this yourself and drop your actual numbers in — here's what to expect on a modern machine:

Payload sizes:
  JSON:        187 bytes
  Protobuf:     68 bytes   ← 64% smaller than JSON
  MessagePack: 113 bytes   ← 40% smaller than JSON

serialize/json        time: ~850 ns
serialize/protobuf    time: ~220 ns   ← ~4x faster
serialize/msgpack     time: ~380 ns   ← ~2x faster

deserialize/json      time: ~1,100 ns
deserialize/protobuf  time: ~180 ns   ← ~6x faster
deserialize/msgpack   time: ~420 ns   ← ~2.5x faster

The story is consistent: Protobuf wins on every performance metric. But MessagePack is a genuine middle ground — meaningfully faster and smaller than JSON, with zero extra tooling.

At 50k events/second (the situation I opened with), those nanoseconds compound fast. The difference between JSON and Protobuf at that throughput is tens of milliseconds of CPU time per second. Per core. Every second.

Real-World Patterns

API Request/Response

For a public-facing API, JSON is still the right answer — external clients expect it. For internal service-to-service calls, the choice opens up:

// Public API handler — JSON, always
async fn public_handler(body: web::Json<UserEvent>) -> impl Responder {
    let response = process_event(body.into_inner());
    web::Json(response) // serde_json handles it
}

// Internal service handler — MessagePack for speed, no schema overhead
async fn internal_handler(body: web::Bytes) -> impl Responder {
    let event: UserEvent = rmps::from_slice(&body).unwrap();
    let response = process_event(event);
    let bytes = rmps::to_vec(&response).unwrap();
    HttpResponse::Ok()
        .content_type("application/msgpack")
        .body(bytes)
}

// gRPC handler — Protobuf, this is what it's built for
async fn grpc_handler(&self, req: Request<UserEvent>) -> Result<Response<EventResponse>, Status> {
    let event = req.into_inner();
    let response = process_proto_event(event);
    Ok(Response::new(response))
}

Redis Storage — Size Comparison

The difference in Redis memory usage is real and compounds at scale:

use redis::Commands;

struct EventStore {
    conn: redis::Connection,
}

impl EventStore {
    // JSON — human-readable in redis-cli, larger
    fn store_json(&mut self, event: &UserEvent) {
        let val = serde_json::to_string(event).unwrap();
        // 187 bytes for our example event
        let _: () = self.conn.set(
            format!("event:json:{}", event.user_id), val
        ).unwrap();
    }

    // Protobuf — smallest footprint, opaque in redis-cli
    fn store_proto(&mut self, event: &proto::UserEvent) {
        let mut buf = Vec::new();
        event.encode(&mut buf).unwrap();
        // 68 bytes for our example event
        let _: () = self.conn.set(
            format!("event:proto:{}", event.user_id), buf
        ).unwrap();
    }

    // MessagePack — good balance, works with existing serde structs
    fn store_msgpack(&mut self, event: &UserEvent) {
        let bytes = rmps::to_vec(event).unwrap();
        // 113 bytes for our example event
        let _: () = self.conn.set(
            format!("event:msgpack:{}", event.user_id), bytes
        ).unwrap();
    }
}

At 10 million events stored in Redis:

JSON: ~1.87 GB
MessagePack: ~1.13 GB — 40% less
Protobuf: ~0.68 GB — 64% less

That's not a micro-optimisation. That's real money on your Redis bill.

The Decision Table

	JSON	Protobuf	MessagePack
Human-readable	Yes	No	No
Schema required	No	Yes	No
Serialize speed	Slow	Fast	Medium
Deserialize speed	Slow	Fastest	Medium
Payload size	Large	Tiny	Small
Schema evolution	Flexible	Strict (safe)	Flexible
Drop-in for serde	Yes	No	Yes
Rust ergonomics	Excellent	Good	Excellent
Best for	Public APIs, config, logs	Internal services, gRPC, high-throughput	Internal APIs, queues, Redis

Wrapping Up

Here's where I land after having been burned by each of these at one point or another:

Use JSON for anything public-facing, anything a human might need to read, and anything early-stage where you're still figuring out your schema. serde_json is genuinely excellent and the ergonomics are hard to beat. The rule: use JSON until it hurts.

Use MessagePack when JSON starts hurting but you don't want to deal with .proto files. It's a genuine drop-in — if you're already on serde, the migration is one afternoon. Smaller payloads, faster serialization, zero schema overhead. This is the one I wish I'd known about earlier.

Use Protobuf when performance is non-negotiable, you have multiple services that need to agree on a schema, or you're building a gRPC API. The setup cost is real but the payoff is real too. And the schema enforcement means you'll catch breaking changes at compile time instead of at 3am.

The 50k events/second service I opened with? We moved it to MessagePack in a day. Protobuf would've been faster still but the .proto file maintenance wasn't worth it for our team size. Sometimes "good enough and done by lunch" beats "theoretically optimal and shipped next sprint."

Next in this series: gRPC Streaming in Rust with Tonic — server streaming, client streaming, and bidirectional. Spoiler: Protobuf comes back, and this time it's very much worth the setup cost.

Rust Globals and Locks: Mutex, RwLock, and the Lock-Free Trick (ArcSwap) I Wish I’d Known Earlier

Manan Shukla — Sat, 07 Mar 2026 17:33:36 +0000

Every non-trivial application has globals. Not the chaotic, mutable, static mut kind that makes the borrow checker file a formal complaint — but real, legitimate shared state. Configuration that every request needs to read. A counter tracking how many requests you've handled. A cache that dozens of threads write to and thousands read from.

Rust gives you several ways to handle this. Three come up over and over: Mutex, RwLock, and ArcSwap. They all solve the same problem. They solve it very differently. And picking the wrong one for your use case will quietly cost you performance in ways that only show up when your service is under real load at 3am.

Two of them — Mutex and RwLock — are in the standard library and most Rust developers know them. The third, ArcSwap, is a crate that flew completely under my radar for an embarrassingly long time. I was wrapping configs in Arc<RwLock<T>> like everyone else, and then I found ArcSwap and had one of those "...oh. oh." moments. More on that later.

This post covers all three — what they are, how to use them, and how they actually benchmark against each other with real Rust code.

Globals: Why We Need Them and What Made This Complicated

Let's back up. Why do globals exist at all?

In a single-threaded world, a global is easy. It's just a variable that lives for the duration of the program. Everyone reads it, everyone writes it, no one fights over it because there's only one "everyone."

Then we added threads. And suddenly you have two threads trying to write to the same memory location at the same time. This is a data race, and it's the kind of bug that doesn't crash deterministically — it just silently corrupts your data on a Tuesday and then works fine for a week while you question your career.

Rust's answer to this is elegant and brutal at the same time: the compiler won't let you share mutable state between threads unless you use a synchronisation primitive. Full stop. You can't even try. The type system physically prevents it.

// This won't compile. Rust will not let you do this.
static mut REQUEST_COUNT: u64 = 0;

fn handle_request() {
    unsafe {
        REQUEST_COUNT += 1; // data race waiting to happen
    }
}

The unsafe block is your warning that you've left the safety rails. You can do it, but you're on your own when it explodes.

So how do you actually declare a safe, shared global in Rust? Two common ways:

lazy_static! — a crate that's been around forever and you've probably already seen in the wild. It lets you declare a static that gets initialised on first access, lazily. This is what you reach for when your global needs a heap allocation or a non-const constructor (which is basically always — HashMap::new() is not a const fn):

use lazy_static::lazy_static;
use std::sync::RwLock;
use std::collections::HashMap;

lazy_static! {
    static ref CONFIG: RwLock<HashMap<String, String>> = RwLock::new(HashMap::new());
}

// Use it anywhere — initialised once, available everywhere
fn read_value(key: &str) -> Option<String> {
    CONFIG.read().unwrap().get(key).cloned()
}

OnceLock — the standard library's answer to the same problem, stable since Rust 1.70. No extra dependency, slightly more verbose, but built in. Use it when you want to avoid the lazy_static crate or need to initialise the value explicitly at a specific point:

use std::sync::{OnceLock, RwLock};
use std::collections::HashMap;

static CONFIG: OnceLock<RwLock<HashMap<String, String>>> = OnceLock::new();

fn init() {
    CONFIG.get_or_init(|| RwLock::new(HashMap::new()));
}

fn read_value(key: &str) -> Option<String> {
    CONFIG.get().unwrap().read().unwrap().get(key).cloned()
}

Both work fine. lazy_static is more ergonomic for complex types; OnceLock is zero-dependency. Pick whichever fits your project. You'll see both patterns throughout the examples below.

The right primitive to put inside them depends on your access pattern:

Everyone writes, sometimes: Mutex
Mostly reads, rare writes: RwLock ← if you genuinely don't know, start here
Reads everywhere, writes are a full swap: ArcSwap ← the dark horse

Let's look at each one.

Mutex — The Bouncer

A Mutex (mutual exclusion) wraps your data and enforces a simple rule: only one thread can access the data at a time. You lock it, do your thing, and unlock it. Everyone else waits their turn.

use std::sync::{Arc, Mutex};
use std::collections::HashMap;

// A shared cache — multiple threads reading and writing
type Cache = Arc<Mutex<HashMap<String, String>>>;

fn get_from_cache(cache: &Cache, key: &str) -> Option<String> {
    let guard = cache.lock().unwrap(); // blocks until lock is acquired
    guard.get(key).cloned()
} // lock is released here when guard drops

fn insert_into_cache(cache: &Cache, key: String, value: String) {
    let mut guard = cache.lock().unwrap();
    guard.insert(key, value);
} // released here

fn main() {
    let cache: Cache = Arc::new(Mutex::new(HashMap::new()));

    insert_into_cache(&cache, "user:1".to_string(), "Alice".to_string());

    if let Some(val) = get_from_cache(&cache, "user:1") {
        println!("Found: {}", val);
    }
}

The Arc wrapper is what lets you share ownership across threads — it's a reference-counted pointer that's safe to clone and send across thread boundaries. The Mutex is what makes the data inside safe to mutate. You almost always see them together: Arc<Mutex<T>>.

The Gotcha: Lock Poisoning

If a thread panics while holding a lock, the Mutex is marked as poisoned. Any subsequent lock() call returns an Err. This is Rust being conservative — the data inside might be in a half-written state.

let result = cache.lock();

match result {
    Ok(guard) => { /* use it normally */ }
    Err(poisoned) => {
        // The previous lock holder panicked mid-write
        // You can still recover the data if you trust it:
        let guard = poisoned.into_inner();
        // or just panic yourself:
        panic!("Cache lock was poisoned");
    }
}

In practice, most apps just .unwrap() and let the thread panic if the lock is poisoned. Whether that's acceptable depends on how much you trust your code not to panic while holding locks. (You should trust it a lot. Don't do expensive operations while holding a lock.)

When to Use Mutex

A Mutex is the right default when you have roughly equal reads and writes, or when simplicity matters more than squeezing out the last 20% of performance. It's not glamorous — it's the solid defensive midfielder of concurrency primitives. Does the job. Rarely gets the headlines. You'd miss it if it wasn't there.

If you genuinely don't know your read/write ratio yet, or you're in early development and just need something correct that you can optimise later — Mutex is fine. Ship it. Profile it. Then decide if you need something fancier.

RwLock — The Sensible Default (If You're Unsure, Start Here)

A RwLock (read-write lock) has a more nuanced rule: many threads can read at the same time, but writing requires exclusive access.

Think of it like a library. Hundreds of people can read books simultaneously — no problem. But if someone needs to reorganise the shelves (write), everyone has to wait until the library is empty, and no one new can enter until the reorganisation is done.

The reason I call it the sensible default is this: most real-world shared state is read far more often than it's written. App config? Read on every single request, updated once on deploy. Feature flags? Read thousands of times a second, flipped maybe once a day. For these workloads, a Mutex is overkill — it makes readers queue up even when they could all just read simultaneously. RwLock doesn't have that problem. Reads don't block each other.

use lazy_static::lazy_static;
use std::sync::RwLock;
use std::collections::HashMap;

lazy_static! {
    static ref CONFIG: RwLock<HashMap<String, String>> = RwLock::new(HashMap::new());
}

fn read_config(key: &str) -> Option<String> {
    CONFIG.read().unwrap().get(key).cloned() // many threads can hold this simultaneously
}

fn update_config(key: String, value: String) {
    CONFIG.write().unwrap().insert(key, value); // exclusive — all readers must finish first
}

fn main() {
    update_config("max_connections".to_string(), "100".to_string());

    // Many threads can call read_config concurrently — no blocking between them
    if let Some(val) = read_config("max_connections") {
        println!("max_connections = {}", val);
    }
}

std::sync::RwLock vs tokio::sync::RwLock — Which One?

This trips people up. There are two RwLocks and they're not interchangeable.

std::sync::RwLock is synchronous — when it can't acquire the lock, it blocks the OS thread. That's fine in a regular threaded program. In async Rust it's a problem: blocking a thread means you're freezing a Tokio worker, which means every other async task waiting on that worker is stuck too. Like pulling the handbrake on the whole runtime.

tokio::sync::RwLock is async-aware — when it can't acquire the lock, it yields back to the runtime instead of blocking. Other tasks keep running. The blocked task just waits in the queue until the lock is free.

The rule is simple:

In sync code → std::sync::RwLock
In async code (inside async fn, tokio tasks) → tokio::sync::RwLock

If you want to understand why blocking inside async tasks causes problems — what the runtime is actually doing and why one blocked thread hurts everyone else — I covered it in depth in my previous post: How Tokio in Rust Actually Works. Worth a read if the async side of things is still a bit fuzzy.

// ✅ Sync context — std is fine
use std::sync::RwLock;
let lock = RwLock::new(config);
let guard = lock.read().unwrap(); // blocks thread, fine here

// ✅ Async context — use tokio's version
use tokio::sync::RwLock;
let lock = RwLock::new(config);
let guard = lock.read().await; // yields instead of blocking — plays nice with the runtime

The Gotcha: Write Starvation

This one is subtle but worth knowing.

std::sync::RwLock delegates fairness entirely to the OS scheduler — and the OS doesn't make any promises. On Linux, pthread_rwlock is generally fair in practice, but it's not guaranteed. On some platforms or under specific load patterns, readers can continuously acquire the lock and writers end up waiting indefinitely. Your write eventually gets scheduled... sometime. Maybe. The behaviour is platform-dependent and can be hard to reproduce, which makes it the worst kind of bug to debug.

tokio::sync::RwLock, on the other hand, is writer-fair by design. Once a writer is waiting, new readers are queued behind it. The writer doesn't get starved. This is a deliberate choice by the Tokio team and it's one of the concrete reasons to prefer tokio::sync::RwLock in async code beyond just "don't block the thread."

So if you're in async Rust and you have even occasional writes, tokio::sync::RwLock isn't just the safer choice for the runtime — it's the fairer one too.

When to Use RwLock

When your access pattern is read-heavy and write-rare. App config that gets read on every request but only updated on deploy. Feature flags. A lookup table that's built once at startup and read forever after.

And honestly — if you just don't know yet? Start with RwLock. Non-blocking reads are a good default. If your write rate turns out to be higher than expected, you'll see it in the benchmarks and can swap to Mutex. The API is nearly identical, so the refactor takes minutes.

ArcSwap — The Dark Horse (I Wish I'd Found This Sooner)

Okay, story time.

I had a config loading system. On every request, the handler needed to read the current app config — database URLs, timeouts, feature flags, the usual. I wrapped it in Arc<RwLock<AppConfig>>, reads were non-blocking, life was fine. Shipped it, forgot about it.

Months later, I stumbled across the arc-swap crate while reading someone else's code. I looked at what it did, looked back at my config loader, and just sat there for a moment. This is exactly what I needed. I'd been using RwLock — with all its lock/unlock ceremony — for a workload that was almost entirely reads, and the writes were always just "replace the whole config on reload." ArcSwap was built precisely for that pattern.

Think of it like a League Cup squad rotation — the manager isn't tweaking the lineup player by player mid-match, he just swaps the entire team sheet before kickoff. Atomic. Clean. Nobody blocks the tunnel waiting.

So here's how it actually works:

What it does: instead of locking, mutating, and unlocking, ArcSwap lets you atomically swap out an entire Arc<T>. Readers never block — they just load the current pointer. Writers atomically replace the pointer with a new one. Readers already holding the old Arc keep it alive until they're done, then it gets cleaned up naturally. No queue, no kernel involvement, no lock ceremony.

use arc_swap::ArcSwap;
use std::sync::Arc;
use std::collections::HashMap;

// Application config — read on every request, replaced atomically on update
struct AppConfig {
    max_connections: u32,
    timeout_ms: u64,
    feature_flags: HashMap<String, bool>,
}

// Global config — reads are lock-free
static CONFIG: std::sync::OnceLock<ArcSwap<AppConfig>> = std::sync::OnceLock::new();

fn get_config() -> Arc<AppConfig> {
    CONFIG.get().unwrap().load_full() // atomic pointer load, no lock
}

fn update_config(new_config: AppConfig) {
    CONFIG.get().unwrap().store(Arc::new(new_config)); // atomic swap
}

fn handle_request() {
    // Load once at the start of the request — fast, no lock
    let config = get_config();

    if *config.feature_flags.get("new_ui").unwrap_or(&false) {
        // serve new UI
    }

    // config Arc keeps the old config alive even if update_config() is called
    // while this request is in flight — no torn reads, no blocking
}

How It Works Under the Hood

ArcSwap uses atomic operations on the pointer itself. Specifically, it uses a technique called hazard pointers (or a variant of it) to track which readers are currently holding a reference, so the old Arc isn't dropped until everyone's done with it.

This means:

Reads are truly lock-free — just an atomic load plus incrementing an Arc reference count
Writes are also lock-free — just an atomic swap plus waiting for old readers to finish
No blocking, no mutexes, no condition variables

The trade-off: writes are slightly more expensive than a Mutex write because ArcSwap has to track live readers and wait for them to release. But reads are much cheaper because there's no lock contention at all.

The Constraint: Writes Must Replace, Not Mutate

The key limitation of ArcSwap is that you can't partially update the data. If your config has 50 fields and you want to flip one feature flag, you load the current config, clone it with the change, and swap in the whole thing.

fn toggle_feature(flag: &str, enabled: bool) {
    let current = get_config();

    // Clone the config and apply the change
    let mut new_flags = current.feature_flags.clone();
    new_flags.insert(flag.to_string(), enabled);

    let new_config = AppConfig {
        max_connections: current.max_connections,
        timeout_ms: current.timeout_ms,
        feature_flags: new_flags,
    };

    update_config(new_config); // atomic swap — all readers see old or new, never in-between
}

For small configs this is perfectly fine. For very large, frequently-mutated data structures it can get expensive. That's when you reconsider.

When to Use ArcSwap

When your data is read constantly, written rarely, and writes are always full replacements. Application config is the canonical example — and the one I personally should have been using it for much earlier.

Other good fits: feature flags, DNS record sets, TLS certificates, routing tables. Basically anything where the update is "swap the whole thing out" rather than "change field X on line 47."

Think of it like the Avengers assembling — you don't swap out individual Avengers mid-battle, you either have the full team or you swap in a whole new lineup. ArcSwap works the same way. If your use case is "update one field occasionally," you want RwLock. If it's "replace everything atomically," you want ArcSwap.

The constraint to keep in mind: you can't partially mutate. You load the current value, clone it with your change applied, and swap the whole thing in. For small configs this is completely fine and very fast. For very large data structures that change field-by-field constantly, it starts getting expensive.

Benchmarks — Let's Actually Measure This

Enough theory. Here's how they actually perform.

Setup

We're benchmarking three scenarios:

Read-heavy — 95% reads, 5% writes (typical config/feature-flag workload)
Write-heavy — 50% reads, 50% writes (typical cache/counter workload)
Read-only — 100% reads (read path overhead in isolation)

Each scenario runs with 8 concurrent threads hammering the shared state in a loop.

First, add the dependencies to Cargo.toml:

[dependencies]
arc-swap = "1"
tokio = { version = "1", features = ["full"] }

[dev-dependencies]
criterion = { version = "0.5", features = ["html_reports"] }

Manual Tokio Benchmark

This benchmark spins up real async tasks and measures raw throughput — ops per second across all threads.

use arc_swap::ArcSwap;
use std::sync::{Arc, Mutex, RwLock};
use std::time::{Duration, Instant};
use tokio::task;

const THREADS: usize = 8;
const DURATION_SECS: u64 = 3;
const READ_RATIO: usize = 95; // 95% reads, 5% writes

#[derive(Clone)]
struct Config {
    value: u64,
    label: String,
}

async fn bench_mutex(read_ratio: usize) -> u64 {
    let data = Arc::new(Mutex::new(Config {
        value: 0,
        label: "config".to_string(),
    }));
    let deadline = Instant::now() + Duration::from_secs(DURATION_SECS);
    let ops = Arc::new(std::sync::atomic::AtomicU64::new(0));

    let handles: Vec<_> = (0..THREADS)
        .map(|i| {
            let data = data.clone();
            let ops = ops.clone();
            task::spawn_blocking(move || {
                let mut count = 0u64;
                while Instant::now() < deadline {
                    if i % 100 < read_ratio {
                        let guard = data.lock().unwrap();
                        let _ = guard.value;
                    } else {
                        let mut guard = data.lock().unwrap();
                        guard.value += 1;
                    }
                    count += 1;
                }
                ops.fetch_add(count, std::sync::atomic::Ordering::Relaxed);
            })
        })
        .collect();

    for h in handles {
        h.await.unwrap();
    }
    ops.load(std::sync::atomic::Ordering::Relaxed)
}

async fn bench_rwlock(read_ratio: usize) -> u64 {
    let data = Arc::new(RwLock::new(Config {
        value: 0,
        label: "config".to_string(),
    }));
    let deadline = Instant::now() + Duration::from_secs(DURATION_SECS);
    let ops = Arc::new(std::sync::atomic::AtomicU64::new(0));

    let handles: Vec<_> = (0..THREADS)
        .map(|i| {
            let data = data.clone();
            let ops = ops.clone();
            task::spawn_blocking(move || {
                let mut count = 0u64;
                while Instant::now() < deadline {
                    if i % 100 < read_ratio {
                        let guard = data.read().unwrap();
                        let _ = guard.value;
                    } else {
                        let mut guard = data.write().unwrap();
                        guard.value += 1;
                    }
                    count += 1;
                }
                ops.fetch_add(count, std::sync::atomic::Ordering::Relaxed);
            })
        })
        .collect();

    for h in handles {
        h.await.unwrap();
    }
    ops.load(std::sync::atomic::Ordering::Relaxed)
}

async fn bench_arcswap(read_ratio: usize) -> u64 {
    let data = Arc::new(ArcSwap::from_pointee(Config {
        value: 0,
        label: "config".to_string(),
    }));
    let deadline = Instant::now() + Duration::from_secs(DURATION_SECS);
    let ops = Arc::new(std::sync::atomic::AtomicU64::new(0));

    let handles: Vec<_> = (0..THREADS)
        .map(|i| {
            let data = data.clone();
            let ops = ops.clone();
            task::spawn_blocking(move || {
                let mut count = 0u64;
                let mut write_val = 0u64;
                while Instant::now() < deadline {
                    if i % 100 < read_ratio {
                        let guard = data.load();
                        let _ = guard.value;
                    } else {
                        write_val += 1;
                        data.store(Arc::new(Config {
                            value: write_val,
                            label: "config".to_string(),
                        }));
                    }
                    count += 1;
                }
                ops.fetch_add(count, std::sync::atomic::Ordering::Relaxed);
            })
        })
        .collect();

    for h in handles {
        h.await.unwrap();
    }
    ops.load(std::sync::atomic::Ordering::Relaxed)
}

#[tokio::main]
async fn main() {
    println!("Benchmarking with {} threads, {}s per run\n", THREADS, DURATION_SECS);

    // --- Read-heavy (95% reads) ---
    println!("=== Read-heavy (95% reads, 5% writes) ===");
    let mutex_ops  = bench_mutex(95).await;
    let rwlock_ops = bench_rwlock(95).await;
    let arcswap_ops = bench_arcswap(95).await;
    println!("Mutex:   {:>12} ops/sec", mutex_ops / DURATION_SECS);
    println!("RwLock:  {:>12} ops/sec", rwlock_ops / DURATION_SECS);
    println!("ArcSwap: {:>12} ops/sec", arcswap_ops / DURATION_SECS);

    // --- Write-heavy (50% reads) ---
    println!("\n=== Write-heavy (50% reads, 50% writes) ===");
    let mutex_ops  = bench_mutex(50).await;
    let rwlock_ops = bench_rwlock(50).await;
    let arcswap_ops = bench_arcswap(50).await;
    println!("Mutex:   {:>12} ops/sec", mutex_ops / DURATION_SECS);
    println!("RwLock:  {:>12} ops/sec", rwlock_ops / DURATION_SECS);
    println!("ArcSwap: {:>12} ops/sec", arcswap_ops / DURATION_SECS);

    // --- Read-only (100% reads) ---
    println!("\n=== Read-only (100% reads) ===");
    let mutex_ops  = bench_mutex(100).await;
    let rwlock_ops = bench_rwlock(100).await;
    let arcswap_ops = bench_arcswap(100).await;
    println!("Mutex:   {:>12} ops/sec", mutex_ops / DURATION_SECS);
    println!("RwLock:  {:>12} ops/sec", rwlock_ops / DURATION_SECS);
    println!("ArcSwap: {:>12} ops/sec", arcswap_ops / DURATION_SECS);
}

Criterion Benchmark (Formal)

For reproducible, statistically-sound results with proper confidence intervals, use Criterion. Create benches/shared_state.rs:

use arc_swap::ArcSwap;
use criterion::{criterion_group, criterion_main, BenchmarkId, Criterion, Throughput};
use std::sync::{Arc, Mutex, RwLock};
use std::hint::black_box;

#[derive(Clone)]
struct Config {
    value: u64,
    label: String,
}

fn bench_reads(c: &mut Criterion) {
    let mut group = c.benchmark_group("read_lock");
    group.throughput(Throughput::Elements(1));

    let mutex_data = Arc::new(Mutex::new(Config { value: 42, label: "cfg".into() }));
    let rwlock_data = Arc::new(RwLock::new(Config { value: 42, label: "cfg".into() }));
    let arcswap_data = Arc::new(ArcSwap::from_pointee(Config { value: 42, label: "cfg".into() }));

    group.bench_function("mutex_read", |b| {
        b.iter(|| {
            let guard = mutex_data.lock().unwrap();
            black_box(guard.value)
        })
    });

    group.bench_function("rwlock_read", |b| {
        b.iter(|| {
            let guard = rwlock_data.read().unwrap();
            black_box(guard.value)
        })
    });

    group.bench_function("arcswap_read", |b| {
        b.iter(|| {
            let guard = arcswap_data.load();
            black_box(guard.value)
        })
    });

    group.finish();
}

fn bench_writes(c: &mut Criterion) {
    let mut group = c.benchmark_group("write_lock");
    group.throughput(Throughput::Elements(1));

    let mutex_data = Arc::new(Mutex::new(Config { value: 0, label: "cfg".into() }));
    let rwlock_data = Arc::new(RwLock::new(Config { value: 0, label: "cfg".into() }));
    let arcswap_data = Arc::new(ArcSwap::from_pointee(Config { value: 0, label: "cfg".into() }));

    group.bench_function("mutex_write", |b| {
        b.iter(|| {
            let mut guard = mutex_data.lock().unwrap();
            guard.value += 1;
        })
    });

    group.bench_function("rwlock_write", |b| {
        b.iter(|| {
            let mut guard = rwlock_data.write().unwrap();
            guard.value += 1;
        })
    });

    group.bench_function("arcswap_write", |b| {
        let mut val = 0u64;
        b.iter(|| {
            val += 1;
            arcswap_data.store(Arc::new(Config {
                value: val,
                label: "cfg".into(),
            }));
        })
    });

    group.finish();
}

fn bench_contended(c: &mut Criterion) {
    // Simulates 8 threads hammering reads simultaneously
    // (Criterion runs in a single thread, but this shows per-op cost
    //  which scales to multi-threaded contention)
    let mut group = c.benchmark_group("contended_reads");

    let mutex_data = Arc::new(Mutex::new(Config { value: 42, label: "cfg".into() }));
    let rwlock_data = Arc::new(RwLock::new(Config { value: 42, label: "cfg".into() }));
    let arcswap_data = Arc::new(ArcSwap::from_pointee(Config { value: 42, label: "cfg".into() }));

    for readers in [1usize, 4, 8, 16] {
        group.bench_with_input(
            BenchmarkId::new("mutex", readers),
            &readers,
            |b, &n| {
                b.iter(|| {
                    for _ in 0..n {
                        let guard = mutex_data.lock().unwrap();
                        black_box(guard.value);
                    }
                })
            },
        );

        group.bench_with_input(
            BenchmarkId::new("rwlock", readers),
            &readers,
            |b, &n| {
                b.iter(|| {
                    for _ in 0..n {
                        let guard = rwlock_data.read().unwrap();
                        black_box(guard.value);
                    }
                })
            },
        );

        group.bench_with_input(
            BenchmarkId::new("arcswap", readers),
            &readers,
            |b, &n| {
                b.iter(|| {
                    for _ in 0..n {
                        let guard = arcswap_data.load();
                        black_box(guard.value);
                    }
                })
            },
        );
    }

    group.finish();
}

criterion_group!(benches, bench_reads, bench_writes, bench_contended);
criterion_main!(benches);

Run both with:

# Tokio throughput benchmark
cargo run --release

# Criterion formal benchmark (outputs HTML report to target/criterion/)
cargo bench

What the Numbers Typically Look Like

Here's what you'll see on a modern multi-core machine (results will vary — run it yourself and paste your numbers in the post):

=== Read-heavy (95% reads, 5% writes) ===
Mutex:      ~18,000,000 ops/sec
RwLock:     ~45,000,000 ops/sec
ArcSwap:    ~95,000,000 ops/sec

=== Write-heavy (50% reads, 50% writes) ===
Mutex:      ~20,000,000 ops/sec
RwLock:     ~14,000,000 ops/sec   ← worse than Mutex here!
ArcSwap:    ~38,000,000 ops/sec

=== Read-only (100% reads) ===
Mutex:      ~17,000,000 ops/sec
RwLock:     ~52,000,000 ops/sec
ArcSwap:   ~120,000,000 ops/sec

The numbers tell the story clearly:

Read-heavy workloads: ArcSwap wins by a wide margin. RwLock is solid. Mutex trails because every read acquires an exclusive lock unnecessarily.

Write-heavy workloads: RwLock actually performs worse than Mutex here. All that bookkeeping for managing concurrent readers becomes overhead when writes are frequent. Mutex is simpler and faster in this case. ArcSwap still wins, but only if your writes are full replacements — if you need partial mutation, ArcSwap isn't an option.

Read-only: This is ArcSwap's home turf. It's doing an atomic pointer load. Mutex and RwLock are both running kernel-level lock operations that ArcSwap simply doesn't need.

The takeaway: RwLock is not automatically better than Mutex. It's better when reads dominate. The moment writes become frequent, you're paying coordination costs for a benefit you're not actually getting.

Real-World Patterns

App Config with ArcSwap

use arc_swap::ArcSwap;
use std::sync::{Arc, OnceLock};
use std::collections::HashMap;

#[derive(Debug)]
pub struct AppConfig {
    pub db_url: String,
    pub max_connections: u32,
    pub feature_flags: HashMap<String, bool>,
}

static CONFIG: OnceLock<ArcSwap<AppConfig>> = OnceLock::new();

pub fn init_config(config: AppConfig) {
    CONFIG.get_or_init(|| ArcSwap::from_pointee(config));
}

pub fn current_config() -> Arc<AppConfig> {
    CONFIG.get().expect("config not initialized").load_full()
}

pub fn reload_config(new_config: AppConfig) {
    CONFIG.get().expect("config not initialized").store(Arc::new(new_config));
    // All new readers see the new config immediately.
    // In-flight requests holding the old Arc finish cleanly.
}

Request Counter with Mutex

For write-heavy state like metrics, Mutex (or better yet, atomics) is the right tool:

use std::sync::{Arc, Mutex};
use std::collections::HashMap;

pub struct Metrics {
    counters: Mutex<HashMap<String, u64>>,
}

impl Metrics {
    pub fn new() -> Self {
        Metrics { counters: Mutex::new(HashMap::new()) }
    }

    pub fn increment(&self, key: &str) {
        let mut guard = self.counters.lock().unwrap();
        *guard.entry(key.to_string()).or_insert(0) += 1;
    }

    pub fn snapshot(&self) -> HashMap<String, u64> {
        self.counters.lock().unwrap().clone()
    }
}

// For simple counters, just use atomics directly — no lock at all:
use std::sync::atomic::{AtomicU64, Ordering};

static REQUEST_COUNT: AtomicU64 = AtomicU64::new(0);

pub fn record_request() {
    REQUEST_COUNT.fetch_add(1, Ordering::Relaxed);
}

pub fn get_request_count() -> u64 {
    REQUEST_COUNT.load(Ordering::Relaxed)
}

Quick aside: if all you need is a counter, skip Mutex<u64> entirely and use AtomicU64. It's lock-free and faster than any of the three we've been benchmarking. Reserve Mutex for when you need to protect a whole data structure, not a single integer.

Read-Heavy Cache with RwLock

use std::sync::{Arc, RwLock};
use std::collections::HashMap;

pub struct Cache {
    inner: RwLock<HashMap<String, Arc<Vec<u8>>>>,
}

impl Cache {
    pub fn new() -> Self {
        Cache { inner: RwLock::new(HashMap::new()) }
    }

    pub fn get(&self, key: &str) -> Option<Arc<Vec<u8>>> {
        // Many threads can read simultaneously
        let guard = self.inner.read().unwrap();
        guard.get(key).cloned()
    }

    pub fn insert(&self, key: String, value: Vec<u8>) {
        // Exclusive — but this is rare in a read-heavy cache
        let mut guard = self.inner.write().unwrap();
        guard.insert(key, Arc::new(value));
    }

    pub fn invalidate(&self, key: &str) {
        let mut guard = self.inner.write().unwrap();
        guard.remove(key);
    }
}

The Decision Matrix

*ArcSwap write performance is good when writes are infrequent swaps. If you're swapping on every operation, the allocation cost adds up.

Wrapping Up

Here's my honest take after having used all three:

Mutex is your workhorse. Equal reads and writes, or you just need something correct right now. Don't overthink it. Ship it.

RwLock is the sensible default if you genuinely don't know your read/write ratio. Non-blocking reads are a good property to have. Just remember to use the tokio version in async code, and watch out if your writes turn out to be more frequent than expected — that's when it starts costing you.

ArcSwap is the one I wish someone had told me about earlier. If your global is read constantly and written by swapping the whole value — config reloads, feature flag updates, anything like that — it's the right tool. Lock-free reads, atomic swaps, no poisoning. It just sits there quietly doing its job at 120 million ops/sec while your RwLock is still queuing readers.

It's the Jamie Vardy of the Rust ecosystem — not the most famous name in the room, but when the situation calls for it, nothing else comes close.

And if you just need a number? AtomicU64. Don't reach for a lock when an atomic will do.

The next time someone asks "what do I use for a global?", you now have an actual answer — and the benchmarks to back it up.

(Though Arc<Mutex<T>> is still fine for most things. The borrow checker forgives. The benchmarks don't.)

Next in this series: Serialization Showdown — Protobuf vs JSON vs MessagePack in Rust. We'll compare ergonomics, schema evolution, payload sizes, and yes — more benchmarks.

How Tokio in Rust Actually Works: Runtimes, Tasks, and the Magic Behind `async fn`

Manan Shukla — Tue, 03 Mar 2026 17:02:37 +0000

If you've written async Rust, you've typed #[tokio::main] dozens of times. You've stared at it. You've trusted it. You've never once questioned what it actually does — because it works, and we don't interrogate things that work.

Until they don't. And then you're at 2am, runtime panicking, no idea why, Googling "tokio block_on nested" like a person who has made some choices.

This post is for that version of you. We're tearing open the black box — how Tokio's runtime is structured, what happens when you await something, how the scheduler keeps your CPU cores busy, and what that macro expands into. By the end, async Rust should feel a lot less like voodoo and a lot more like something you actually understand.

The Problem with Threads (Or: Why We Can't Have Nice Things)

Let's start with why async exists at all. OS threads are the classic way to handle concurrency, and they're genuinely great. The problem is they're also kind of... needy.

// Spawning OS threads — simple, familiar, a little reckless
use std::thread;

fn main() {
    let handles: Vec<_> = (0..10_000)
        .map(|i| {
            thread::spawn(move || {
                thread::sleep(std::time::Duration::from_secs(1));
                println!("Thread {} done", i);
            })
        })
        .collect();

    for h in handles {
        h.join().unwrap();
    }
}

Go ahead, run it. On most Linux systems, 10,000 threads will burn through 80–160 MB of RAM just for stacks — and that's before they do a single useful thing. The kernel also has to context-switch between all of them: saving registers, flushing caches, pretending each thread is the most important thing in the world. It adds up fast.

Each thread is basically a golden retriever. Full of energy, needs its own space, demands constant attention, and if you have 10,000 of them your house (server) is destroyed.

Now the async version:

// Spawning Tokio tasks — same result, dramatically less chaos
#[tokio::main]
async fn main() {
    let handles: Vec<_> = (0..10_000)
        .map(|i| {
            tokio::spawn(async move {
                tokio::time::sleep(std::time::Duration::from_secs(1)).await;
                println!("Task {} done", i);
            })
        })
        .collect();

    for h in handles {
        h.await.unwrap();
    }
}

Same result. But Tokio tasks aren't OS threads — they're tiny state machines living entirely in user space. A task waiting on I/O consumes almost nothing. No kernel involvement, no stack sitting idle eating RAM, no context switching. It's like replacing 10,000 golden retrievers with 10,000 cats — they'll all just sit there doing nothing until something interesting happens, and that's perfectly fine.

The trade-off: async is more complex to understand. Worth it? For I/O-bound workloads, absolutely.

How Tokio Actually Works

The Reactor Model (A.K.A. "Don't call us, we'll call you")

Tokio is built on a reactor pattern. Instead of a thread sitting there blocking on an I/O call like an anxious person refreshing their email, Tokio registers interest with the OS and parks the thread. When data actually arrives, the OS taps Tokio on the shoulder, and Tokio wakes the right task.

The OS-level mechanism differs by platform:

Linux: epoll
macOS/BSD: kqueue
Windows: IOCP (I/O Completion Ports)

Tokio wraps all of this via the mio crate, so you get one clean async API regardless of platform. You don't need to think about any of this — but it's satisfying to know it exists.

Here's the rough mental model:

Your async tasks
      │
      ▼
Tokio Scheduler  ◄──── wakes tasks when ready
      │
      ▼
  I/O Driver (mio)
      │
      ▼
  OS (epoll / kqueue / IOCP)
      │
      ▼
  Network, Files, Timers

When a task awaits a network read that isn't ready yet, it doesn't spin. It tells the I/O driver "wake me when this socket has data," and the thread moves on to run something else. This is how a single thread can juggle thousands of concurrent connections without breaking a sweat.

The `Future` Trait and the Poll Mechanism (The "Are We There Yet?" of Programming)

Every async fn in Rust compiles down to a type that implements the Future trait. You can think of a Future as a state machine that Tokio pokes repeatedly until it's done.

pub trait Future {
    type Output;
    fn poll(self: Pin<&mut Self>, cx: &mut Context<'_>) -> Poll<Self::Output>;
}

Poll has two variants:

Poll::Ready(value) — done! Here's your value.
Poll::Pending — not yet, come back later

Tokio is essentially a machine that calls poll() on tasks and handles the Pending responses politely instead of panicking. But here's the key question: how does Tokio know when "later" is? That's what the Waker is for.

The Context passed to poll carries a Waker — basically a callback. When your future can't make progress (waiting on a TCP read, for example), it clones the Waker and stores it somewhere safe. When the OS eventually signals that data has arrived, Tokio uses that stored Waker to re-queue the task. It's like leaving your number at a restaurant instead of standing in the doorway blocking everyone.

Let's demystify this completely by hand-rolling a minimal future:

use std::future::Future;
use std::pin::Pin;
use std::sync::{Arc, Mutex};
use std::task::{Context, Poll, Waker};
use std::thread;
use std::time::Duration;

// A future that resolves after a background thread finishes some "work"
// (in this case, sleeping, which is relatable)
struct DelayedValue {
    value: Arc<Mutex<Option<u32>>>,
    waker: Arc<Mutex<Option<Waker>>>,
}

impl DelayedValue {
    fn new(delay: Duration, value: u32) -> Self {
        let shared_value = Arc::new(Mutex::new(None));
        let shared_waker: Arc<Mutex<Option<Waker>>> = Arc::new(Mutex::new(None));

        let v = shared_value.clone();
        let w = shared_waker.clone();

        thread::spawn(move || {
            thread::sleep(delay);
            *v.lock().unwrap() = Some(value);

            // "Hey Tokio, I'm ready now" — this is the entire magic
            if let Some(waker) = w.lock().unwrap().take() {
                waker.wake();
            }
        });

        DelayedValue {
            value: shared_value,
            waker: shared_waker,
        }
    }
}

impl Future for DelayedValue {
    type Output = u32;

    fn poll(self: Pin<&mut Self>, cx: &mut Context<'_>) -> Poll<Self::Output> {
        let value = self.value.lock().unwrap();

        if let Some(v) = *value {
            Poll::Ready(v) // Done!
        } else {
            // Leave our number with the waker so we get called back
            *self.waker.lock().unwrap() = Some(cx.waker().clone());
            Poll::Pending // "Not yet, but I promise I'll call"
        }
    }
}

#[tokio::main]
async fn main() {
    let result = DelayedValue::new(Duration::from_millis(100), 42).await;
    println!("Got: {}", result); // Got: 42
}

That's the whole trick. Return Pending, store the waker, call waker.wake() when ready. No magic, no wizardry — just a very disciplined callback system. Every tokio::time::sleep, every TcpStream::read, every channel.recv() is doing exactly this under the hood.

The Task Scheduler and Work Stealing (Robin Hood, but for CPU Cores)

Tokio's multi-threaded runtime spawns one worker thread per logical CPU core. Each worker has its own local task queue. When you call tokio::spawn(...), the new task lands in the spawning thread's queue.

Now here's where it gets elegant: what happens when one thread's queue is empty while another is drowning in work? Work stealing. An idle thread reaches into a busy thread's queue and takes tasks from the back — quietly, without asking, like borrowing someone's chips at the office.

Thread 0 queue: [Task A, Task B, Task C]  ← very busy
Thread 1 queue: []                         ← bored, about to steal
Thread 2 queue: [Task D]
Thread 3 queue: [Task E, Task F]

After stealing:
Thread 0: [Task A, Task B]
Thread 1: [Task C]  ← got something to do now

This is completely invisible to you as a developer, which is the point. You don't partition work across threads. You don't think about which core handles which connection. You just spawn tasks, and Tokio distributes the load. It's the async equivalent of "it just works" — and in this case it actually does.

Unwrapping `#[tokio::main]` (Spoiler: It's Not That Deep)

Let's look at what the macro actually expands to. Install cargo-expand and try it yourself:

cargo install cargo-expand
cargo expand

You write this:

#[tokio::main]
async fn main() {
    println!("Hello from async!");
}

And the macro generates roughly this:

fn main() {
    tokio::runtime::Builder::new_multi_thread()
        .enable_all()
        .build()
        .unwrap()
        .block_on(async {
            println!("Hello from async!");
        })
}

That's it. Genuinely. That's the whole macro. It builds a multi_thread runtime, enables all the drivers (I/O, time), and calls .block_on() which runs your async main and blocks the OS thread until it completes.

block_on is the bridge between the synchronous world (your process entry point) and the async world. Everything inside the closure is async. Everything outside is normal, boring, synchronous Rust. The macro just spares you from writing the boilerplate every time — which is kind, because nobody wants to write that every time.

Runtime Flavours — Pick Your Fighter

Tokio ships two runtime flavours out of the box, plus a builder for when you want to get weird about it.

`multi_thread` — The Default

// This:
#[tokio::main]
async fn main() { /* ... */ }

// Is the same as this:
#[tokio::main(flavor = "multi_thread", worker_threads = 4)]
async fn main() { /* ... */ }
// (worker_threads defaults to your CPU count if you omit it)

Multiple worker threads, work-stealing scheduler, I/O tasks running in parallel. This is what you want for servers, gRPC services, anything handling concurrent I/O.

The default worker_threads count equals your number of logical CPU cores. On a 4-core machine you get 4 workers. On a 32-core beast you get 32. You can override it, but the default is usually sensible.

When to use it: Almost always. If you're writing a service that handles requests, use this.

`current_thread` — The Minimalist

#[tokio::main(flavor = "current_thread")]
async fn main() {
    // Everything runs on this one thread. Just this one. Cooperatively.
}

No parallelism. Tasks take turns on a single thread. It's cheaper to set up, has zero synchronization overhead, and works in environments that simply don't support multiple threads — like WebAssembly.

Here's the gotcha that will ruin your day if you forget it: any blocking call blocks everything. With multi_thread, one frozen thread leaves the others running. With current_thread, if one task decides to call std::fs::read_to_string synchronously, every single other task in your runtime grinds to a halt and stares at the ceiling until it finishes. Every. Single. One.

// current_thread + blocking call = everything stops
#[tokio::main(flavor = "current_thread")]
async fn main() {
    tokio::spawn(async {
        std::thread::sleep(std::time::Duration::from_secs(5)); // 💀 blocks all tasks
    });

    // This won't run for 5 seconds. You wanted async. You did not get async.
    tokio::spawn(async {
        println!("I thought I'd run concurrently... I was wrong");
    });
}

When to use it: CLI tools, scripts, tests, WASM, embedded. Anywhere you want async APIs without the overhead of a thread pool.

`Runtime::Builder` — For When You Want Full Control

Sometimes the macro isn't enough. You want to name your threads so they show up nicely in htop. You want a specific stack size. You want lifecycle hooks. This is where Runtime::Builder comes in:

use tokio::runtime::Builder;

fn main() {
    let runtime = Builder::new_multi_thread()
        .worker_threads(8)                    // Don't let Tokio decide
        .thread_name("my-worker")             // Show up nicely in profilers
        .thread_stack_size(3 * 1024 * 1024)   // 3 MB instead of 8 MB — save some RAM
        .on_thread_start(|| println!("Worker online"))
        .on_thread_stop(|| println!("Worker going home"))
        .enable_io()
        .enable_time()
        .build()
        .unwrap();

    runtime.block_on(async {
        // Your app here
    });
}

A particularly powerful pattern is running two separate runtimes — one for I/O tasks and one for CPU-heavy work. Think of it like having a front-of-house team handling requests and a kitchen team doing the actual cooking. You don't want them stealing each other's threads:

use tokio::runtime::Builder;
use std::thread;

fn main() {
    // Front of house: handles all the async I/O
    let io_runtime = Builder::new_multi_thread()
        .worker_threads(16)
        .thread_name("io-worker")
        .enable_all()
        .build()
        .unwrap();

    // Kitchen: CPU-intensive work, fewer threads (match physical cores)
    let cpu_runtime = Builder::new_multi_thread()
        .worker_threads(4)
        .thread_name("cpu-worker")
        .build()
        .unwrap();

    let io_handle = io_runtime.handle().clone();

    thread::spawn(move || {
        cpu_runtime.block_on(async move {
            // Heavy compute here — dispatch I/O back to the other runtime
            io_handle.spawn(async { /* network calls, DB queries, etc. */ });
        });
    });

    io_runtime.block_on(async {
        // Main entry point
    });
}

This is the kind of thing you reach for when building a database engine, a media transcoder, or anything where "just use one runtime" stops being good enough.

Async vs. Threads: An Honest Side-by-Side

Let's put both approaches next to each other for the same job: handling 1,000 concurrent "connections" that each wait 100ms.

With OS threads:

use std::thread;
use std::time::Duration;

fn handle_connection(id: u32) {
    // The whole thread just... sits here. Blocking. Thinking about its life choices.
    thread::sleep(Duration::from_millis(100));
    println!("Connection {} handled", id);
}

fn main() {
    let handles: Vec<_> = (0..1_000)
        .map(|i| thread::spawn(move || handle_connection(i)))
        .collect();

    for h in handles {
        h.join().unwrap();
    }
}
// ~1000 threads. Heavy. Memory-hungry. Gets the job done, but at a cost.

With Tokio:

use tokio::time::{sleep, Duration};

async fn handle_connection(id: u32) {
    // Yields control here — other tasks run while we wait
    sleep(Duration::from_millis(100)).await;
    println!("Connection {} handled", id);
}

#[tokio::main]
async fn main() {
    let handles: Vec<_> = (0..1_000)
        .map(|i| tokio::spawn(handle_connection(i)))
        .collect();

    for h in handles {
        h.await.unwrap();
    }
}
// ~1000 tasks. Tiny footprint. Scales to 100,000 without breaking a sweat.

Both complete in about 100ms. The difference is in what they consume getting there.

When Threads Are Actually the Right Answer

I want to be honest with you: async is not always better. For CPU-bound work — compressing files, running cryptography, processing images — async can actively hurt you. A heavy computation holds a thread and can't yield, which means it's blocking the scheduler from running other tasks on that thread. You've accidentally built a traffic jam.

For CPU work, the right tools are:

tokio::task::spawn_blocking — ships the work to a dedicated blocking thread pool, away from your async workers
rayon — a data-parallelism library built specifically for this

// ❌ Don't do this — you're holding up the async scheduler
tokio::spawn(async {
    let result = crunch_massive_dataset(); // Blocks the thread. Other tasks: 😤
    result
});

// ✅ Do this instead
tokio::spawn(async {
    let result = tokio::task::spawn_blocking(|| {
        crunch_massive_dataset() // Runs on a separate blocking thread pool
    }).await.unwrap();
    result
});

spawn_blocking is your escape hatch. Use it whenever you need to call synchronous, potentially slow code from inside an async context.

The Mental Model, All Together

Here's the complete picture of what happens every time you await something:

You write async fn — Rust compiles it into a state machine implementing Future
tokio::spawn drops that future into the scheduler's task queue
A worker thread picks it up and calls poll()
If the future needs to wait (I/O, timer, channel), it stores a Waker and returns Poll::Pending
The I/O driver watches the underlying OS event (epoll/kqueue/IOCP)
When the event fires, the driver calls waker.wake(), re-queuing the task
A worker thread (could be a different one — work stealing!) picks it up and polls again
Repeat until Poll::Ready — task is complete, handle is resolved

And here's your quick reference for picking the right tool:

Scenario	Reach For
Concurrent I/O (HTTP, sockets, DB)	`tokio::spawn` + `async fn`
CPU-bound work (compute, encoding)	`spawn_blocking` or `rayon`
"I understand async but don't want it right now"	`std::thread`
Single-threaded env / WASM / CLI	`current_thread` flavor
Fine-grained runtime control	`Runtime::Builder`
Simple async binary	`#[tokio::main]` and don't overthink it

Wrapping Up

Tokio isn't magic. It's a well-engineered scheduler sitting on top of OS primitives you already know. #[tokio::main] is just Runtime::new().block_on(...). Tasks are futures being polled by worker threads. The work-stealing scheduler is why you don't have to think about load balancing. And the difference between current_thread and multi_thread is the difference between a fine Tuesday and a very confusing production incident.

Understanding this doesn't mean you'll reimplement Tokio — please don't. But it does mean that next time something goes sideways in your async code, you'll have a mental model to debug against instead of just staring at the error and hoping it fixes itself.

(It won't fix itself. It never does.)

Next up in this series: Shared State in Rust — Mutex, RwLock, and ArcSwap. What they are, when to use each, and how they actually perform under real contention. Spoiler: the answer is not always "just use a Mutex."

Forem: Manan Shukla

gRPC Streaming in Rust: The Four Patterns You Need to Know (with Tonic)

Why gRPC? The Quick Version

Project Setup

Pattern 1 — Unary (The Baseline)

Pattern 2 — Server Streaming

Pattern 3 — Client Streaming

Pattern 4 — Bidirectional Streaming

Wiring It All Up

Error Handling and Gotchas

When to Use What

Wrapping Up

Serialization Showdown in Rust: JSON Was Fine Until It Wasn't

Why Serialization Is Worth Thinking About

JSON — The One You Already Know (serde_json)

Serde Ergonomics Worth Knowing

When JSON Is the Right Answer

Protobuf — The Serious One (prost)

When Protobuf Is the Right Answer

MessagePack — The Middle Ground (rmp-serde)

When MessagePack Is the Right Answer

Schema Evolution — The Conversation Nobody Wants to Have

JSON — Flexible but Unguarded

Protobuf — Strict but Safe (If You Follow the Rules)

MessagePack — Flexible Like JSON, With One Trap

Benchmarks — The Part You Scrolled Here For

What the Numbers Typically Look Like

Real-World Patterns

API Request/Response

Redis Storage — Size Comparison

The Decision Table

Wrapping Up

Rust Globals and Locks: Mutex, RwLock, and the Lock-Free Trick (ArcSwap) I Wish I’d Known Earlier

Globals: Why We Need Them and What Made This Complicated

Mutex — The Bouncer

The Gotcha: Lock Poisoning

When to Use Mutex

RwLock — The Sensible Default (If You're Unsure, Start Here)

std::sync::RwLock vs tokio::sync::RwLock — Which One?

The Gotcha: Write Starvation

When to Use RwLock

ArcSwap — The Dark Horse (I Wish I'd Found This Sooner)

How It Works Under the Hood

The Constraint: Writes Must Replace, Not Mutate

When to Use ArcSwap

Benchmarks — Let's Actually Measure This

Setup

Manual Tokio Benchmark

Criterion Benchmark (Formal)

What the Numbers Typically Look Like

Real-World Patterns

App Config with ArcSwap

Request Counter with Mutex

Read-Heavy Cache with RwLock

The Decision Matrix

Wrapping Up

How Tokio in Rust Actually Works: Runtimes, Tasks, and the Magic Behind `async fn`

The Problem with Threads (Or: Why We Can't Have Nice Things)

How Tokio Actually Works

The Reactor Model (A.K.A. "Don't call us, we'll call you")

The Future Trait and the Poll Mechanism (The "Are We There Yet?" of Programming)

The Task Scheduler and Work Stealing (Robin Hood, but for CPU Cores)

Unwrapping #[tokio::main] (Spoiler: It's Not That Deep)

Runtime Flavours — Pick Your Fighter

multi_thread — The Default

current_thread — The Minimalist

Runtime::Builder — For When You Want Full Control

Async vs. Threads: An Honest Side-by-Side

When Threads Are Actually the Right Answer

The Mental Model, All Together

Wrapping Up

JSON — The One You Already Know (`serde_json`)

Protobuf — The Serious One (`prost`)

MessagePack — The Middle Ground (`rmp-serde`)

The `Future` Trait and the Poll Mechanism (The "Are We There Yet?" of Programming)

Unwrapping `#[tokio::main]` (Spoiler: It's Not That Deep)

`multi_thread` — The Default

`current_thread` — The Minimalist

`Runtime::Builder` — For When You Want Full Control