Loading post...
Please wait
May 9, 2026
A comprehensive learning resource for software engineers who want to actually understand what their async code is doing — ~10,000 words, 15+ diagrams, and practical mental models
If you've written async Rust, you've probably felt that moment of confusion. You add async to a function, sprinkle some .await calls, wrap it in #[tokio::main], and it compiles. But then questions creep in:
tokio::join! actually run things concurrently?Send bounds on spawned futures?Most tutorials stop at "here's the syntax." This guide goes deeper. We'll build a mental model from the ground up — futures, tasks, the reactor, the scheduler — using analogies that map to real engineering concepts, plus diagrams you can reason about.
By the end, you'll not only write better async Rust, you'll understand why it works the way it does.
Target audience: Software engineers comfortable with Rust basics who want to master async. No prior Tokio internals knowledge required.
Traditional synchronous code has a fundamental problem: when you wait for I/O (network, disk, database), your thread does nothing but hold memory.
// Synchronous - thread BLOCKS here
let response = reqwest::blocking::get("https://api.example.com/users/42")?;
// Thread sits idle for 50-500ms holding ~1MB stack
let user: User = response.json()?;
Why ~1MB? Each OS thread has its own private stack (typically 1MB by default on Linux/Windows). This stack is reserved upfront so the thread can store local variables, function call frames, and return addresses as it executes. Even when the thread is blocked waiting for I/O, that stack memory stays allocated—it can't be reclaimed because the OS doesn't know how much the thread will need when it resumes.
A typical web server handling 10,000 concurrent connections with sync code needs 10,000 threads = ~10 GB RAM just for stacks. Most of those threads are sitting parked, waiting for network responses, wasting memory on reserved stacks that aren't being used productively.
Async Rust flips this: when a task needs to wait, it explicitly yields control back to the runtime so the thread can do other work.
// Async - task YIELDS here
let response = reqwest::get("https://api.example.com/users/42").await?;
// Task parked, thread FREE to run other tasks
let user: User = response.json().await?;
Key distinction: The thread doesn't block. The task pauses, and the thread picks up another task.
Why this matters for I/O tasks:
I/O operations (network requests, disk reads, database queries) are inherently waiting operations—they take 50–500ms where no actual CPU work happens. With async:
| Aspect | Sync (blocking) | Async (non-blocking) |
|---|---|---|
| 10,000 I/O requests | 10,000 threads × 1MB = 10GB RAM | ~1 thread × 1MB + futures = ~few MB |
| CPU utilization | Threads idle 99% of time waiting | Thread always doing work |
| Memory overhead | 1MB per connection (stack) | ~100 bytes per connection (Future struct) |
| Scalability | Limited by thread count/memory | Limited only by I/O bandwidth |
The thread never sits idle—it's constantly polling many futures. Each async task is just a small Future struct (often <100 bytes) instead of a full thread stack. This is why async is orders of more efficient for I/O-bound workloads like web servers, databases, or network services, where the bottleneck is waiting for external systems, not CPU computation.
graph TD
subgraph SYNC["SYNC: Thread-Based Memory Layout"]
direction TB
header["10,000 threads = ~10GB RAM wasted"]
thread0["Thread 0 Stack<br/>(1MB reserved)"]
thread1["Thread 1 Stack<br/>(1MB reserved)"]
thread2["Thread 2 Stack<br/>(1MB reserved)"]
threadMore["... (10,000 total)<br/>Most threads BLOCKED waiting for I/O"]
sharedHeap["Shared Heap<br/>(small, shared by all threads)"]
header --> thread0
thread0 --> thread1
thread1 --> thread2
thread2 --> threadMore
threadMore --> sharedHeap
end
style header fill:#ff6b6b,color:#fff,stroke:#c0392b
style thread0 fill:#ffd93d,stroke:#f39c12
style thread1 fill:#ffd93d,stroke:#f39c12
style thread2 fill:#ffd93d,stroke:#f39c12
style threadMore fill:#ffd93d,stroke:#f39c12
style sharedHeap fill:#6bcb77,stroke:#27ae60,color:#fff
graph TD
subgraph ASYNC["ASYNC: Single-Threaded Memory Layout"]
direction TB
header["1 thread + 10,000 futures = ~2MB RAM"]
singleStack["Single Thread Stack<br/>(1MB - only ONE stack!)"]
heapLabel["Heap Memory"]
future0["Future 0<br/>(~100 bytes)"]
future1["Future 1<br/>(~100 bytes)"]
future2["Future 2<br/>(~100 bytes)"]
futureMore["... (10,000 futures)<br/>All YIELD, thread FREE to work"]
header --> singleStack
singleStack --> heapLabel
heapLabel --> future0
future0 --> future1
future1 --> future2
future2 --> futureMore
end
style header fill:#6bcb77,color:#fff,stroke:#27ae60
style singleStack fill:#ffd93d,stroke:#f39c12
style heapLabel fill:#4ecdc4,stroke:#16a085,color:#fff
style future0 fill:#4ecdc4,stroke:#16a085,color:#fff
style future1 fill:#4ecdc4,stroke:#16a085,color:#fff
style future2 fill:#4ecdc4,stroke:#16a085,color:#fff
style futureMore fill:#4ecdc4,stroke:#16a085,color:#fff
graph LR
subgraph SYNC["SYNC<br/>10,000 Threads"]
direction TB
s1["10,000 Threads"] --> s2["10,000 Stacks<br/>(1MB each)"]
s2 --> s3["10GB RAM"]
s3 --> s4["Most BLOCKED<br/>wasting memory"]
end
subgraph ASYNC["ASYNC<br/>1 Thread + Futures"]
direction TB
a1["1 Thread"] --> a2["1 Stack<br/>(1MB)"]
a2 --> a3["+ 10,000 Futures<br/>(~100 bytes each)"]
a3 --> a4["~2MB RAM<br/>(5000x smaller!)"]
a4 --> a5["Thread always<br/>DOING WORK"]
end
SYNC --> ASYNC
style SYNC fill:#ff6b6b,color:#fff,stroke:#c0392b
style s1 fill:#ff6b6b,color:#fff
style s2 fill:#ff6b6b,color:#fff
style s3 fill:#ff6b6b,color:#fff
style s4 fill:#ff6b6b,color:#fff
style ASYNC fill:#6bcb77,color:#fff,stroke:#27ae60
style a1 fill:#6bcb77,color:#fff
style a2 fill:#6bcb77,color:#fff
style a3 fill:#6bcb77,color:#fff
style a4 fill:#6bcb77,color:#fff
style a5 fill:#6bcb77,color:#fff
These diagrams visually show:
| Sync Model | Async Model |
|---|---|
| One chef per order. Chef stands at stove waiting for water to boil. 100 orders = 100 chefs. | One chef (worker thread), many order tickets (tasks). Chef starts pasta, sets timer, chops veggies for next order while water boils. |
| Blocking: Chef's time frozen | Yielding: Chef multitasks |
| Scales poorly (linear chef cost) | Scales well (one chef, many tickets) |
flowchart TD
A[Async Rust] --> B[Futures<br/>Lazy state machines<br/>Recipe cards]
A --> C[Tasks<br/>Futures + runtime<br/>Work orders]
A --> D[Runtime<br/>Scheduler + Executor + Reactor<br/>Kitchen + dispatcher]
B -.->|driven by| D
C -.->|managed by| D
D -.->|polls| B
If you're coming from JavaScript, you have a mental model that looks like this:
// JavaScript - EAGER promises
async function fetchUser(id) {
const response = await fetch(`/users/${id}`); // STARTS IMMEDIATELY
return response.json();
}
const userPromise = fetchUser(42); // Kitchen ALREADY cooking
// ... later ...
const user = await userPromise;
In JS, calling an async function starts the work immediately. The promise is "eager" — it's already in flight before you even await it. Like ordering at a restaurant: the kitchen starts cooking the moment you place the order.
Rust is fundamentally different.
// Rust - LAZY futures
async fn fetch_user(id: u64) -> Result<User, Error> {
let response = reqwest::get(format!("/users/{id}")).await?;
Ok(response.json().await?)
}
let future = fetch_user(42); // Returns a Future - NOTHING HAPPENS YET
// ... later ...
let user = future.await; // NOW the work starts
Calling fetch_user(42) doesn't make a network request. It returns a future — a state machine that describes the work to be done. The work only happens when something drives that future to completion.
Think of a future like a recipe card. Writing the recipe doesn't cook the food. You need a cook (the runtime) to actually follow the steps.
flowchart LR
A["Call async fn"] --> B["Returns Future<br>(lazy recipe / to-do list)"]
B --> C["Spawn Task via Runtime"]
C --> D["Runtime drives Future<br>to completion"]
Without a runtime driving it, a future is just data sitting in memory — no CPU cycles spent, no I/O initiated, no progress made.
Key insight: A future is pure data. It implements the
Futuretrait, which has one method:poll(). Until something callspoll(), the future is inert.
When you write:
async fn example() {
let a = step_one().await;
let b = step_two(a).await;
step_three(b).await;
}
The compiler transforms this into a struct that implements Future:
// SIMPLIFIED conceptual version - actual output is more complex
enum ExampleFuture {
Start,
WaitingOnStepOne { step_one_future: StepOneFuture },
WaitingOnStepTwo { a: OutputA, step_two_future: StepTwoFuture },
WaitingOnStepThree { b: OutputB, step_three_future: StepThreeFuture },
Done(OutputC),
}
impl Future for ExampleFuture {
type Output = OutputC;
fn poll(self: Pin<&mut Self>, cx: &mut Context<'_>) -> Poll<Self::Output> {
match self {
// ... state machine transitions ...
}
}
}
Each .await point becomes a state variant. The future stores whatever local variables are needed to resume after that await. This is why futures are lazy — they're just data structures until polled.
PinYou'll often see Pin<&mut Self> in future signatures. This exists because a future's internal references must not move in memory.
async fn problematic() {
let data = vec![1, 2, 3];
let ptr = &data[0] as *const i32; // Pointer into `data`
some_async_op().await; // YIELD - future may be moved!
println!("{}", unsafe { *ptr }); // UB if `data` moved!
}
If the future gets moved in memory between .await points (e.g., when the runtime moves it between threads), any internal pointers become invalid. Pin guarantees the future stays at a fixed memory address.
Practical rule: You rarely interact with Pin directly. The compiler handles it when you .await or use tokio::spawn. Just know it exists for memory safety.
poll Contractfn poll(self: Pin<&mut Self>, cx: &mut Context<'_>) -> Poll<Self::Output>
Poll::Ready(value) — work complete, here's the resultPoll::Pending — not ready yet, call me again laterPending, the future must arrange for cx.waker().wake() to be called when readyThe waker is how the future says "hey, I'm ready to continue, come poll me again." This is the critical link between async code and the runtime.
A future by itself is passive. To make it run, you need a task — a future that's been handed to a runtime for execution.
// This creates a task and schedules it
tokio::spawn(async {
let user = fetch_user(42).await;
println!("Got user: {:?}", user);
});
If a future is a recipe, a task is a work order handed to a kitchen (the runtime). The kitchen has cooks (worker threads) who pick up work orders and follow the recipes.
sequenceDiagram
participant App as Your Code
participant Runtime as Tokio Runtime
participant Task as Task (Future + Metadata)
participant Worker as Worker Thread
App->>Runtime: spawn(async { ... })
Runtime->>Task: Wrap future + allocate metadata
Runtime->>Worker: Push to local queue
Worker->>Task: Poll future (drive to next .await)
Task-->>Worker: Returns Poll::Pending (yielded)
Note over Worker: Runs OTHER tasks while waiting
Worker->>Task: Poll again when ready
Task-->>Worker: Returns Poll::Ready(value)
A Tokio task isn't just the future. It carries metadata:
async blocks that need local variables)Size comparison that matters:
That's ~15,000× smaller. A single process can hold millions of tasks. This is why async Rust scales to massive concurrency without drowning in thread overhead.
flowchart LR
subgraph Memory Comparison
T[Tokio Task<br/>~64 bytes]
O[OS Thread<br/>~1 MB+]
end
T -->|15,000x smaller| O
stateDiagram-v2
[*] --> Created
Created --> Queued
Queued --> Running
Running --> Parked
Parked --> Queued
Running --> Completed
Completed --> [*]
Parked --> Cancelled
Cancelled --> [*]
note right of Created : tokio spawn
note right of Queued : Pushed to scheduler queue
note right of Running : Worker pops task
note right of Parked : .await returns Pending
note right of Queued : Waker.wake called
note right of Completed : .await returns Ready
note right of [*] : Cleanup
note right of Cancelled : Drop JoinHandle or abort
note right of [*] : Cleanup
Tokio isn't a single thing. It's a composition of seven specialized components working together. Here's the mental model:
Note on component count: You'll sometimes see "Tokio has 4 components" (scheduler, executor, timer, I/O driver) or "6 components" (adding tasks and queues as separate). The seven-component model here separates concerns for learning clarity — tasks/queues are part of the scheduler but deserve their own mental models.
flowchart TD
subgraph Runtime [Tokio Runtime]
direction TB
subgraph Scheduler [Scheduler & Executor]
Sched[Scheduler Work-stealing dispatcher]
Exec[Executor Drives futures via poll]
end
subgraph Queues [Task Queues]
LQ1[Local Queue 1 Ring buffer 256 slots]
LQ2[Local Queue 2 Ring buffer 256 slots]
LQ3[Local Queue 3 Ring buffer 256 slots]
LQ4[Local Queue 4 Ring buffer 256 slots]
GQ[Global Queue Overflow buffer]
end
Reactor[Reactor / IO Driver epoll / kqueue / IOCP via Mio]
Timer[Timer Wheel Timeouts, sleep, intervals]
end
OS[Operating System Network, Filesystem, Timers]
Sched --> LQ1
Sched --> LQ2
Sched --> LQ3
Sched --> LQ4
Sched --> GQ
Exec -.-> Sched
Reactor --> OS
Timer --> OS
Sched --> Reactor
Reactor --> Sched
Let's break down each piece in depth.
The scheduler decides which task runs on which thread. Its superpower: work stealing.
sequenceDiagram
participant T1 as Thread 1 (idle)
participant T2 as Thread 2 (busy)
participant GQ as Global Queue
participant LQ2 as Thread 2 Local Queue
T1->>LQ2: Steal task from tail
LQ2-->>T1: Task acquired
Note over T1: Now productive!
T2->>LQ2: Push new tasks to head
T1->>GQ: If local empty, check global
GQ-->>T1: Task or empty
T1->>T2: If both empty, steal from peer
Thread 1 Local Queue (head ←→ tail): [Task A] [Task B] [Task C]
↑ ↑
Owner pushes Thief steals
here here
Before work stealing (pre-November 2019), Tokio used a single global queue with a mutex. Every task submission fought for the same lock. The work-stealing redesign made Tokio 10× faster because:
// Multi-threaded (default, uses all cores)
#[tokio::main]
async fn main() { ... }
// Single-threaded (for testing, or when you need strict ordering)
#[tokio::main(flavor = "current_thread")]
async fn main() { ... }
// Custom worker count
tokio::runtime::Builder::new_multi_thread()
.worker_threads(4)
.enable_all()
.build()
.unwrap()
.block_on(async { ... });
Each worker thread owns a local queue implemented as a ring buffer of ~256 slots.
flowchart LR
subgraph RingBuffer [Ring Buffer 256 slots in L1 Cache]
direction LR
Head[Head Index] --> Slot1[Slot 0]
Slot1 --> Slot2[Slot 1]
Slot2 --> Slot3[Slot 2]
Slot3 --> SlotN[Slot N]
SlotN --> Tail[Tail Index]
Tail -.-> Head
end
Worker[Worker Thread] -->|Push to head| Head
Worker -->|Pop from head| Head
Thief[Thief Thread] -->|Steal from tail| Tail
| Property | Benefit |
|---|---|
| Fixed size (~256 slots × 8 bytes = 2 KB) | Fits entirely in L1 CPU cache |
| Head pointer hot in cache | pop is a single cache hit + pointer increment |
| Lock-free for owner | No mutex, no syscall, no cache line bouncing |
| Atomic steal from tail | Thieves use CompareAndSwap — minimal contention |
| O(1) push/pop/steal | Predictable latency, no allocation |
CPU Core 1 CPU Core 2
┌─────────────────────┐ ┌─────────────────────┐
│ L1 Cache (32 KB) │ │ L1 Cache (32 KB) │
│ ┌─────────────────┐ │ │ ┌─────────────────┐ │
│ │ Local Queue │ │ │ │ Local Queue │ │
│ │ [Task][Task]... │ │ ←HOT │ │ [Task][Task]... │ │ ←HOT
│ └─────────────────┘ │ │ └─────────────────┘ │
└─────────────────────┘ └─────────────────────┘
│ │
└──────────────┬─────────────────────┘
▼
┌─────────────────────┐
│ L2/L3 Cache / RAM │
│ Global Queue │
│ (only when needed) │
└─────────────────────┘
The common case — a worker picking up its own task — involves zero locks and a cache hit. This is a huge part of why Tokio achieves high throughput.
When a local queue fills up (more than 256 tasks queued for one thread), excess tasks spill to the global queue — a single queue shared across all workers.
flowchart TD
Spawn[Task Spawned] --> LocalFull{Local Queue<br/>Full?}
LocalFull -- No --> PushLocal[Push to Local Queue]
LocalFull -- Yes --> PushGlobal[Push to Global Queue]
Worker[Worker Picks Work] --> CheckLocal{Local Queue<br/>Empty?}
CheckLocal -- No --> PopLocal[Pop from Local]
CheckLocal -- Yes --> CheckGlobal{Global Queue<br/>Empty?}
CheckGlobal -- No --> PopGlobal[Pop from Global]
CheckGlobal -- Yes --> Steal[Steal from Peer]
This design ensures:
The executor is surprisingly simple: it calls poll() on futures repeatedly until they complete.
// CONCEPTUAL executor loop (simplified)
fn run_task(task: &mut Task) {
let waker = task.waker();
let mut cx = Context::from_waker(&waker);
loop {
// SAFETY: future is pinned in task allocation
let future = unsafe { task.future.get_unchecked_mut() };
match future.as_mut().poll(&mut cx) {
Poll::Ready(value) => {
// Task done - store result, run drop glue, clean up
task.complete(value);
break;
}
Poll::Pending => {
// Task yielded - it registered our waker
// Executor yields control back to scheduler
break; // Actually returns to scheduler loop
}
}
}
}
The executor doesn't decide which task runs next — that's the scheduler's job. The executor just knows how to drive one future forward.
Key point: The executor is stateless. It's just a function that takes a task and polls it once. The scheduler decides when to call the executor again.
This is where async I/O becomes possible. The reactor sits between your tasks and the operating system's event notification APIs.
sequenceDiagram
participant Task as Async Task
participant Reactor as Reactor (IO Driver)
participant Mio as Mio
participant OS as OS (epoll/kqueue/IOCP)
Task->>Reactor: Needs to wait for I/O (.await)
Reactor->>Task: Takes Waker, registers interest
Reactor->>Mio: Register fd + waker
Mio->>OS: epoll_ctl / kevent / RegisterWait
Note over Task: Task PARKED - worker thread FREE
Worker->>OtherTask: Runs other tasks!
OS->>Mio: Event ready (data arrived)
Mio->>Reactor: Callback triggered
Reactor->>Task: Wake! (mark runnable)
Reactor->>Scheduler: Re-queue task
Scheduler->>Worker: Task available (maybe DIFFERENT thread!)
Worker->>Task: Poll again --> Poll::Ready(data)
epoll_wait, kevent, GetQueuedCompletionStatus)waker.wake() to re-queueMio (Metal IO, by Carl Lerche) predates Tokio and provides the thin, safe Rust layer over:
| OS | Syscall | Mio Abstraction |
|---|---|---|
| Linux | epoll |
Poll::new() + registry.register() |
| macOS/BSD | kqueue |
Same API |
| Windows | IOCP (I/O Completion Ports) |
Same API |
Mio is not a runtime — it's just the registry + event loop primitive. Tokio builds its reactor on top.
Historical note: Mio was originally called "mio" (metal I/O) and was created by Carl Lerche in 2014, before Tokio existed. It was designed as the minimal safe abstraction over OS event APIs. Tokio 0.1 (2016) built on top of it. The separation allows Mio to stay lean while Tokio handles higher-level concerns like scheduling, timers, and task management.
// Simplified Waker structure
struct Waker {
// RawWaker contains:
// - data: *const () -- pointer to task
// - vtable: RawWakerVTable -- function pointers
// - clone: fn(*const ()) -> RawWaker
// - wake: fn(*const ()) -- "wake this task"
// - wake_by_ref: fn(*const ())
// - drop: fn(*const ())
inner: RawWaker,
}
impl Waker {
fn wake(self) {
// Calls the vtable's wake function with task pointer
// Which ultimately calls scheduler.schedule(task)
}
}
When you .await on I/O, the future:
cx.waker()Poll::PendingWhen I/O completes, the reactor calls waker.wake(), which tells the scheduler "this task is runnable again."
Tokio also needs to handle sleep, timeouts, and intervals. A naive approach (one timer per task, sorted heap) degrades with millions of timers: O(log n) per operation.
Tokio uses a hierarchical timer wheel — O(1) insertion and removal, O(1) per tick.
flowchart TD
subgraph TimerWheel [Hierarchical Timer Wheel]
TW1[Wheel 1: Milliseconds<br/>64 slots, 1ms per slot]
TW2[Wheel 2: Seconds<br/>64 slots, 1s per slot]
TW3[Wheel 3: Minutes<br/>64 slots, 1m per slot]
TW4[Wheel 4: Hours<br/>64 slots, 1h per slot]
end
Task[T: sleep 1h 30m 45s] --> TW1
Task --> TW2
Task --> TW3
Task --> TW4
Wheel 1 (ms): [0][1][2]...[63] → ticks every 1ms
↓ cascades after 64ms
Wheel 2 (s): [0][1][2]...[63] → each slot = 64ms, ticks every 64ms
↓ cascades after ~4s
Wheel 3 (m): [0][1][2]...[63] → each slot = 4s, ticks every 4s
↓ cascades after ~4m
Wheel 4 (h): [0][1][2]...[63] → each slot = 4m, ticks every 4m
This scales to millions of timers with minimal overhead.
Implementation detail: Tokio's timer wheel is actually more sophisticated — it uses a single array of slots with multiple "tick levels" rather than separate arrays, and employs a "missed tick" optimization where if the reactor wakes up late, it processes multiple slots at once to catch up.
| Setting | Resolution | Use Case |
|---|---|---|
| Default | 1ms | General purpose |
enable_time() with tick_duration |
Configurable | High-precision needs |
pause() / resume() |
Test-controlled | Deterministic testing |
// For testing - manual time control
tokio::time::pause();
tokio::time::advance(Duration::from_secs(10)).await;
// All sleep(10s) complete instantly
How the pieces connect at runtime startup:
// Simplified Tokio runtime initialization
fn build_runtime() -> Runtime {
// 1. Create the I/O driver (reactor)
let io_driver = IoDriver::new(); // Wraps Mio's Poll
// 2. Create scheduler with worker threads
let scheduler = Scheduler::new(num_workers, io_driver.handle());
// 3. Spawn worker threads
for i in 0..num_workers {
let scheduler = scheduler.clone();
let io_handle = io_driver.handle().clone();
std::thread::spawn(move || {
worker_main(scheduler, io_handle);
});
}
// 4. Start I/O driver on dedicated thread (or first worker)
std::thread::spawn(move || {
io_driver.run(); // Blocks on epoll_wait/kevent/GetQueuedCompletionStatus
});
Runtime { scheduler, io_driver, ... }
}
Worker thread main loop (simplified):
fn worker_main(scheduler: Scheduler, io: IoHandle) {
loop {
// 1. Try local queue (lock-free, fast)
if let Some(task) = scheduler.pop_local() {
scheduler.execute(task); // polls future once
continue;
}
// 2. Try global queue
if let Some(task) = scheduler.pop_global() {
scheduler.execute(task);
continue;
}
// 3. Steal from peer
if let Some(task) = scheduler.steal() {
scheduler.execute(task);
continue;
}
// 4. Nothing to do - park thread briefly
// (or help drive I/O if configured)
std::thread::park_timeout(Duration::from_micros(100));
}
}
The reactor runs on its own thread (or integrated into workers), blocking on the OS event syscall. When events arrive, it wakes tasks and pushes them to scheduler queues.
Let's trace a single async task from spawn to completion, seeing how all components interact.
flowchart TD
Start([Spawn Task]) --> Push[Push to Local Queue]
Push --> Schedule[Scheduler picks task]
Schedule --> Exec[Executor polls future]
Exec --> Await{Task hits .await?}
Await -- No (ready) --> Complete([Task Complete])
Await -- Yes (I/O) --> Reactor[Reactor takes waker]
Reactor --> Register[Register interest with OS via Mio]
Register --> Park[Park task - Worker FREE]
Park --> WorkerRuns[Worker runs OTHER tasks]
OS[OS completes I/O] --> Signal[Signal reactor via Mio]
Signal --> Wake[Reactor wakes task]
Wake --> Requeue[Scheduler re-queues task]
Requeue --> DifferentThread{Maybe DIFFERENT thread!}
DifferentThread --> Exec2[Executor polls again]
Exec2 --> Await
sequenceDiagram
participant Main as Main Thread
participant RT as Tokio Runtime
participant W1 as Worker Thread 1
participant W4 as Worker Thread 4
participant Reactor as Reactor
participant OS as Operating System
participant DB as Database
Main->>RT: #[tokio::main] starts runtime
RT->>W1: Spawn 4 worker threads
Main->>RT: spawn(async { fetch_user().await })
RT->>W1: Task queued in Local Queue 1
W1->>W1: Pop task, executor polls
Note over W1: fetch_user() starts DB query
W1->>Reactor: .await on DB --> register waker
Reactor->>OS: epoll_ctl (wait for DB response)
Reactor-->>W1: Task parked
W1->>W1: Pick NEXT task from queue
OS->>DB: Execute query
DB->>OS: Response ready
OS->>Reactor: epoll returns event
Reactor->>Reactor: Wake task (call waker)
Reactor->>RT: Re-queue task
RT->>W4: Task queued in Local Queue 4
W4->>W4: Pop task, executor polls
Note over W4: fetch_user() resumes, returns data
W4-->>Main: Task complete
After the I/O completes, the task might resume on a different worker thread than where it started.
This is why Rust requires Send bounds on spawned futures — the task can move across threads between .await points.
// This COMPILES - String is Send
tokio::spawn(async {
let s = String::from("hello");
do_async_work().await;
println!("{}", s);
});
// This FAILS - Rc is NOT Send
tokio::spawn(async {
let rc = Rc::new(5);
do_async_work().await; // Task might move to another thread!
println!("{}", rc); // Rc not safe across threads
});
Rule of thumb: If your async block captures non-Send types (like Rc, RefCell, raw pointers, or !Send thread-locals), don't spawn it. Use tokio::task::spawn_local on a single-threaded runtime instead.
Now that you understand the runtime, let's look at the primitives you use daily — and what they actually do under the hood.
.awaitlet user = fetch_user(id).await;
let posts = fetch_posts(user.id).await;
What happens: Single task, driven to completion step by step. No concurrency.
flowchart LR
T[Task] --> F1[fetch_user]
F1 --> F2[fetch_posts]
F2 --> Done[Complete]
Analogy: Cooking a recipe step by step. Stir sauce, then boil pasta. Simple, predictable, but no parallelism.
When to use: Dependencies between operations (need user before posts), or when you want simple, readable code.
tokio::join!let (user, posts) = tokio::join!(
fetch_user(id),
fetch_posts(id)
);
What happens: Single task drives multiple futures concurrently by polling them in a loop.
flowchart TD
T[Single Task] --> Loop[Poll Loop]
Loop --> F1[fetch_user]
Loop --> F2[fetch_posts]
F1 -.->|Pending| Loop
F2 -.->|Pending| Loop
F1 -->|Ready| Done1[User Ready]
F2 -->|Ready| Done2[Posts Ready]
Done1 --> Both[Both Ready]
Done2 --> Both
Both --> Complete
Under the hood: join! creates a future that wraps both futures and implements poll by polling each child in turn until both are Ready. If one returns Pending, it stores that state and moves to the next.
Analogy: Cooking multiple pots on a stove simultaneously. You stir pot A, then pot B, then back to A. Total time = slowest pot, not sum.
When to use: Independent operations that you need all results from. Prefer over spawning separate tasks when operations are fast and related.
tokio::select!tokio::select! {
result = fetch_user(id) => {
println!("User fetched: {:?}", result);
}
_ = sleep(Duration::from_secs(5)) => {
println!("Timeout!");
}
}
What happens: Polls both futures. First to return Ready wins; the other is dropped (cancelled).
flowchart TD
T[Task] --> Select[select! Future]
Select --> F1[fetch_user]
Select --> F2[sleep 5s]
F1 -->|Ready first| Win1[Handle result]
F2 -->|Ready first| Win2[Handle timeout]
Win1 -.-> Drop[Cancel & drop other future]
Win2 -.-> Drop
Analogy: A drag race. Winner takes all, loser gets scrapped mid-race.
When select! drops the losing future, that future's destructor runs immediately. If it was halfway through writing to a database, sending a network packet, or holding a lock — that work is abandoned mid-flight.
// DANGEROUS: Not cancellation-safe
tokio::select! {
_ = write_to_database(&data) => {},
_ = timeout => {}, // If timeout wins, write_to_database is dropped MID-WRITE
}
What makes something cancellation-safe?
stream.read().await)sleep, timeoutrx.recv().await)mutex.lock().await) — acquiring is safe, holding across .await is notWhat is NOT cancellation-safe?
Drop that must runSolutions:
use tokio::sync::oneshot;
let (tx, rx) = oneshot::channel();
tokio::spawn(async move {
let _ = write_to_database(&data).await;
let _ = tx.send(()); // Signal completion
});
tokio::select! {
_ = rx => println!("Write completed"),
_ = timeout => println!("Timed out, but write continues in background"),
}
tokio::select! with biased for priority (but still drops losers):tokio::select! {
biased; // Poll arms in order, first ready wins
result = critical_operation() => { ... }
_ = less_important() => { ... }
}
tokio::spawntokio::spawn(async {
log_event(data).await; // Independent background work
});
What happens: Creates a new task with its own lifecycle. The spawner doesn't wait.
sequenceDiagram
participant Main as Main Task
participant RT as Runtime
participant NewTask as New Spawned Task
Main->>RT: spawn(async { ... })
RT->>NewTask: Create task, push to queue
RT-->>Main: Returns JoinHandle immediately
Main->>Main: Continues WITHOUT waiting
RT->>NewTask: Runs independently on worker pool
Analogy: Delegating a side quest to an NPC while you continue the main storyline.
JoinHandle: Returns a JoinHandle that lets you await the result later if needed — or drop it to detach completely.
let handle = tokio::spawn(async {
expensive_computation().await
});
// ... do other work ...
let result = handle.await?; // Wait for result when ready
When to use: Truly independent background work (logging, metrics, fire-and-forget notifications).
When NOT to use: When you need the result soon — prefer join! for structured concurrency.
async fn fetch_dashboard_data(user_id: u64) -> Result<Dashboard> {
// Start all independent fetches concurrently
let (user_fut, posts_fut, notifications_fut, settings_fut) = (
fetch_user(user_id),
fetch_posts(user_id),
fetch_notifications(user_id),
fetch_settings(user_id),
);
// Race: user is critical, others have timeouts
let user = tokio::select! {
u = user_fut => u?,
_ = sleep(Duration::from_secs(2)) => return Err(Error::UserTimeout),
};
// Others: concurrent with individual timeouts
let (posts, notifications, settings) = tokio::join!(
async { timeout(Duration::from_secs(5), posts_fut).await },
async { timeout(Duration::from_secs(3), notifications_fut).await },
async { timeout(Duration::from_secs(1), settings_fut).await },
);
Ok(Dashboard { user, posts: posts?, notifications: notifications?, settings: settings? })
}
poll Semantics DeeplyThe poll method is where all the magic happens. Let's understand it at a deeper level:
trait Future {
type Output;
fn poll(self: Pin<&mut Self>, cx: &mut Context<'_>) -> Poll<Self::Output>;
}
Key insight: poll is not a blocking call. It must return quickly — either Ready with the result, or Pending with the waker registered. If you do heavy work inside poll, you block the entire runtime.
The Context and Waker relationship:
struct Context<'a> {
waker: &'a Waker,
}
impl<'a> Context<'a> {
fn from_waker(waker: &'a Waker) -> Self { ... }
fn waker(&self) -> &'a Waker { &self.waker }
}
The Context carries the waker — the future's handle to say "I'm ready, come poll me again." This is how async I/O works:
// Inside an async read implementation (simplified)
fn poll_read(self: Pin<&mut Self>, cx: &mut Context<'_>, buf: &mut [u8]) -> Poll<IoResult<usize>> {
// Check if data already in internal buffer
if let Some(n) = self.buffer.read(buf) {
return Poll::Ready(Ok(n));
}
// No data - register waker with reactor
self.reactor.register_read(self.fd, cx.waker().clone());
// Return Pending - we'll be woken when data arrives
Poll::Pending
}
The waker is cloned and stored by the reactor. When the OS says "data ready on fd 42", the reactor calls waker.wake(), which ultimately calls schedule(task) on the scheduler.
Async tasks run on a small pool of worker threads (typically one per CPU core). If you run synchronous, CPU-heavy work in an async task, you monopolize that thread:
async fn bad_example() {
let hash = compute_expensive_hash(data); // SYNC, CPU-bound!
// During this time, the worker thread CANNOT run other tasks
// All other tasks on this thread are STARVED
}
flowchart TD
Worker[Worker Thread] --> Task1[Task 1 compute_hash]
Task1 -->|Blocks 500ms| Worker
Worker -->|Task 2 fetch_user| Task2
Worker -->|Task 3 db_query| Task3
Worker -->|Task 4 websocket_msg| Task4
It's like a single-lane road where a tractor (heavy sync work) blocks all the bicycles (async I/O tasks) behind it.
spawn_blockingasync fn good_example() {
let hash = tokio::spawn_blocking(move || {
compute_expensive_hash(data) // Runs on BLOCKING thread pool
}).await?;
// Async thread was FREE the whole time
}
flowchart TD
AsyncTask[Async Task on Worker Thread] --> SpawnBlock[spawn_blocking]
SpawnBlock --> BlockingPool[Blocking Thread Pool<br>Separate from worker pool]
BlockingPool --> HeavyWork[compute_expensive_hash]
HeavyWork --> Result[Result sent back via channel]
Result --> AsyncTask[Async Task resumes]
NoteNote[Worker thread ran OTHER tasks<br>while blocking work happened]
AsyncTask --> NoteNote
spawn_blocking WorksJoinHandle — async task .awaits itspawn_blocking = dedicated truck lane on the side — bikes zip by unhinderedspawn_blocking| Use Case | Example |
|---|---|
| CPU-intensive crypto | blake3::hash(), ring::digest, TLS handshakes |
| Image/video processing | image::imageops, ffmpeg bindings |
| Synchronous libraries | Legacy DB drivers, rusqlite, sled |
| File I/O on platforms without async FS | Windows file ops (though tokio::fs exists) |
| Heavy serialization | serde_json::to_vec on huge structs, bincode |
| Scenario | Why |
|---|---|
| Regular async I/O | Already non-blocking (network, tokio::fs, timers) |
| Quick calculations | Overhead of thread handoff > compute time |
Holding locks across .await |
Deadlock risk — use async mutex instead |
tokio::runtime::Builder::new_multi_thread()
.max_blocking_threads(1000) // Default: 512
.build()
For CPU-bound workloads, set to num_cpus. For mixed I/O + CPU, higher is fine (threads park when idle).
How the blocking pool works internally:
ThreadPool from the main worker poolrecv() when idlemax_blocking_threads for spawn_blocking with LifoSlot optimization// Conceptual blocking pool worker
fn blocking_worker(receiver: Receiver<BlockingJob>) {
while let Ok(job) = receiver.recv() {
let result = (job.closure)();
job.sender.send(result).ok(); // Wake the async task
}
}
One of Rust's best async features is structured concurrency: tasks are scoped, and you can't accidentally leak them.
spawn Aloneasync fn leaky() {
tokio::spawn(async { loop { do_work().await } }); // Fire and forget
tokio::spawn(async { loop { do_other_work().await } });
// Function returns, but tasks keep running FOREVER
// No way to cancel, await, or handle errors
}
JoinSet — Scoped Task Groupsuse tokio::task::JoinSet;
async fn fetch_all_users(ids: Vec<u64>) -> Result<Vec<User>> {
let mut set = JoinSet::new();
for id in ids {
set.spawn(async move {
fetch_user(id).await
});
}
let mut users = Vec::new();
while let Some(result) = set.join_next().await {
users.push(result?); // Propagates errors, panics
}
Ok(users)
}
What happens: All tasks are tracked in the JoinSet. When the set is dropped (goes out of scope), any remaining tasks are cancelled automatically. No orphaned tasks.
flowchart TD
Scope[Function Scope] --> JoinSet[JoinSet]
JoinSet --> Task1[Task 1]
JoinSet --> Task2[Task 2]
JoinSet --> TaskN[Task N]
Task1 -.->|join_next| Results[Results/Errors]
Task2 -.->|join_next| Results
TaskN -.->|join_next| Results
Scope -.->|Drop| Cancel[Auto-cancel remaining]
JoinSet vs Raw spawn| Feature | spawn |
JoinSet |
|---|---|---|
| Auto-cancel on drop | ❌ | ✅ |
| Collect results in order | Manual | join_next() |
| Error propagation | Manual | Automatic |
| Panic handling | Silent | Captured in JoinError |
| Abort all | Manual | set.abort_all() |
async fn parent() {
let child = tokio::spawn(async {
// This task is a CHILD of parent
do_work().await
});
// If parent is cancelled, child gets cancelled too
// (via Drop of JoinHandle or explicit abort)
child.await
}
This mirrors Rust's ownership model: child tasks are "owned" by their parent scope.
use tokio::signal;
use tokio::sync::broadcast;
use tokio::task::JoinSet;
async fn run_server(shutdown: broadcast::Receiver<()>) -> Result<()> {
let mut shutdown = shutdown;
let mut tasks = JoinSet::new();
// Spawn server
tasks.spawn(async {
http_server().await
});
// Spawn background workers
for i in 0..4 {
tasks.spawn(async move {
worker_loop(i).await
});
}
tokio::select! {
_ = shutdown.recv() => {
println!("Shutdown signal received");
tasks.abort_all(); // Cancel all tasks
}
result = tasks.join_next() => {
// A task finished unexpectedly
result??;
}
}
// Wait for graceful cleanup
while let Some(res) = tasks.join_next().await {
res??; // Collect any errors
}
Ok(())
}
#[tokio::main]
async fn main() -> Result<()> {
let (tx, _rx) = broadcast::channel(1);
let server = tokio::spawn(run_server(tx.subscribe()));
signal::ctrl_c().await?;
tx.send(())?; // Signal all tasks to shut down
server.await??;
Ok(())
}
Don't spawn 100,000 tasks at once. Limit concurrency to protect resources.
Why bounding matters: Without limits, a burst of requests can:
use tokio::sync::Semaphore;
use std::sync::Arc;
async fn bounded_fetch(urls: Vec<String>, max_concurrent: usize) -> Vec<Result<Response>> {
let semaphore = Arc::new(Semaphore::new(max_concurrent));
let mut handles = Vec::with_capacity(urls.len());
for url in urls {
let permit = semaphore.clone().acquire_owned().await.unwrap();
handles.push(tokio::spawn(async move {
let _permit = permit; // Held until task completes
fetch_url(&url).await
}));
}
let mut results = Vec::with_capacity(handles.len());
for handle in handles {
results.push(handle.await.unwrap());
}
results
}
flowchart TD
Sem[Semaphore: 10 permits] --> T1[Task 1]
Sem --> T2[Task 2]
Sem --> T3[Task 3]
Sem -.->|Wait| T11[Task 11]
Sem -.->|Wait| T12[Task 12]
T1 -->|Release| Sem
T2 -->|Release| Sem
T3 -->|Release| Sem
Advanced: Dynamic concurrency based on system load
use tokio::sync::Semaphore;
use std::sync::Arc;
use std::sync::atomic::{AtomicUsize, Ordering};
struct AdaptiveSemaphore {
semaphore: Semaphore,
current_load: AtomicUsize,
target_latency: Duration,
}
impl AdaptiveSemaphore {
async fn acquire(&self) -> SemaphorePermit<'_> {
// Adjust permits based on observed latency
let permit = self.semaphore.acquire().await;
AdaptivePermit { permit, adaptive: self }
}
}
use tokio::time::{timeout, Duration}; use tokio::select;
async fn fetch_with_timeout(url: &str) -> Result<Response, Error> { let fetch_fut = fetch_url(url); let timeout_fut = sleep(Duration::from_secs(10));
select! {
result = fetch_fut => result,
_ = timeout_fut => Err(Error::Timeout),
}
}
// Or use the convenience function: async fn fetch_with_timeout_v2(url: &str) -> Result<Response, Error> { timeout(Duration::from_secs(10), fetch_url(url)) .await .map_err(|_| Error::Timeout)? }
---
### Pattern 3: Retry with Exponential Backoff
```rust
use tokio::time::{sleep, Duration};
use std::time::Instant;
async fn fetch_with_retry(url: &str, max_retries: u32) -> Result<Response> {
let mut last_error = None;
let mut delay = Duration::from_millis(100);
for attempt in 0..=max_retries {
match fetch_url(url).await {
Ok(resp) => return Ok(resp),
Err(e) => {
last_error = Some(e);
if attempt < max_retries {
sleep(delay).await;
delay = delay * 2; // Exponential backoff
delay = delay.min(Duration::from_secs(30)); // Cap
}
}
}
}
Err(last_error.unwrap())
}
use tokio_stream::{Stream, StreamExt};
use tokio::sync::mpsc;
async fn process_stream<S>(mut stream: S) -> Result<()>
where
S: Stream<Item = Item> + Unpin,
{
let (tx, mut rx) = mpsc::channel::<ProcessedItem>(100); // Buffer = backpressure
// Producer task
let producer = tokio::spawn(async move {
while let Some(item) = stream.next().await {
let processed = process_item(item).await?;
if tx.send(processed).await.is_err() {
break; // Receiver dropped
}
}
Ok::<_, Error>(())
});
// Consumer
while let Some(item) = rx.recv().await {
write_to_db(item).await?;
}
producer.await??;
Ok(())
}
use tokio::time::{interval, Duration};
use std::sync::Arc;
use tokio::sync::Mutex;
struct RateLimiter {
interval: Mutex<tokio::time::Interval>,
max_per_second: u32,
}
impl RateLimiter {
fn new(max_per_second: u32) -> Self {
Self {
interval: Mutex::new(interval(Duration::from_millis(1000 / max_per_second as u64))),
max_per_second,
}
}
async fn acquire(&self) {
self.interval.lock().await.tick().await;
}
}
// Usage
let limiter = Arc::new(RateLimiter::new(100)); // 100 req/s
async fn limited_fetch(url: &str) -> Result<Response> {
limiter.acquire().await;
fetch_url(url).await
}
use tokio::sync::Mutex;
use std::collections::VecDeque;
struct Pool<T> {
available: Mutex<VecDeque<T>>,
in_use: Mutex<usize>,
max_size: usize,
factory: Box<dyn Fn() -> Result<T> + Send + Sync>,
}
impl<T> Pool<T> {
async fn get(&self) -> Result<Pooled<T>> {
let mut available = self.available.lock().await;
if let Some(item) = available.pop_front() {
*self.in_use.lock().await += 1;
return Ok(Pooled { item, pool: self });
}
let mut in_use = self.in_use.lock().await;
if *in_use < self.max_size {
*in_use += 1;
drop(in_use);
let item = (self.factory)()?;
return Ok(Pooled { item, pool: self });
}
// Pool exhausted - wait for release
drop((available, in_use));
// ... wait on condition variable ...
}
}
struct Pooled<'a, T> {
item: T,
pool: &'a Pool<T>,
}
impl<T> Drop for Pooled<'_, T> {
fn drop(&mut self) {
self.pool.available.lock().await.push_back(std::mem::replace(&mut self.item, /* placeholder */));
*self.pool.in_use.lock().await -= 1;
}
}
Rust 1.75+ supports async fn in traits, but before that (and for dynamic dispatch), you need workarounds:
// Modern Rust (1.75+)
trait Database {
async fn get_user(&self, id: u64) -> Result<User>;
}
// Pre-1.75 or for dyn Trait - use async-trait crate
#[async_trait]
trait Database {
async fn get_user(&self, id: u64) -> Result<User>;
}
// Object-safe version (for dyn Database)
trait Database: Send + Sync {
fn get_user<'a>(&'a self, id: u64) -> Pin<Box<dyn Future<Output = Result<User>> + Send + 'a>>;
}
Why this matters for Tokio: When you store Box<dyn Database>, the async methods return boxed futures. Tokio handles these transparently, but there's a small allocation overhead.
Sometimes you need a custom executor for specialized workloads:
use tokio::executor::Executor;
use std::future::Future;
use std::pin::Pin;
use std::task::{Context, Poll};
struct DedicatedExecutor {
sender: crossbeam_channel::Sender<Pin<Box<dyn Future<Output = ()> + Send>>>,
// Worker thread runs loop { receiver.recv().await }
}
impl Executor for DedicatedExecutor {
fn spawn(&self, future: Pin<Box<dyn Future<Output = ()> + Send>>) -> Result<(), SpawnError> {
self.sender.send(future).map_err(|_| SpawnError::shutdown())
}
}
// Use with #[tokio::main] via custom runtime builder
tokio::runtime::Builder::new_multi_thread()
.executor(DedicatedExecutor::new())
.build()
Use cases:
Sometimes you need separate runtimes:
// Runtime 1: High-throughput HTTP server
let http_rt = tokio::runtime::Builder::new_multi_thread()
.worker_threads(8)
.thread_name("http-worker")
.build()?;
// Runtime 2: Low-latency trading engine
let trading_rt = tokio::runtime::Builder::new_multi_thread()
.worker_threads(2)
.thread_name("trading-worker")
.enable_time()
.build()?;
// Each runtime is isolated - separate thread pools, timers, reactors
http_rt.spawn(http_server());
trading_rt.spawn(trading_engine());
Caution: Don't mix runtimes in the same async call chain. Pass data between them via channels.
For libraries that only offer sync APIs:
// Wrapper pattern for sync libraries
struct AsyncWrapper<T> {
inner: T,
blocking_pool: tokio::task::JoinHandle<()>, // or use spawn_blocking per-call
}
impl<T> AsyncWrapper<T> {
async fn call<F, R>(&self, f: F) -> R
where
F: FnOnce(&T) -> R + Send + 'static,
R: Send + 'static,
T: Sync,
{
let inner = &self.inner;
tokio::task::spawn_blocking(move || f(inner))
.await
.unwrap()
}
}
// Usage
let db = AsyncWrapper::new(SqliteConnection::open("db.sqlite")?);
let users = db.call(|conn| conn.query("SELECT * FROM users")).await?;
Advanced: Batched async wrapper for throughput
use tokio::sync::mpsc;
use std::sync::Arc;
struct BatchedAsyncClient<T> {
sender: mpsc::Sender<BatchRequest<T>>,
}
struct BatchRequest<T> {
items: Vec<T>,
responder: oneshot::Sender<Vec<Result<R>>>,
}
impl<T, R> BatchedAsyncClient<T>
where
T: Send + 'static,
R: Send + 'static,
{
fn new<F>(batch_size: usize, batch_timeout: Duration, process_fn: F) -> Self
where
F: Fn(Vec<T>) -> Vec<Result<R>> + Send + 'static,
{
let (tx, mut rx) = mpsc::channel::<BatchRequest<T>>(100);
tokio::spawn(async move {
let mut buffer = Vec::with_capacity(batch_size);
let mut interval = tokio::time::interval(batch_timeout);
loop {
tokio::select! {
biased;
Some(req) = rx.recv() => {
buffer.push(req);
if buffer.len() >= batch_size {
Self::process_batch(&mut buffer, &process_fn);
}
}
_ = interval.tick() => {
if !buffer.is_empty() {
Self::process_batch(&mut buffer, &process_fn);
}
}
}
}
});
Self { sender: tx }
}
fn process_batch<F>(buffer: &mut Vec<BatchRequest<T>>, process_fn: &F)
where
F: Fn(Vec<T>) -> Vec<Result<R>>,
{
let items: Vec<T> = buffer.drain(..).flat_map(|r| r.items).collect();
let results = process_fn(items);
// Distribute results back to responders...
}
pub async fn call(&self, item: T) -> Result<R> {
let (tx, rx) = oneshot::channel();
self.sender.send(BatchRequest { items: vec![item], responder: tx }).await.ok();
rx.await.unwrap().into_iter().next().unwrap()
}
}
tokio-console — Live Runtime Inspection# Install
cargo install tokio-console
# Run your app with console enabled
TOKIO_CONSOLE_BIND=0.0.0.0:6669 cargo run
# In another terminal
tokio-console
Shows live:
flowchart TD
App[Your App<br/>tokio::spawn + console_subscriber]
Console[tokio-console CLI]
App -->|gRPC| Console
Console -->|Dashboard| User[Engineer]
Enable in your app:
// Cargo.toml
tokio = { version = "1", features = ["rt-multi-thread", "macros", "sync", "time", "tracing"] }
console-subscriber = "0.6"
console-api = "0.2"
// main.rs
use console_subscriber::ConsoleLayer;
fn main() {
tracing_subscriber::registry()
.with(ConsoleLayer::builder().spawn())
.init();
// Your app...
}
What tokio-console reveals that logs can't:
tracing — Structured Logging with Spansuse tracing::{info, instrument, span, Level};
#[instrument]
async fn fetch_user(id: u64) -> Result<User> {
info!("Fetching user from DB");
let span = span!(Level::DEBUG, "db_query", user_id = id);
let _enter = span.enter();
let user = db.get_user(id).await?;
info!(user.id, user.name, "User fetched");
Ok(user)
}
// Output:
// 2024-01-15T10:30:00.123Z INFO my_app: fetch_user: Fetching user from DB user_id=42
// 2024-01-15T10:30:00.145Z DEBUG my_app: db_query: query executed user_id=42 duration_ms=22
// 2024-01-15T10:30:00.145Z INFO my_app: fetch_user: User fetched user_id=42 user_name="Alice"
Why spans matter: They show nesting and duration of async operations, letting you see which .await points are slow.
poll Debugging (for Tests)#[cfg(test)]
mod tests {
use futures::task::noop_waker;
use std::task::Context;
use std::pin::Pin;
#[tokio::test]
async fn test_future_polling() {
let mut future = Box::pin(async_operation());
let waker = noop_waker();
let mut cx = Context::from_waker(&waker);
// First poll - should return Pending (registers waker)
let polled = future.as_mut().poll(&mut cx);
assert!(polled.is_pending());
// Simulate wake (in real test, you'd trigger the actual event)
// ...
// Second poll - should return Ready
let polled = future.as_mut().poll(&mut cx);
assert!(polled.is_ready());
}
}
// Add to your test suite
#[tokio::test]
async fn test_no_blocking_in_async() {
// This will panic if sync code blocks
tokio::spawn(async {
// If this contains blocking code, the test times out
tokio::time::timeout(Duration::from_secs(1), suspect_function())
.await
.expect("Function blocked!");
}).await.unwrap();
}
Better: Use tokio's blocking feature with #[tokio::test] which has built-in blocking detection.
| Symptom | Likely Cause | Diagnosis |
|---|---|---|
| High CPU, low throughput | Blocking in async task | tokio-console shows running tasks with no I/O |
| Tasks stuck in "parked" | Missing wake / deadlock | Check reactor registration, waker storage |
| Memory growing unbounded | Tasks not completing / leaking | JoinSet not joined, channels not closed |
| Uneven core usage | Work stealing not working | Check local queue depths in tokio-console |
| Latency spikes | Global queue contention | Too many spawns, consider batching |
| Increasing task count | Task leak (forgotten JoinHandle) | Audit all spawn sites, use JoinSet |
| Reactor not waking tasks | Missing interest registration | Verify Mio registration, check waker clone |
| Deadlock on mutex | Sync mutex held across .await |
Use tokio::sync::Mutex instead |
When things go wrong, work through this list:
tracing with trace level — see every poll/waketokio-console — visualize runtime stategrep for .blocking_ or sync I/OSend bounds — compiler errors point to non-Send capturesselect! arms — cancellation safety on all losers| Workload | Why Async Wins | Typical Speedup |
|---|---|---|
| High-concurrency HTTP servers | 10K+ connections, mostly waiting on I/O | 10-100× vs threads |
| WebSocket/chat servers | Long-lived connections, sporadic messages | Essential (thread model impossible) |
| Database connection pooling | Many queries, network round-trips | 5-20× |
| Microservice orchestration | Fan-out to multiple services, wait for all | Matches fan-out factor |
| Workload | Consider Sync Instead |
|---|---|
| CPU-bound batch processing | Rayon parallel iterators easier |
| Simple CLI tools | No concurrency needed |
| Low-concurrency services (<100 conn) | Overhead not worth it |
| Heavy numerical computing | Use dedicated compute frameworks |
Every async operation has overhead:
// Overhead comparison (rough estimates)
sync_fn() // ~1-5 ns (direct call)
async_fn().await // ~50-200 ns (poll + scheduler)
spawn(async {}).await // ~1-5 µs (queue + wake + reschedule)
Rule: Don't async everything. Use sync for hot paths, async for I/O boundaries.
Latency: Time for ONE request to complete
Throughput: Requests completed PER SECOND
Async improves throughput (more concurrent requests) but can add latency (scheduler overhead). For p99 latency-sensitive workloads, consider:
// Dedicated runtime for latency-critical path
let rt = tokio::runtime::Builder::new_multi_thread()
.worker_threads(2) // Fewer threads = less contention
.enable_time()
.build()
.unwrap();
// Pin critical tasks to specific workers using thread-local storage
rt.spawn(async {
// This runs on one of the 2 dedicated threads
latency_critical_operation().await
});
Why fewer threads can reduce latency: With fewer worker threads, there's less contention on the global queue, less cache thrashing, and more predictable scheduling. The tradeoff is reduced total throughput.
Beyond the obvious overhead, async has subtle costs:
| Cost | Description | Mitigation |
|---|---|---|
| State machine bloat | Each .await adds a variant; large futures = more memory |
Split large async fns, use Box::pin for recursion |
| Vtable indirection | dyn Future requires double indirection |
Prefer concrete types, impl Future return |
| Waker allocation | Each task needs a waker (Arc internally) | Reuse tasks where possible |
| Cache misses | Task state scattered in memory | Local queues help, but cross-thread moves hurt |
| False sharing | Adjacent tasks in queue on different threads | Rust's allocator helps, but still a factor |
Tokio's own benchmarks: 150,000+ HTTP requests/second on a single thread.
This is possible because:
But your code adds overhead. Profile before assuming.
# CPU profiling
cargo install flamegraph
cargo flamegraph --bin my_app
# Or with perf
perf record --call-graph=dwarf ./my_app
perf report
# Memory profiling
cargo install dhat-heap
./my_app # with #[global_allocator] = dhat::DhatAlloc
# Async-specific: tokio-metrics
# https://github.com/tokio-rs/tokio-metrics
Key metrics to watch:
poll_duration histogram (per task type)queue_depth (per worker)wake_latency (time from wake to poll)task_count (should stabilize, not grow indefinitely)blocking_pool_queue_depth (should be near zero)Here's how to add instrumentation to your Tokio app:
use tokio_metrics::RuntimeMonitor;
use tokio::runtime::Runtime;
fn main() {
let rt = Runtime::new().unwrap();
let monitor = RuntimeMonitor::new(rt.handle().clone());
// Spawn a task that logs metrics every second
rt.spawn(async move {
let mut interval = tokio::time::interval(Duration::from_secs(1));
loop {
interval.tick().await;
let metrics = monitor.poll();
println!("workers: {}, queue_depth: {}, blocking: {}",
metrics.num_workers,
metrics.remote_queue_depth,
metrics.blocking_pool_size);
}
});
// ... your app ...
}
Sample output interpretation:
workers: 8, queue_depth: 12, blocking: 0 // Healthy - work distributed
workers: 8, queue_depth: 500, blocking: 45 // BACKPRESSURE - too much work
workers: 8, queue_depth: 0, blocking: 0 // Idle - no work
// ❌ BAD - blocks worker thread
async fn hash_data(data: &[u8]) -> Hash {
blake3::hash(data) // CPU-intensive sync call!
}
// ✅ GOOD - offloads to blocking pool
async fn hash_data(data: Vec<u8>) -> Hash {
tokio::spawn_blocking(move || blake3::hash(&data))
.await
.unwrap()
}
// ❌ DANGEROUS - write may be half-done when cancelled
tokio::select! {
_ = write_to_db(&data) => {},
_ = timeout => {},
}
// ✅ SAFE - write runs to completion in background
let (tx, rx) = oneshot::channel();
tokio::spawn(async move {
write_to_db(&data).await;
let _ = tx.send(());
});
tokio::select! {
_ = rx => {},
_ = timeout => {},
}
// ❌ BAD - OOM risk with 1M URLs
for url in urls {
tokio::spawn(fetch(url));
}
// ✅ GOOD - bounded with semaphore
let sem = Arc::new(Semaphore::new(100));
for url in urls {
let permit = sem.clone().acquire_owned().await.unwrap();
tokio::spawn(async move {
let _ = permit;
fetch(url).await
});
}
Send Bounds// ❌ FAILS - Rc not Send
tokio::spawn(async {
let rc = Rc::new(data);
do_async().await;
use(rc);
});
// ✅ WORKS - use Arc (thread-safe refcount)
tokio::spawn(async {
let arc = Arc::new(data);
do_async().await;
use(arc);
});
// ✅ ALSO WORKS - single-threaded runtime
#[tokio::main(flavor = "current_thread")]
async fn main() {
tokio::task::spawn_local(async {
let rc = Rc::new(data);
do_async().await;
use(rc);
}).await;
}
// ❌ DISASTER - two separate thread pools, reactors, timers
#[tokio::main]
async fn main() {
let rt = async_std::task::spawn(async { ... }); // Different runtime!
}
Pick one runtime per process. If you need async-std compat, use tokio's compat module.
JoinHandle in Async Fn// ❌ DEADLOCK RISK - blocks worker waiting for... worker
async fn bad() {
let handle = tokio::spawn(work());
handle.await; // If work() needs a worker, DEADLOCK
}
// ✅ GOOD - use join! for concurrent waiting
async fn good() {
let (a, b) = tokio::join!(work_a(), work_b());
}
// ✅ ALSO GOOD - channel for result
async fn good2() {
let (tx, rx) = oneshot::channel();
tokio::spawn(async move {
let result = work().await;
let _ = tx.send(result);
});
rx.await.unwrap()
}
.await Futures// ❌ SILENT BUG - future never runs!
async fn broken() {
fetch_user(42); // Returns future, immediately dropped
println!("User fetched!"); // Never actually fetched
}
// ✅ CORRECT
async fn fixed() {
let user = fetch_user(42).await; // Actually drives the future
println!("User fetched: {:?}", user);
}
// Rust compiler warns about unused futures, but only if the return
// type implements `Future` and isn't `()`. Easy to miss with custom types.
? with .await Incorrectly// ❌ WRONG - ? binds tighter than .await
async fn wrong() -> Result<User> {
fetch_user(42).await?; // Parses as (fetch_user(42).await)?
// If fetch_user returns Result, this works but is confusing
}
// ✅ CLEARER - explicit grouping
async fn clear() -> Result<User> {
let user = fetch_user(42).await?;
Ok(user)
}
// ⚠️ TRICKY: when the future itself returns Result
async fn tricky() -> Result<User> {
// fetch_user returns Result<User, Error>, not Future<Output=Result<...>>
// This is fine:
fetch_user(42).await?
// But if you have a Future<Output=Result<...>>:
let fut = async { Ok::<User, Error>(User::new()) };
fut.await?; // Works
}
| Concept | Mental Model | Practical Implication |
|---|---|---|
| Future | Recipe / to-do list | Lazy — does nothing until driven |
| Task | Work order in kitchen | Spawn to run; ~64 bytes each |
| Scheduler | Traffic controller + work stealing | Keeps all cores busy automatically |
| Local Queue | Chef's personal prep station | Lock-free, cache-friendly, fast |
| Global Queue | Overflow pantry | Fallback when local is full |
| Reactor | Dispatcher talking to OS | Bridges async tasks to epoll/kqueue/IOCP |
| Executor | Cook following recipe | Calls poll() repeatedly |
| Timer Wheel | Hierarchical clock | O(1) timers at scale |
.await |
"Pause here, resume later" | Yields control, allows concurrency |
join! |
Cook multiple pots | Concurrent within one task |
select! |
Drag race | First wins, others cancelled |
spawn |
Delegate to NPC | Independent task lifecycle |
spawn_blocking |
Truck on side road | Offload CPU work, don't block bikes |
JoinSet |
Scoped task group | Auto-cancel on drop, collect results |
| Semaphore | Bouncer at club | Limit concurrent access |
| Waker | "Tap on shoulder" | How runtime knows to re-poll |
runtime/scheduler/ and runtime/io/tokio-console: https://github.com/tokio-rs/console — Live debuggingtokio-metrics: https://github.com/tokio-rs/tokio-metrics — Runtime instrumentationtracing: https://docs.rs/tracing — Structured diagnosticsAsync Rust feels verbose compared to Go's go func() or JavaScript's await. But that verbosity is honesty. You're not hiding concurrency behind a runtime you can't see — you're explicitly choosing:
You own the model. Like manual memory management but for concurrency. Once the mental model clicks, you stop fighting the compiler and start building systems that scale predictably.
The next time you write #[tokio::main], you'll know exactly what seven components just started up — and why your code runs fast.
Keep this handy while coding:
┌─────────────────────────────────────────────────────────────────┐
│ ASYNC RUST DECISION TREE │
├─────────────────────────────────────────────────────────────────┤
│ │
│ Need to run something concurrently? │
│ │ │
│ ▼ │
│ ┌─────────┐ │
│ │ Is it │──Yes──▶ tokio::spawn (fire & forget) │
│ │ independ│ │
│ │ ent? │───No──▶ Need ALL results? │
│ └─────────┘ │ │
│ ▼ │
│ ┌─────────┐ │
│ │ Need │──Yes──▶ tokio::join! (concurrent) │
│ │ fastest │ │
│ │ result? │───No──▶ tokio::select! (race) │
│ └─────────┘ │
│ │
│ Doing CPU-heavy work? │
│ │ │
│ ▼ │
│ ┌─────────────────────────────────────────────────────────┐ │
│ │ tokio::spawn_blocking(move || heavy_work()) │ │
│ │ │ │
│ │ NOT: async { heavy_work() }.await ← BLOCKS RUNTIME! │ │
│ └─────────────────────────────────────────────────────────┘ │
│ │
│ Need to limit concurrency? │
│ │ │
│ ▼ │
│ tokio::sync::Semaphore::new(N) // N = max concurrent │
│ │
│ Need timeout? │
│ │ │
│ ▼ │
│ tokio::time::timeout(Duration::from_secs(5), future).await │
│ │
│ Spawning many tasks? │
│ │ │
│ ▼ │
│ tokio::task::JoinSet::new() // Auto-cancels on drop │
│ │
└─────────────────────────────────────────────────────────────────┘
If you've made it this far, you now have a mental model that puts you ahead of 90% of async Rust practitioners. The key insights to carry forward:
spawn_blocking, never in async — the single most important ruleJoinSet, semaphores, timeouts prevent disasterstokio-console and tracing are your eyes insideThe next time you write #[tokio::main], you'll know exactly what seven components just started up — and why your code runs fast.