8.5 · stdin, stdout, stderr: Standard Streams

Domain 8 — I/O System Duration: ~15 minutes Library components: std::io::stdin, std::io::stdout, std::io::stderr, std::io::Stdin, std::io::Stdout, std::io::Stderr, print!, println!, eprint!, eprintln!

Introduction

Each process starts with three streams. Rust gives each stream a global, thread-safe handle: io::stdin(), io::stdout(), and io::stderr(). The handles look simple, and the first println! of each Rust programmer uses one. But the handles contain three design decisions:

• A locking model: each access synchronizes on a global lock.
• A buffering policy: stdout is line-buffered, and stderr is unbuffered.
• A failure mode: println! panics when stdout is a closed pipe.

You must know all three to write a CLI tool that behaves correctly.

This tutorial shows:

• The three handles, their lock types (StdinLock, StdoutLock), and why it is important to hold a lock during a burst of writes.
• print!/println!/eprint!/eprintln! and their contract: they panic on failure.
• The buffering differences between stdout and stderr, and when to call flush.
• Broken pipes: why println! panics under | head, and the graceful-exit pattern.
• The testable-CLI pattern: how to write the program core against the BufRead and Write traits.

Three Global Streams and Their Locks

Each handle gives access to a resource that is global to the process, and a lock protects that resource. Stdout also wraps its output in a LineWriter (tutorial 8.1), which is the cause of the line buffering. Since Rust 1.61, the lock guards are 'static. Thus io::stdout().lock() is valid as one expression, and you do not need a separate binding for the handle.

Figure: The standard streams' locking and buffering model

The macros use the same mechanism. On each call, println! acquires the stdout lock, formats, writes, and releases the lock. For a burst of output, this has two disadvantages: an overhead on each call, and a risk of interleaved output. A line from a different thread can appear between any two of your calls. The usual pattern is to hold the lock:

use std::io;
use std::io::Write;   // necessary for writeln! on the lock

// `?` needs a function that returns io::Result.
let mut out = io::stdout().lock();          // StdoutLock<'static>
for row in 1..=3u32 {
    // No other thread can write to stdout between these lines.
    writeln!(out, "report row {row}: value={}", row * 100)?;
}
// `out` drops at the end of its scope, and the drop releases the lock.

Two more details are important. First, the stdout lock is reentrant: a println! from the same thread while you hold the lock does not deadlock. (A different thread blocks.) Second, StdinLock implements BufRead, so you can call read_line and lines (tutorial 8.1) directly on it.

The Print Macros and Their Contract

The four macros write to the two output streams:

The macros are convenient, but they have a disadvantage. They return (), so they cannot report a failed write. Instead, they panic with a message such as failed printing to stdout: Broken pipe (os error 32). Some programs have their output as their product (each program that you can use in a pipeline). For those programs, that panic is a real category of bug. Slide 5 shows how to handle it.

The general rule to select a stream has two parts. stdout is for the payload: the data that the user requested, and that the subsequent pipeline stage reads. stderr is for commentary: progress, warnings, and diagnostics. cargo obeys this rule. Its build messages go to stderr, so cargo run --quiet | jq sends only the program output to jq.

A read from stdin blocks until input arrives, so the examples in this domain never read it. Comments in the examples show the connection (io::stdin().lock() as a BufRead). The pattern on slide 6 lets you test the read code with no keyboard.

Buffering: stdout vs. stderr

The two output streams make opposite trade-offs between latency and throughput:

• stdout is line-buffered. Bytes collect in an internal LineWriter. The LineWriter writes them to the OS when a \n arrives, on flush, or when the buffer is full. This is good for throughput, but the result can surprise you when you print a partial line.
• stderr is unbuffered. Each write goes directly to the OS. Diagnostics arrive even if the process crashes immediately after the write. An error channel needs this behavior.

The typical symptom is a prompt that does not appear.

// This snippet needs the same imports as the previous snippet.
let mut out = io::stdout().lock();
write!(out, "progress: ")?;   // no newline: the text stays in the buffer
out.flush()?;                 // now the text is visible
for step in ["25%", "50%", "75%", "100%"] {
    write!(out, "{step} ")?;
    out.flush()?;             // makes each increment visible immediately
}
writeln!(out)?;               // the newline ends the line and flushes it

The strict form of the rule is as follows. The documentation promises line buffering only when stdout is a terminal. The standard library may use block buffering for pipes in a future release. Code that needs a partial line to be visible immediately must call flush. That call is correct with each buffering policy.

08_14_stdout_stderr_locking.rs prints these lines. The three [diag] lines go to stderr:

one println!, one lock/unlock round-trip
a second println!, a second round-trip
report row 1: value=100
report row 2: value=200
report row 3: value=300
println! while holding the lock: fine on the same thread
progress: 25% 50% 75% 100%
[diag] this line goes to stderr, unbuffered   # (stderr: the position relative to stdout can vary)
[diag] stderr lock held for a two-line burst   # (stderr)
[diag] no flush needed — stderr writes go straight out   # (stderr)
stdin().lock() is BufRead — reading skipped to stay non-interactive

All assertions passed.

Broken Pipes: Why println! Panics

Run yourtool | head -3. head exits after three lines, and its exit closes the read end of the pipe. The subsequent write to stdout fails with ErrorKind::BrokenPipe. If that write came from println!, the process panics.

On Unix, the startup code of Rust sets SIGPIPE to ignored. Thus the OS does not kill the process silently, which is the default for shell tools. Instead, the failure arrives as a regular io::Error. println! cannot return the error, so it panics.

A closed pipe in the middle of a pipeline is not an error. It means that the consumer has all the data that it wants. The solution has two parts: write through fallible calls, and filter the one harmless kind:

// Writes `total` lines to `out`. Returns the number of lines that it wrote.
fn stream_lines<W: Write>(out: &mut W, total: u32) -> io::Result<u32> {
    let mut written = 0u32;
    for n in 1..=total {
        writeln!(out, "result line {n}")?;   // a failed write is an Err value, not a panic
        written += 1;
    }
    Ok(written)
}

// The filter: BrokenPipe is a normal end, and each other error is a failure.
fn run<W: Write>(out: &mut W) -> io::Result<()> {
    match stream_lines(out, 100) {
        Ok(_) => Ok(()),
        Err(e) if e.kind() == ErrorKind::BrokenPipe => Ok(()),   // the consumer closed the pipe
        Err(e) => Err(e),                                        // a real failure: propagate it
    }
}

// With real output, pass the locked stdout: run(&mut io::stdout().lock())

A real broken pipe needs a second process. Thus example 08_16 reproduces the failure deterministically with a mock writer. The mock writer accepts 45 bytes, and then each write returns BrokenPipe. The example proves two facts. The failure occurs on the fourth writeln! call. The filter accepts only BrokenPipe: a StorageFull error continues to propagate.

CLI tools in production, such as ripgrep, use this same graceful-exit pattern.

The Testable-CLI Pattern

All the topics of this domain lead to one structural decision. The core of the program should take impl BufRead and impl Write parameters. It should not use the global streams directly.

// Copies the lines of `input` to `output` and puts a number before each non-empty line.
// Returns the count of numbered lines.
fn number_lines<R: BufRead, W: Write>(input: R, mut output: W) -> io::Result<u32> {
    let mut numbered = 0u32;
    for line in input.lines() {
        let line = line?;                 // line: String, with no newline at the end
        if line.is_empty() {
            writeln!(output)?;            // an empty line gets no number
        } else {
            numbered += 1;
            writeln!(output, "{numbered:>4}  {line}")?;   // for example "   1  alpha"
        }
    }
    output.flush()?;                      // sends buffered bytes to the destination
    Ok(numbered)
}

The same function then operates in each configuration:

• Tests: call number_lines(&b"alpha\n\nbeta\n"[..], &mut Vec::new()). You can assert the exact bytes, and you do not start a process. The in-memory types from tutorial 8.2 are valid arguments, and io::empty() and io::sink() cover edge cases.
• Production: call number_lines(stdin.lock(), stdout.lock()). The locks are the trait implementations that you need. Because you hold them for the full run, you also get the burst-lock advantage from slide 2 at no cost.

This pattern is not a feature of the standard library. It is the design pattern that the trait hierarchy exists to make possible. It is also the reason why tutorials 8.1–8.4 told you to write code against traits.

Summary

Code Examples