Agents are streaming producers by nature: token streams from models, event streams from tools, log streams to dashboards. When any consumer falls behind, something must give. Backpressure — the discipline of propagating "slow down" upstream instead of buffering infinitely — is what separates streaming systems from leaky ones.
The unmanaged failure is always the same shape. A pipeline streams model tokens into a parser into a database; the database slows for maintenance; the parser buffers; the buffer grows; memory exhausts at 3 a.m. Every queue in the chain was "just temporary." Reactive Streams and backpressure-aware runtimes solved this for software by making demand flow upstream: consumers request n items, producers send only what was requested. The same contract belongs in agent pipelines — and agent pipelines add a twist: the ultimate producer bills by the token, so applying backpressure saves money, which is not true of most producer-consumer systems.
The agent-specific design surface is what to do when demand cannot reach the true source — you cannot tell a model mid-generation to think slower, but you can choose where buffering is honest. Bounded queues with explicit overflow policy beat unbounded queues by construction: overflow sheds load deliberately — dropping verbose trace events before dropping tool calls, coalescing UI updates, deferring non-critical persistence — versus the OOM that drops everything.
Pressure Rules
- Bound every queue: an unbounded buffer is a deferred crash with interest; size it, name its overflow policy, alert on 80%.
- Shed by value class: rank streams (user-visible > audit-critical > verbose trace) so overload degrades gracefully and predictably.
- Pause generation, not consumption: when the consumer is the bottleneck and the model supports it, stop generating — unfinished tokens cost the same as finished ones.
- Measure queue depth as SLI: latency percentiles lie during saturation; queue depth tells the truth earlier.
The reframing that makes the pattern stick: a streaming agent pipeline is not a sequence of calls but a chain of promises about rate. Backpressure is how those promises stay honest — and like most honesty, it is much cheaper before 3 a.m. than after.