A million-token context window is not a million tokens of usable memory. Attention dilutes, costs scale, latency grows, and — the measurable part — models reason measurably worse over stuffed contexts. Context economics is the discipline of spending window budget like the scarce resource it is.
The empirical record is unambiguous. The "lost in the middle" studies (Liu et al., 2023) showed retrieval accuracy peaking at the edges of the window and sagging in the middle — models reliably use what is at the start and end of context and under-use what is buried. LongBench-style evaluations show effective reasoning degrading well before the advertised limit. And every stuffed token bills twice: once in price, once in latency, since transformer inference scales with context length. "Just put it all in the prompt" is the strategy teams outgrow first.
Treating the window as a budget with line items changes the design conversation. The system prompt and tool schemas are fixed overhead — keep them tight, they bill on every call. Retrieved evidence is the variable spend — rank ruthlessly, admit only what clears a relevance bar. History is the compounding liability — summarize or evict it deliberately. Scratch space is the investment — a well-maintained notes file displaces thousands of tokens of raw transcript.
Budget Rules of Thumb
- Order matters: instructions and critical evidence at the edges of the window; reference material in the middle where attention is weakest.
- Measured fill ratio: track utilization — what fraction of admitted context the model actually cites. A 40% utilization at 100k tokens is a retrieval problem, not a context-size solution.
- Summarize at the boundary: compact history into structured state at natural task seams, not mid-task — mid-run compaction corrupts in-flight reasoning.
- Price the alternative: before widening the window, price the retrieval tier that would fit the same information in 5% of the tokens.
The discipline mirrors memory management in operating systems for a reason — same constraints, different substrate. The engineers who thrive will be the ones who treat the context window the way good systems people treat RAM: as a cache over a well-organized store, never as the store itself.