Long agent runs generate more history than any window holds, so something must be evicted. Compaction — summarizing and structuring history instead of truncating it — is the difference between an agent that runs for days and one that amnesia-crashes at hour two.
Truncation, the naive policy, fails in a characteristic way: it keeps the most recenttokens, which are full of tool output and half-finished observations, while dropping the goal, constraints, and early decisions that made them meaningful. The agent remains fluent and becomes purposeless. Structured compaction instead maintains an explicit state artifact — goal, constraints, decisions made with reasons, open questions, next steps — and rewrites it at task boundaries. The transcript becomes disposable; the state file becomes the run's continuity of consciousness.
The hierarchy matters because different layers compress differently. Raw observations compress hard — tool output older than a few steps reduces to its outcome. Decisions compress poorly and must be preserved nearly verbatim, with their reasons — a decision without its rationale gets re-litigated later, wasting the argument that settled it. Summaries themselves get summarized: a hierarchy of daily summaries over hourly summaries over raw logs, each level answerable without unfurling the one below, mirrors the log-structured storage designs every database already proved.
Compaction in Practice
- Compact at seams: task boundaries, test-green moments, natural phase changes — never mid-inference with in-flight reasoning depending on evicted content.
- Keep a verbatim core: the goal, the constraints, and user-stated preferences are pinned — the summarizer may not touch them.
- Link, don't inline: large artifacts stay in storage; the state holds pointers and short extracts.
- Test the lossiness: periodically ask the compacted agent to restate its mission and constraints; the answers are your compaction quality metric.
The framing that keeps priorities honest: compaction is not compression engineering, it is curriculum design for a successor self — you are deciding what the agent at step 200 will know, and therefore what it will be capable of.