Test-time compute is now a dial you can turn: reasoning models allocate thousands of hidden thinking tokens per problem, and "budget forcing" is the craft of controlling how many they spend — and when to force them to spend more.
Reasoning models separate their internal deliberation from their final answer. Left alone, that deliberation ends when the model feels done — which correlates poorly with when the problem is actually solved. o1-preview and DeepSeek-R1 demonstrated that scaling thinking tokens improves hard-task accuracy dramatically; the s1 paper (Muennighoff et al., 2025) showed the control side: simply appending "Wait" when a model tried to conclude — forcing it to keep deliberating — improved accuracy on competition math measurably, and setting a token budget let operators trade latency against correctness deliberately.
The economics deserve first-class treatment. Thinking tokens cost real money and real seconds, and the return on them is wildly task-dependent: a retrieval query needs almost none, an agentic debugging session can profitably spend tens of thousands. Treating budget as a global constant — always maximum, or always minimum — wastes money on the easy cases and fails the hard ones.
Operating the Dial
- Classify before you spend: a cheap router model estimates difficulty and assigns a thinking budget tier — most queries need the lowest tier.
- Escalate on evidence, not vibes: low self-consistency across samples or a failed verifier trigger the next budget tier, not blind retries at the same tier.
- Watch for the cliff: beyond a task-specific point, extra thinking plateaus or degrades — overthinking is a measured phenomenon, not a joke.
- Expose the knob: log thinking tokens per request as a first-class metric; unmanaged, it becomes the largest line item on the invoice.
The historic shift is easy to state: for seventy years, more capability meant more pretraining compute. Reasoning models made capability a runtime purchase. Budget forcing is the operations discipline for that new world — the same way autoscaling turned capacity from a procurement decision into a control loop.