More thinking is not monotonic. Past a task-specific point, additional deliberation makes models worse — confident answers get second-guessed into errors, simple tasks get elaborate scaffolds that collapse, and latency burns while accuracy declines. Overthinking is a measured failure mode, and managing it is an operations discipline.
The evidence arrived alongside reasoning models. Apple's 2024 "illusion of thinking" study showed accuracy collapsing to zero beyond a complexity threshold even with unlimited thinking tokens — the models increasingly "reasoned" themselves off correct solutions. Papers on "thought degradation" documented long chains circling back to earlier wrong drafts; practitioners watched reasoning models flip correct answers after thousands of tokens of second-guessing. The mechanism is intuitive once stated: deliberation is a random walk around a belief, and given enough steps, the walk wanders away from a correct starting point as often as it wanders toward a correction of an incorrect one — unless the walk itself has verification built in.
The engineering response treats deliberation as a budgeted, observable resource rather than a virtue. Simple classes of queries should be routed away from heavy thinking — the model that agonizes over "what is the capital of France" is not being thorough. Escalation belongs to evidence: disagreement across samples, failed checks, or user flags — not to a default "think harder."
Symptoms and Countermeasures
- Answer flipping: the final answer differs from an intermediate correct one — snapshot reasoning state and prefer early-exit when intermediate checks pass.
- Scaffold inflation: elaborate multi-step plans for trivial inputs — route by difficulty first; most queries do not deserve a plan.
- Confidence decay: hedging language grows with thinking length while accuracy does not — measure answer stability across budget tiers and pick the knee of the curve.
- Loop signature: near-duplicate reasoning steps with no new evidence — a stall detector should cut the run and return the best verified candidate.
The uncomfortable summary: knowing when not to think is becoming a distinct capability from thinking well, and the systems that ship both will beat the systems that ship only depth.