Chain-of-thought follows a single line of reasoning; Tree of Thoughts explores many, evaluates them mid-flight, prunes the dead ends, and backtracks when a branch collapses. It turns inference into search.

The technique (Yao et al., 2023) reframes a reasoning problem as traversal over a tree whose nodes are coherent "thoughts" — paragraphs, plans, or equation blocks, not single tokens. At each level the model expands promising nodes into candidates, a value prompt scores each candidate's promise of reaching the goal, and a classical search strategy — breadth-first, depth-first, or beam — decides what survives. On Game-of-24 style tasks requiring deliberate planning, the paper reported jump from single-prompt solve rates of ~4% to ~74% — not because the model got smarter, but because it was finally allowed to be wrong and recover.

The honest costs are quadratic: k candidates per level multiply both tokens and latency, and the value judgments are themselves model calls that can be wrong. Tree of Thoughts pays off only where mistakes are cheap to detect but expensive to make — planning, puzzles, agent strategy — and is overkill for tasks a single competent pass solves anyway.

Where the Leverage Is

• Prune aggressively: the value model matters more than the generator; a mediocre generator with a sharp critic beats the reverse.
• Fail fast: apply cheap domain checks (arithmetic, unit tests, schema validation) before expensive value judgments.
• Cache aggressively: identical thought prefixes across branches are the same computation — dedupe by prefix hash.
• Hybridize: use ToT for the strategic skeleton and ordinary CoT within each step; exploring token-level branches is burning money.

Conceptually, Tree of Thoughts closed a loop that traces to Newell and Simon: problem solving is search through a problem space. The 2023 contribution was showing that a transformer plus a classical search loop — the two architectures AI spent seventy years perfecting separately — compose into something neither is alone.