What Is Tree of Thoughts Prompting?
Tree of thoughts is a way of organising a search over intermediate steps. Rather than extending one line of working to its end, an implementation generates several candidate next steps from the current state, evaluates the states they lead to, and expands only the ones worth continuing — which makes the working a tree, with branching and backtracking, instead of a single chain read from beginning to end.
The four choices every implementation makes
The method is usually set out as four independent decisions, and separating them is what turns "try several options" into something that can be built. Each can be answered differently without disturbing the others, which is why two implementations of the same idea can look nothing alike.
First, decomposition: what one thought is. A thought is a unit small enough that alternatives to it can be compared, and large enough that comparing them means something — a line of a proof, one equation in a sequence, a single placement in a puzzle, one sentence of a plan. This choice fixes the depth of the tree, and together with the branching factor it fixes what the search will cost.
Second, generation: how the candidates at a state are produced. Either several are sampled independently from the same prompt, which spreads them when the space of continuations is wide, or one call is asked to propose several distinct continuations together, which costs less and tends to return candidates that differ deliberately rather than by accident.
Third, evaluation: how a partial state is judged before it is finished. Two forms are usual — scoring each state on its own against a rubric or a short verdict such as certain, possible or ruled out, or presenting several states side by side and asking for a comparison. Evaluation is what makes this a search rather than an expensive fan-out, and it is the component that fails most quietly: a state that cannot be judged until it is complete leaves the search with nothing to prune on.
Fourth, the search rule itself. Breadth-first keeps the best few states at each level and advances them together. Depth-first follows one branch until its evaluation falls below a threshold, then returns to the most recent state that still had an unexplored candidate and continues from there. That return is backtracking, and it is the property neither a single chain nor a handful of independent chains has.
| Arrangement | What it does with the extra work |
|---|---|
| One chain of steps | Extends a single path to its end. Nothing is compared, and an early wrong turn is carried into the answer. |
| Several independent chains | Answers the same question repeatedly and keeps the answer that recurs. Whole paths are compared, at the end, and only where answers can be matched against each other. |
| A tree of partial states | Compares states as they are produced, drops the weak ones, and can return to a branch it abandoned earlier. |
The three differ in when a comparison happens: never, once at the end, or at every level. Comparing at every level is what buys backtracking, and it is also what makes the cost grow with the branching factor at each level rather than with the number of finished attempts.
What the tree costs, and what a task must have for it to be worth building
Start with the bill, because it is larger than the description suggests. A chain pays for one path. A tree pays for every candidate it generates and, separately, for every evaluation it runs, at every level it keeps open — generations multiplied by the branching factor, with evaluations on top of them. Both are ordinary output tokens and ordinary latency. Nothing about the structure makes them cheaper, and a tree that prunes weakly degenerates into an exhaustive enumeration performed at the price of generated text.
Then the precondition, which is the one that decides whether the method applies at all. The search works only where partial states can be judged before they are finished. A half-built proof, a partly filled grid, the first three steps of a route: each can be called hopeless while there is still something to be saved by abandoning it. A task whose value is visible only once the work is complete — a piece of prose, a summary, a single lookup — gives the evaluator nothing to act on, and the search then spends its branching factor to arrive where one chain would have arrived anyway.
Last, the standing of the judge. The evaluation that prunes the tree is produced by the same process as the candidates being pruned, usually from the same weights under a different prompt. It is not an oracle and not an external check. Where a task has a real verifier — a test suite, a solver, a constraint checker, a unit conversion that either balances or does not — putting that in the evaluation slot changes the character of the search, because the pruning becomes sound rather than plausible. Where no verifier exists, the tree explores under the same limitations that generated its branches, and its advantage over a single chain reduces to having tried more than one thing before committing.
How the Registry classifies a branching search task
Generating several candidate paths, scoring them and keeping one is filed as data processing in this registry's classification. The classification attaches to the selection rather than to the subject matter: candidates are derived from a supplied specification, compared against one another, and all but one discarded — a computation over supplied inputs whatever the candidates happen to be about.
The Registry's canonical brief for this entry is filed as:
Explore several candidate solution paths for a planning task and keep the branch that survives its own check.
Submitted for assessment it is
classified as Data processing, and its wording is hashed once — to
3e9a70bc24af095b…, the first sixteen of sixty-four hexadecimal
characters — with the wording itself never stored. The hash is what the derivation
reads.
That class's own page is /tasks/analysis.
Classification is one of three inputs. The other two are the configuration submitted with the task, and the permanent chart derived from that configuration — fixed by the model name, the training cutoff and the temperature alone, and never reading the task at all. The same ascendant, ruling planet and harmony therefore appear on every assessment a given configuration receives, whatever it was asked to do. The derivation is published in full at /method.
What this page does not claim about tree of thoughts
How wide a tree should be, how deep it should go, and whether the branch a search keeps is the branch that was right are all measurements of one run on one task. This registry runs no searches, scores no branches and holds no such measurement. It does not know what an evaluator returns for your states, and it has never compared a branching search against a single chain on anything.
The Registry does not run this task, does not inspect any system's output for it, and validates no assessment it issues against what afterwards happens. What it does is compute — from a published method, for one submitted task and one submitted configuration — a verdict and a recommended execution window. It computes neither on this page.
Questions about tree of thoughts
- What is tree of thoughts prompting?
- Tree of thoughts organises intermediate steps as a search tree: several candidate next steps are generated from the current state, the resulting partial states are evaluated, and only the promising ones are expanded. Weak branches are dropped and an abandoned branch can be returned to.
- How is tree of thoughts different from chain of thought?
- A chain extends one path and cannot go back. A tree keeps several partial states alive at once, judges them before they are complete, and can abandon one to resume from an earlier state. The extra machinery is a generator of alternatives, an evaluator of partial states, and a rule for which state to expand next.
- What counts as one thought?
- Whatever unit of intermediate work the implementation chose: a line of a derivation, one equation, a single move, one sentence of a plan. The size is a design decision rather than a property of the method, and it fixes both the depth of the tree and the cost of searching it.
- Does the method need a separate evaluator?
- It needs an evaluation step. Whether that step is a separate component is a design choice, and most implementations prompt the same weights differently instead. Where the task has a genuine verifier — tests, a solver, a constraint check — using it in the evaluation slot makes the pruning sound rather than merely plausible.
- When is a tree not worth its cost?
- When partial states cannot be judged before they are complete, since there is then nothing to prune on, and when a single pass already answers the question. The cost of a tree is paid at every level whether or not the branching found anything a chain would have missed.
Related entries
Order an assessment for data processing tasks
The Registry issues a permanent, numbered task risk assessment for one submitted task and one submitted configuration. EUR 1.90 Standard, EUR 4.90 Extended, EUR 14.90 Full Chart, which adds the permanent chart. Assessments from EUR 1.90; machine-readable at /pricing.json.