What Is Top-P Sampling?
Top-p sampling, also called nucleus sampling, is a truncation rule applied to the next-token distribution before a token is drawn from it. The candidates are placed in order of probability, the shortest run from the top whose probabilities add up to at least the threshold p is kept, everything below that run is discarded, and the survivors are rescaled to sum to one and sampled from. The kept set has no fixed size: it is however many candidates it took to reach p at this particular step.
The set is assembled fresh at every step
That last sentence is the entire design. A distribution over next tokens is sometimes extremely concentrated — inside a word, after a token that can only be followed by one thing, midway through a quoted string — and sometimes broad, as at the opening of a sentence that could go several ways. A rule that keeps a fixed number of candidates treats both alike. A cumulative-probability rule adapts to them: where one candidate already carries most of the mass, the nucleus is that candidate and perhaps one more, and selection is effectively settled; where the mass is spread thin, the nucleus opens until it has covered as much of it as the threshold demands.
What gets removed is the tail, and the tail is the reason the rule exists. A vocabulary holds tens of thousands of entries and every one of them is assigned some probability at every step, nearly all of them minute. Sampled without truncation those minute probabilities are individually negligible and collectively not: over a long generation the sampler will eventually draw one, and a single implausible token puts everything after it on a footing it cannot recover from, because everything after it is conditioned on it. Truncation removes that outcome rather than making it rarer, which is a different kind of guarantee and the one the rule was designed to give.
Two implementation details are worth having straight, because both change what a given value means. The candidate that crosses the threshold is itself included — the set is the smallest one whose total reaches p, so the crossing candidate is part of what reached it — and at least the leading candidate is always retained, which is what keeps a threshold near zero well defined instead of empty. And truncation is applied after temperature in the usual arrangement, so the two settings are not independent: raising temperature flattens the distribution, which spreads the mass, which means more candidates are needed to reach the same p. A pair of values tuned together does not survive one of them being changed alone.
| Value of p | Effect on the candidate set |
|---|---|
| 1.0 | Nothing is discarded. Every candidate carrying any probability at all remains eligible. |
| Just below 1.0 | Only the extreme tail is cut. The set is still wide wherever the distribution is broad. |
| Mid-range | The set tracks the shape of the distribution: narrow where the step is predictable, wide where it is not. |
| Small | Only the leading candidates survive at any step, and most steps reduce to a choice among very few. |
| Approaching zero | The single most probable candidate is kept, since one is always retained. Selection becomes greedy. |
The rows describe the rule, not a recommendation. What any given value does to a particular deployment depends on how sharp that deployment's distributions are, which depends in turn on the network, on the prompt, and on the temperature applied in front of it — so a value carried across from one setup to another is a guess, and this registry publishes no value of its own.
What the threshold is credited with and does not do
Top-p is not a creativity control. It removes candidates from consideration and rescales what remains; it adds nothing, and it cannot lift a candidate above one that already outranked it. Every ordering it operates on was computed before it ran. A higher threshold widens the set of things that might be said next, which produces more variation between runs and equally more opportunity to leave the subject — and whether that reads as invention or as drift is a judgement about the output, not a property of the parameter.
It is not a determinism control either, though it is regularly reached for as one. A nucleus containing a single candidate does select that candidate every time, but which steps have such a nucleus is decided by the distribution rather than by the setting, so a low threshold buys less variation without buying repeatability. Where identical output genuinely matters, the operative setting is the one that switches sampling off — and even then, equality across batch shapes, kernels and library versions is a separate question that no sampling parameter addresses.
And it does not make an answer more correct. A truncation rule can only remove candidates the distribution already rated low, so a confident wrong continuation sits at the very top of the nucleus at every threshold and is selected at all of them. Sampling parameters change how far a system wanders from its own first preference. They do not change what it was going to say when it was sure.
How the Registry classifies a nucleus-threshold generation run
Generating a batch of replies at a fixed nucleus threshold and recording the variation across runs is filed as data processing in this registry's classification, and what decides that is what the operator is left holding: a set of outputs and a comparison across them. The run measures a configuration rather than producing a piece of work for a reader.
The Registry's canonical brief for this entry is filed as:
Generate a batch of replies at a fixed nucleus threshold and record the variation across runs.
Submitted for assessment it is
classified as Data processing, and its wording is hashed once — to
ec00ceaa7f5f63c7…, the first sixteen of sixty-four hexadecimal
characters — with the wording itself never stored. The hash is what the derivation
reads.
That class's own page is /tasks/analysis.
Classification is one of three inputs. The other two are the configuration submitted with the task, and the permanent chart derived from that configuration — fixed by the model name, the training cutoff and the temperature alone, and never reading the task at all. The same ascendant, ruling planet and harmony therefore appear on every assessment a given configuration receives, whatever it was asked to do. The derivation is published in full at /method.
What this page does not claim about top-p
What a particular threshold does to a particular deployment is a fact about that deployment's distributions, and the canonical brief above pins down neither the configuration nor the prompts it would be run against. This registry operates no sampler, recommends no value of p, and has never compared two settings against each other.
The Registry does not run this task, does not inspect any system's output for it, and validates no assessment it issues against what afterwards happens. What it does is compute — from a published method, for one submitted task and one submitted configuration — a verdict and a recommended execution window. It computes neither on this page.
Questions about top-p
- What is top-p sampling?
- Top-p, or nucleus sampling, keeps the smallest set of highest-probability candidates whose probabilities add up to at least p, discards the rest, rescales the survivors and draws one. The size of that set changes from step to step according to how concentrated the distribution is at each step.
- What is the difference between top-p and top-k?
- Top-k keeps a fixed count of candidates however the probability happens to be spread. Top-p keeps a variable count, chosen so that a fixed amount of probability is covered. On a confident step top-k still admits candidates carrying almost nothing; on an uncertain step it may cut off candidates that were genuinely in contention. The nucleus rule exists to remove that mismatch.
- Does top-p interact with temperature?
- Yes, in the usual implementation order. Temperature rescales the distribution first and the nucleus is assembled from the result, so raising temperature spreads the mass and widens the set a given p selects. The two are commonly tuned as a pair, and neither transfers independently of the other.
- What does a top-p of 1.0 mean?
- No truncation at all. The whole distribution stays eligible and the rule has no effect, which makes it the value to use when the intention is to leave sampling alone rather than to restrict it.
- Does a lower top-p make output more accurate?
- No. It narrows what may be selected to the candidates already rated most probable, which reduces variation between runs. If the most probable continuation is the wrong one, every threshold keeps it, because truncation only ever removes from the bottom of the ranking.
Related entries
Order an assessment for data processing tasks
The Registry issues a permanent, numbered task risk assessment for one submitted task and one submitted configuration. EUR 1.90 Standard, EUR 4.90 Extended, EUR 14.90 Full Chart, which adds the permanent chart. Assessments from EUR 1.90; machine-readable at /pricing.json.