What Is Reranking?
Reranking is the stage that reorders a shortlist of already-retrieved candidates with a second, costlier scorer before any of them is used. It changes the order and nothing else: every candidate it can promote was returned by the stage in front of it, and every candidate that stage missed is still missing afterwards.
Where the stage sits, and what its depth costs
Retrieval pipelines are built as stages of rising cost per candidate and falling candidate count. The first stage runs against the whole store and must therefore be cheap per document — an index lookup, a term-weighting score. The reranking stage runs only over what came back, so it can afford a scorer that reads a query and a passage together, and it is asked to be right about a hundred candidates rather than a million.
The depth of the shortlist is the parameter that matters, and it pulls in two directions at once. Cost is linear in it: a scorer reading pairs does twice the work for twice the candidates, and the second stage is usually where the pipeline's latency budget is spent. Quality is bounded by it: a document sitting at position two hundred in the first stage's list cannot be promoted by a scorer that was only shown the first fifty. The depth is chosen where those two lines cross, and where they cross is a property of the first stage and the workload rather than of the scorer.
What occupies the stage varies more than the stage does. It may be a scorer that encodes query and passage jointly, a lightweight learned ranker reading precomputed features, or a language model asked to judge or to order the candidates directly. They differ in cost, in what evidence they read, and in whether a candidate is scored alone or against the others in the list — but every one of them is answering a question the first stage did not ask: given that these are the candidates, which of them actually answers this query.
| Stage | What it decides |
|---|---|
| First-stage retrieval | Which candidates exist at all. Its failure is a missing document, and nothing downstream repairs that. |
| Fusion | How candidates from more than one channel are merged into a single shortlist. |
| Reranking | The order of that shortlist. Its failure is a good answer sitting below a worse one. |
| Truncation | How many of the reordered candidates are passed on. Decided by the consumer, not by the scorer. |
| Consumption | What the passages are used for — read by a person, or placed into a prompt. |
Read downwards, the table is also the diagnostic order, and it is worth following in that order rather than starting where the symptom appeared. An answer that is absent is a first-stage defect; an answer that is present but buried is a reranking defect; an answer that is retrieved, ordered first and then not used is neither. Reranking is the stage most often adjusted for a problem belonging to one of its neighbours, because it is the easiest one to swap out.
What reranking cannot recover, and how anyone would know it helped
The ceiling is set upstream. Whether the passage holding the answer came back at all is fixed by the first stage and the shortlist depth, and no amount of care in the second pass adds a candidate to a list. This is the single most useful thing to know about the stage, because a reranker installed to fix a recall problem returns a better-ordered version of the same wrong shortlist, and from the outside that looks like a stage doing its job.
Whether it helped is a measurement, and the measurement needs labels the pipeline does not generate for itself: a set of queries with the passages that should have been returned for them, held apart from whatever was used for tuning. Against that set the first stage is judged on whether the right passage reached the shortlist and the second on where it ended up. Collapsed into one end-to-end number over the pair, the two effects mask each other, and a stage that made things worse can be hidden by a stage that made them better.
The stage also brings failures of its own, and they are operational rather than semantic. It is another service on the critical path with its own latency and availability, and a timeout in it normally degrades to the unreordered list — which is the correct fallback and is invisible unless it is logged, so a reranker can quietly stop contributing without anything breaking. A scorer tuned on one domain's language may also order another domain's passages worse than the cheap stage it displaced, and nothing in the output announces that either.
How the Registry classifies a shortlist reordering task
Reordering a shortlist of retrieved passages by a second scoring pass is filed as information retrieval in this registry's classification, and it is the plainest case in the class: the entire output of the operation is a permutation. No passage is written, none is edited, and the set that goes in is the set that comes out.
The Registry's canonical brief for this entry is filed as:
Reorder a shortlist of retrieved passages by a second scoring pass before any of them enters a prompt.
Submitted for assessment it is
classified as Information retrieval, and its wording is hashed once — to
2fe708444468a168…, the first sixteen of sixty-four hexadecimal
characters — with the wording itself never stored. The hash is what the derivation
reads.
That class's own page is /tasks/retrieval.
Classification is one of three inputs. The other two are the configuration submitted with the task, and the permanent chart derived from that configuration — fixed by the model name, the training cutoff and the temperature alone, and never reading the task at all. The same ascendant, ruling planet and harmony therefore appear on every assessment a given configuration receives, whatever it was asked to do. The derivation is published in full at /method.
What this page does not claim about reranking
How deep a shortlist to reorder, which scorer to put in the stage, and what latency to allow it belong to a particular pipeline, and the canonical brief above fixes none of them. This registry publishes no ranking quality for any scorer and has run no retrieval evaluation of its own.
The Registry does not run this task, does not inspect any system's output for it, and validates no assessment it issues against what afterwards happens. What it does is compute — from a published method, for one submitted task and one submitted configuration — a verdict and a recommended execution window. It computes neither on this page.
Questions about reranking
- What is reranking?
- Reranking is a second pass over a shortlist of retrieved candidates that reorders them with a more expensive scorer than the one used to retrieve them. It runs after retrieval because its cost scales with the number of candidates scored rather than with the size of the store.
- Does reranking improve recall?
- No. Recall is settled by the stage that produced the shortlist. Reranking changes where a candidate sits, so it improves what occupies the top few positions and can do nothing at all about a passage that was never retrieved.
- How many candidates should be reranked?
- Deep enough that the answer is usually inside the shortlist, and shallow enough that the added latency fits the budget. Both ends are properties of a particular first stage and a particular workload, so the depth is measured on the system rather than adopted from another one. None is recommended here.
- What is used as a reranker?
- Commonly a scorer that reads the query and one passage together as a pair; also learned rankers over precomputed features, and language models asked to judge or order candidates directly. They differ in cost and in whether each candidate is scored alone or against the rest of the list.
- Why did adding a reranker not help?
- The most common reason is that the defect was upstream: if the passage holding the answer is not in the shortlist, reordering the shortlist cannot surface it. The check that separates the two cases is whether the right passage was retrieved at all, measured before the second pass runs.
Related entries
Cross-encoder · Hybrid search · Retrieval-augmented generation (RAG)
Order an assessment for information retrieval tasks
The Registry issues a permanent, numbered task risk assessment for one submitted task and one submitted configuration. EUR 1.90 Standard, EUR 4.90 Extended, EUR 14.90 Full Chart, which adds the permanent chart. Assessments from EUR 1.90; machine-readable at /pricing.json.