What Is Retrieval-Augmented Generation (RAG)?
Retrieval-augmented generation is a pattern in which a system searches an external store for relevant passages and places them into the prompt before an answer is produced. The generation step itself is unchanged. What changes is where the material comes from: it arrives at request time, from a store that can be edited between one request and the next, rather than having been fixed when the weights were trained.
The stages a retrieval-augmented answer passes through
A retrieval-augmented pipeline runs in a fixed order. Source documents are split into passages and indexed ahead of time. An incoming question is converted into whatever form the store searches on â an embedding vector, a keyword expression, frequently both. A small set of the closest passages is selected and inserted into the prompt, usually under an instruction that the answer be drawn from them. The prompt is submitted, and the answer comes back.
Every part of that sequence is ordinary engineering and each part fails in an ordinary way: a chunking rule that splits a table down the middle, an index last rebuilt before the document changed, a retrieval step that ranks passages by similarity to the question rather than by whether they contain the answer. The stages are separable, and that is the practical value of the pattern â an answer that used the wrong passage and an answer that used the right passage badly are different defects, found and fixed in different places.
| Stage | What happens |
|---|---|
| Indexing | Source documents are split into passages and stored, usually with an embedding vector for each. |
| Query construction | The incoming question is converted into the form the store searches on. |
| Retrieval | A small set of candidate passages is selected, by vector distance, keyword match, or a combination. |
| Augmentation | The selected passages are inserted into the prompt, commonly with an instruction to answer from them alone. |
| Generation | The prompt is submitted and the answer is produced from what the prompt now contains. |
Only the last stage runs the generating model. The four above it are search infrastructure â an embedding model may well be involved in two of them, but nothing in those stages is being asked to write an answer â which is why most of the effort in a working retrieval system is spent above the line rather than below it.
What retrieval does not fix
Retrieval changes what is in the prompt. It does not change what happens to the prompt. An instruction to answer only from the supplied passages is an instruction, not a constraint the system is incapable of departing from, and a passage that is present is not necessarily a passage that is used.
Two further limits are worth stating plainly, because the pattern is often described as though it had removed them. Retrieval cannot supply what the store does not contain: a missing document produces an answer assembled from the nearest thing that was there, delivered in the same tone as any other. And retrieval adds a second ranking problem on top of the first, so the ceiling on an answer is set by the passages the retrieval step chose. A system that retrieves badly cannot be repaired downstream of the retrieval.
The last limit is procedural rather than technical, and it is the one that decides whether the others ever get found. Because the pattern has two halves, it has two evaluations. Retrieval is judged on whether the passage containing the answer came back at all, and at what rank. Generation is judged on whether the answer that followed is supported by the passages that were actually supplied. A single score taken over the pair collapses both, and it conceals the case that matters most: a correct answer produced from a passage that did not contain it, which is indistinguishable from a working system right up until the store is edited.
How the Registry classifies a retrieval-augmented answering task
Answering a question from an internal document store, with each source cited, is filed under information retrieval in this registry's classification. The classification follows the operation rather than the machinery: material that already exists somewhere is located, narrowed and compressed, and nothing is manufactured that was not there before the task began.
The Registry's canonical brief for this entry is filed as:
Answer a customer question by retrieving passages from an internal document store and citing each source in the reply.
Submitted for assessment it is
classified as Information retrieval, and its wording is hashed once â to
3fbbcf35f01af67fâŚ, the first sixteen of sixty-four hexadecimal
characters â with the wording itself never stored. The hash is what the derivation
reads.
That class's own page is /tasks/retrieval.
Classification is one of three inputs. The other two are the configuration submitted with the task, and the permanent chart derived from that configuration â fixed by the model name, the training cutoff and the temperature alone, and never reading the task at all. The same ascendant, ruling planet and harmony therefore appear on every assessment a given configuration receives, whatever it was asked to do. The derivation is published in full at /method.
What this page does not claim about retrieval-augmented generation
Which store is searched, how its passages were cut, and whether the answer's citations resolve are all properties of a particular deployment, and none of them is encoded in the canonical brief above. Neither is the number of passages placed into the prompt, which is the single setting that decides whether an answer was under-supplied or buried under near-duplicates of the same paragraph.
The Registry does not run this task, does not inspect any system's output for it, and validates no assessment it issues against what afterwards happens. What it does is compute â from a published method, for one submitted task and one submitted configuration â a verdict and a recommended execution window. It computes neither on this page.
Questions about retrieval-augmented generation
- What is retrieval-augmented generation?
- Retrieval-augmented generation is a pattern in which relevant passages are fetched from an external store and placed into the prompt before an answer is generated. The weights are unchanged; the material the answer is drawn from is supplied at request time and can be edited between requests.
- How is RAG different from fine-tuning?
- Fine-tuning changes weights by continuing training on a dataset. Retrieval leaves the weights untouched and changes the prompt instead. The practical difference is where the material lives and how it is updated: a fine-tuned configuration holds what it was trained on until it is trained again, while a retrieval store can be corrected in the time it takes to reindex a document.
- Does retrieval stop a system from producing unsupported statements?
- No. It supplies material an answer can be grounded in and, in most implementations, an instruction to use it. Neither is a guarantee. An answer can cite a supplied passage and still assert something the passage does not say, which is why implementations that care about this check the answer against the passages afterwards rather than assuming the instruction held.
- What is a chunk in a RAG pipeline?
- A chunk is one unit of source text as it was stored: a passage produced by splitting a document at some boundary, such as a fixed token count, a paragraph, a heading, or a semantic break. Chunk boundaries decide what can be retrieved as a single piece, so a fact split across two chunks may be retrievable as neither.
Related entries
Semantic search ¡ Embedding ¡ Context window
â All entries ¡ The AFR-1 method ¡ Models in the registry
Order an assessment for information retrieval tasks
The Registry issues a permanent, numbered task risk assessment for one submitted task and one submitted configuration. EUR 1.90 Standard, EUR 4.90 Extended, EUR 14.90 Full Chart, which adds the permanent chart. Assessments from EUR 1.90; machine-readable at /pricing.json.