What Is a Chunking Strategy?
A chunking strategy is the rule deciding where source documents are cut before they are indexed. The chunk is the unit that gets embedded, the unit that gets retrieved and the unit that arrives in a prompt, so the rule producing it sets the ceiling on what a retrieval system is able to return at all.
The families of splitting rule, and the boundary each one respects
Fixed-window splitting cuts every so many tokens or characters, usually with an overlap so that a sentence straddling a cut survives whole on one side of it. Its indifference to what it is cutting is both the virtue and the defect: it is predictable, it bounds chunk length exactly and it asks nothing of the document, and it is as likely to bisect a table, a code block or a numbered procedure as a paragraph.
Structure-respecting rules cut at boundaries the document already has — sentence and paragraph ends, headings, list items, cells, function definitions. A common implementation walks a list of separators from coarsest to finest, splitting on the coarsest that brings a chunk inside the size limit, which keeps most chunks aligned to something meaningful while still bounding their length. What it requires in exchange is structure that is present and reliable, which a well-formed document has and a scanned page, a meeting transcript or a spreadsheet exported to text does not.
A third family looks at meaning rather than at markup: consecutive stretches of the document are embedded and a cut is placed where the similarity between neighbours drops, on the theory that a fall in similarity marks a change of subject. It produces chunks of uneven length and requires an embedding pass over the corpus before indexing can begin. Like the other two it is a heuristic, and it is worth being precise about which one: what it detects is a change in wording large enough to move the vectors, which is usually a change of subject and is sometimes a change of tone in the middle of one.
| Rule | What it gets right, and what it cuts through |
|---|---|
| Fixed window | Exact size control and no assumptions about the source. Cuts through tables, code and procedures. |
| Overlapping window | A straddling sentence survives somewhere. Duplicates text into the index and into the shortlist. |
| Sentence or paragraph | Chunks that read as units. Lengths vary widely, and one long paragraph can still exceed the limit. |
| Heading or structural | A chunk keeps the section it belongs to. Needs structure the source may not carry. |
| Similarity-based | Boundaries follow changes of subject. Costs an embedding pass, and the boundary is a heuristic. |
No rule in that list is the right one for a corpus of mixed document types, which is the ordinary case. Splitting a policy manual, a support transcript and an API reference under a single strategy means at least two of the three are being cut by a rule chosen for the third. Because the strategy is applied at index time, the evidence arrives much later and in an indirect form: retrieval that works for some sources and not for others, with nothing in the pipeline pointing at the splitter.
Chunk length, missing context, and the cost of changing your mind
Length trades two failures against one another. A chunk small enough to be about a single thing matches a specific question well and is often too thin to answer it — the sentence stating the rule is retrieved without the condition attached to it two sentences further down. A chunk large enough to stand on its own covers several subjects, so its one vector is an average over all of them and matches every one of them weakly. Neither direction has a correct value, only a value that suits a given corpus and a given style of question, which makes it a measurement rather than a default.
The other loss is context that was in the document and is not in the passage. A chunk cut from the middle of a section carries no heading with it, so a paragraph opening with the words "this does not apply where" is retrieved with its antecedent gone and reads as a general statement. The usual repairs are to store the heading path or a short document summary alongside each chunk, or to retrieve a small chunk and supply its surrounding section to the prompt instead of the chunk itself. Both separate the unit that is matched from the unit that is read, which is the right shape and which doubles the bookkeeping.
The strategy is expensive to revisit, and that is the argument for deciding it deliberately rather than by default. Boundaries are fixed when the corpus is indexed, so changing them means re-splitting and re-embedding everything, and it invalidates whatever retrieval evaluation was collected under the old boundaries, because the labelled passages no longer correspond to chunks that exist. One hard limit constrains the choice from above: an embedding pass accepts a maximum input length, and a chunk over it is truncated or rejected depending on the implementation, so an oversized chunk can be indexed with its tail silently absent.
How the Registry classifies a document splitting task
Splitting a set of source documents at chosen boundaries and indexing each passage separately is filed as information retrieval in this registry's classification, even though nothing is being looked up yet. The task is the preparation every later lookup depends on, and it manufactures no material: every passage leaving a splitter was present, in the same words, in what went into it.
The Registry's canonical brief for this entry is filed as:
Split a set of source documents into passages at chosen boundaries and index each passage separately.
Submitted for assessment it is
classified as Information retrieval, and its wording is hashed once — to
9b1f769ed963e481…, the first sixteen of sixty-four hexadecimal
characters — with the wording itself never stored. The hash is what the derivation
reads.
That class's own page is /tasks/retrieval.
Classification is one of three inputs. The other two are the configuration submitted with the task, and the permanent chart derived from that configuration — fixed by the model name, the training cutoff and the temperature alone, and never reading the task at all. The same ascendant, ruling planet and harmony therefore appear on every assessment a given configuration receives, whatever it was asked to do. The derivation is published in full at /method.
What this page does not claim about chunking strategy
No chunk length is recommended here, no overlap is suggested, and no comparison between splitting rules is published, because this registry has measured none. The canonical brief above fixes the operation and leaves the boundaries open, which is the honest shape for it — the right boundary is a property of the documents, and the documents are not part of the brief.
The Registry does not run this task, does not inspect any system's output for it, and validates no assessment it issues against what afterwards happens. What it does is compute — from a published method, for one submitted task and one submitted configuration — a verdict and a recommended execution window. It computes neither on this page.
Questions about chunking strategy
- What is a chunking strategy?
- A chunking strategy is the rule that decides where source documents are cut into passages before indexing. It fixes the unit that is embedded, retrieved and eventually read, which is why it constrains what a retrieval system can return.
- What chunk size should be used?
- It is measured rather than adopted. Smaller chunks match specific questions and often leave out the surrounding condition; larger chunks stand alone and dilute their own vector across several subjects. The length that suits a corpus is found by evaluating retrieval on that corpus, and none is recommended here.
- Why use overlap between chunks?
- So that a sentence sitting across a cut appears whole on at least one side of it. The cost is duplication: the overlapping text is stored twice, and near-identical chunks can then occupy several places in the same shortlist.
- What is semantic chunking?
- Splitting at points where consecutive stretches of a document stop resembling one another in an embedding space, rather than at a fixed length or a markup boundary. It aims at boundaries between subjects and detects changes in wording, which is usually the same thing and is not always.
- Can a chunking strategy be changed later?
- Yes, at the cost of re-splitting and re-embedding the corpus. It also invalidates any retrieval evaluation gathered beforehand, because the labelled passages refer to chunks that no longer exist in that form.
Related entries
Retrieval-augmented generation (RAG) · Embedding · Context window
Order an assessment for information retrieval tasks
The Registry issues a permanent, numbered task risk assessment for one submitted task and one submitted configuration. EUR 1.90 Standard, EUR 4.90 Extended, EUR 14.90 Full Chart, which adds the permanent chart. Assessments from EUR 1.90; machine-readable at /pricing.json.