Context-aware chunking
Every chunking strategy answers "where do I cut?". This one answers a different question: what did this chunk lose by being cut? A fragment full of "it" and "the above" is unretrievable and unusable, so context is added back — a title, a heading path, a generated summary, resolved pronouns.
Overview
The problem is coreference, not boundaries
Prose is written to be read in order, so it leans on everything before it: "it", "this policy", "as described above", "the latter". Cut one paragraph out and those references dangle.
Two consequences. The generator cannot answer from the chunk because it does not know what "it" is. And retrieval never surfaces the chunk in the first place, because the embedding is of text that does not contain the words a user would search for.
Parameters
Visualisation
—Readout
What to watch
- The boundary is unchanged; what is stored changes.
- Pronouns and back-references are what make a raw chunk unusable.
- The enrichment goes into the embedded text, so retrieval improves too.
Context-aware chunking: A Practical Guide
What is context-aware chunking, and what problem does it solve that the other strategies do not?
What gets added
Cheap and free: document title and heading path, prepended. Deterministic, no model call, and usually the largest single improvement.
Neighbour context: the previous and next chunk, or a window around the chunk, stored for generation but not embedded — the parent document retriever is exactly this idea, retrieving small and returning large.
Generated context: an LLM writes a sentence situating the chunk in the document, prepended before embedding. Anthropic's "contextual retrieval" is this, and it works well. It costs one model call per chunk at index time, which prompt caching over the shared document makes affordable.
What it costs, and what to watch
Index time and money, both once. The subtler cost is dilution: prepending 200 tokens of context to a 300-token chunk means the embedding is largely about the context, so chunks from the same section start looking alike and the retriever loses its ability to distinguish them.
Keep the enrichment short relative to the chunk, and measure on a retrieval evaluation set rather than assuming. This is the strategy with the best evidence behind it and it is not free of trade-offs.
Things to try
- Start with everything off. The raw chunk is about refunds, never says so, and cannot be retrieved.
- Turn on resolve pronouns. Similarity jumps, because the searchable nouns are now in the text.
- Turn on over-enrich. The chunk stays findable and stops being distinguishable from its sibling — the cost of adding too much shared context.
What to remember
Context-aware chunking changes what is stored rather than where the cut falls. A chunk full of pronouns and back-references is unusable by the generator and invisible to retrieval, so title, heading path and resolved references are added back. Keep the enrichment short relative to the chunk, or shared context dominates the embedding and chunks stop being distinguishable.