Recursive chunking
Split on the largest natural boundary that fits. Try paragraphs; if a piece is still over the limit, split it on sentences; then on words; then, as a last resort, mid-word. Every fallback loses a little more meaning, so the recursion only descends when it has to.
Overview
Why not just split every N characters
Fixed-size splitting is one line and cuts wherever it lands — mid-sentence, mid-word, mid-number. The retrieved chunk then starts halfway through a thought, and the generator has to answer from a fragment.
Recursive splitting keeps the size limit and spends it on the best boundary available. Chunks vary in length, which is fine: a chunk is a unit of meaning, not a unit of storage.
Parameters
Visualisation
—Readout
What to watch
- Each level is tried only when the level above left something too big.
- Fixed-size splitting cuts mid-sentence; this cuts at a boundary.
- The separator list is where domain knowledge goes.
Recursive chunking: A Practical Guide
What is recursive character chunking, and why is it the default?
The separator list is the whole configuration
The default is roughly ["\n\n", "\n", ". ", " ", ""] — paragraph, line, sentence, word, character. It descends only when a piece still exceeds the limit, so the last entry fires only on text with no whitespace at all.
Change the list to match the document type. Code wants ["\nclass ", "\ndef ", "\n\n"]; Markdown wants heading markers first. That is the cheapest large improvement available to a RAG pipeline, and it is usually left at the default.
Overlap, and what it costs
Chunks usually overlap by 10–20% so a sentence spanning a boundary appears whole in at least one of them. The cost is real: overlap inflates the index, and duplicated text means near-identical chunks compete in the results, crowding out genuinely different ones.
Recursive chunking is the right default and it is still structure-blind — it does not know a heading from a sentence. That is what the structure-aware and semantic variants address.
Things to try
- Drop the chunk size to 60. More pieces fall through to sentence and then word level, and chunks start ending mid-sentence.
- Switch separators to fixed size. Every boundary lands wherever the character count ran out — watch the mid-sentence count climb.
- Raise the size to 260 with recursive separators. One chunk holds the whole passage, which retrieves as a single coarse unit.
What to remember
Recursive chunking splits on the largest natural boundary that fits, descending through a priority list of separators only when a piece is still oversized. The separator list is the whole configuration, and tailoring it to the document type is the cheapest large improvement available to a RAG pipeline.