Parent Document Retriever
Small chunks match precisely and read terribly. Large chunks read well and match vaguely. You do not have to choose: search one and return the other.
Overview
The problem it solves
Chunking forces a trade-off. A small chunk concentrates the query's terms, so its similarity score is high and the match is precise — but on its own it may be a fragment that answers nothing. A large chunk carries the surrounding context that makes an answer possible, but dilutes the matching terms among hundreds of unrelated words, so it scores worse and may not be retrieved at all.
Retrieval
words per indexed chunk
query: "how long is the warranty"
What the model gets
1 — The index: child chunks, scored
—2 — What is handed to the model
The corpus
Parent Document Retrieval: A Practical Guide
Index the needle, return the haystack it came from.
Searching one thing, returning another
A parent document retriever refuses the trade. It splits each document into small child chunks and indexes only those, so retrieval is as precise as small chunks allow. But when a child chunk wins, what gets passed to the model is the parent it came from. The score decides which document is relevant; the parent decides how much text the model sees. Two chunks from the same parent collapse to one passage, so the context window is not spent sending the same document twice.
The chunk-size dilemma, and a way out of it
Chunking forces a compromise. Small chunks embed precisely — a 100-token passage is about one thing, so its vector is sharp. Large chunks carry context — the paragraph that explains what "this" refers to.
You cannot have both from one chunk size. The parent document retriever resolves it by using different units for retrieval and for generation.
Index small. Split documents into child chunks of 100–200 tokens and embed those. Retrieval operates on them, so matching is precise.
Return large. Each child records which parent section it came from. When a child matches, fetch and return the parent — 800–2,000 tokens of surrounding context — and pass that to the model.
query → match a 150-token child → return its 1,000-token parent → into the prompt
So precision comes from the small unit and context from the large one, with no compromise between them.
Why it works
Consider a handbook section on parental leave. Somewhere in it: "Employees with two years' service are entitled to 39 weeks."
Indexed as one 2,000-token section: the embedding averages over pay, notice periods, eligibility, forms and return-to-work arrangements. A query about "39 weeks entitlement" matches weakly, because that sentence is 2% of the vector.
Indexed as 150-token children: one child is almost entirely about entitlement duration. The query matches it strongly.
But that child alone may not say which type of leave it refers to, or that the two-year condition is measured at a particular date. The parent does.
Retrieving the child and returning the parent gets both. This is the same reasoning behind sentence-window retrieval, at a coarser granularity.
Deduplication, and the detail people miss
Several children frequently match the same parent — a query about parental leave may match three passages within one section.
Without deduplication, the same parent is returned three times, consuming the context budget and telling the model nothing new. Worse, it displaces other parents that would have added information.
So the retrieval step must:
- Search over children and take the top k.
- Map each to its parent id.
- Deduplicate by parent id, keeping the best child's score.
- Return the distinct parents, ordered by that score.
That means retrieving more children than the number of parents you want — ask for 20 children to end up with 5 distinct parents. A common implementation mistake is retrieving 5 children and returning 2 parents, silently halving the context passed to the model.
def retrieve_parents(query, k_children=20, n_parents=5):
children = child_index.search(query, k=k_children)
seen, parents = set(), []
for c in children: # already ordered best-first
if c.parent_id not in seen:
seen.add(c.parent_id)
parents.append(parent_store[c.parent_id])
if len(parents) == n_parents:
break
return parentsSmall chunks in, large chunks out
The sections above lay out the chunk-size dilemma and how searching one size while returning another escapes it. Here it is measured -- the same corpus indexed both ways, the retrieval scored, and the deduplication step that is easy to leave out and changes the answer when you do.
Guided experiments
- Read the child list at chunk size 8. The winning chunk is a short fragment. Read it alone and ask whether it actually answers the question — it names a duration but not what the duration applies to.
- Turn off "Return the parent". The model now receives only those fragments. The word count drops sharply, and so does the chance of a usable answer.
- Turn it back on and watch the passage count. Two separate child chunks hit the same document, but only one parent passage is returned — the deduplication is doing real work.
- Drag the chunk size up to 24. Each child is now most of its parent. The top score actually goes up — a longer window catches more of the query's words — but the selection gets worse: the second slot now goes to a different product's document, and the model is sent 53 words instead of 29 for the same question.
Summing up
Precision of retrieval and sufficiency of context are different requirements, and chunk size cannot satisfy both at once. Indexing small children while returning their parents lets the score come from the tightest possible match and the context come from the whole document. The cost is a larger prompt, which is why deduplicating parents matters: without it, three good chunks from one document would send that document three times.
Choosing the two sizes
| Level | Typical size | Purpose |
|---|---|---|
| Child | 100–250 tokens | Precise embedding and matching |
| Parent | 600–2,000 tokens | Context for the model |
The ratio matters more than the absolute numbers. A parent roughly 5–10× the child is the usual arrangement.
Two sensible ways to define the parent:
Structural. The parent is a real section — a Markdown heading's content, an HTML section, a chapter. Semantically coherent by construction, and variable in size.
Fixed-window. The parent is a fixed number of tokens centred on the child. Predictable sizes, and it may cut across topic boundaries.
Structural parents are better where the documents have structure, which most real corpora do. Store the heading path with each parent and prepend it, so the model knows where in the document the passage sits.
A related variant, sentence-window retrieval, indexes single sentences and returns the two or three either side. Same idea, finer granularity, and it suits question answering where the answer is one sentence but its meaning depends on its neighbours.
Costs and trade-offs
Storage. Both levels are stored — children as vectors in the index, parents as text in a document store. The parents are usually kept outside the vector index, since they are fetched by id rather than searched.
Context budget. Parents are large, so fewer fit. Five 1,000-token parents is 5,000 tokens before the question and instructions. That is the real constraint, and it is why parent size and the number returned must be chosen together.
Redundancy. Overlapping parents can repeat content. Structural parents avoid most of this; fixed windows do not.
Complexity. Two stores to keep consistent. When a document changes, both its children and its parent must be updated, and a partial update leaves children pointing at stale parents.
| Flat chunking | Parent-child | |
|---|---|---|
| Retrieval precision | Compromise | High |
| Context completeness | Compromise | High |
| Storage | One store | Two stores |
| Context tokens used | Moderate | Higher |
| Implementation | Simple | Moderate |
When it helps most
Long structured documents — handbooks, manuals, legal texts, standards. The structure gives natural parents and the length makes flat chunking painful.
Questions whose answers need surrounding context to be interpreted — conditions, exceptions, definitions that appear earlier in the section.
Corpora with heavy cross-referencing, where a passage says "as described above".
Where it helps least: short documents that are already about one thing (FAQ entries, product descriptions, support tickets), where the whole document is a reasonable chunk and the extra machinery buys nothing.
Questions people ask
Is this the same as sentence-window retrieval? The same pattern at a different granularity — sentence windows index single sentences and expand by neighbours; parent-child indexes small chunks and returns sections.
How many children should I retrieve? Enough that deduplication still leaves the number of parents you want — typically 3–5× as many.
Should parents overlap? Usually not, if they are structural sections. Overlapping fixed windows waste context.
Can I rerank? Yes, and rerank the children before mapping to parents — the child is the unit whose relevance you measured.
Does it work with hybrid search? Yes. Run both retrievers over the children, fuse the ranks, then map to parents.
What if a parent is enormous? Cap it, or introduce a middle level. A 10,000-token parent defeats the purpose by filling the context.
Recap in one screen
- Index small chunks for precise matching; return their larger parent sections for context.
- It removes the chunk-size compromise rather than splitting the difference.
- Deduplicate by parent id, and retrieve several times more children than the parents you need.
- Structural parents (sections) beat fixed windows where documents have structure.
- Best on long structured documents; unnecessary on short self-contained ones.