Corrective RAG (CRAG)
Naive RAG retrieves, then generates, whatever came back. Retrieval always returns something, so a query with no good match still produces a confident answer built on irrelevant text. Corrective RAG adds a grader between the two steps: judge the evidence first, and when it is poor, rewrite the query, fall back to another source, or decline to answer.
Overview
The failure it fixes
A vector search returns the k nearest chunks whether or not any of them is relevant. There is no null result: ask about something absent from the corpus and you still get five chunks, at low similarity, and the generator dutifully writes an answer from them.
That is the most damaging RAG failure in production, because the answer is fluent, sourced-looking and wrong. The user has no signal that the retrieval missed.
Parameters
Visualisation
—Readout
What to watch
- Retrieval never fails loudly — it returns the nearest thing it has.
- The grader runs before generation, which is the whole design.
- "I don't know" is one of the paths, not a failure of the system.
Corrective RAG (CRAG): A Practical Guide
What is Corrective RAG, and how does it stop a model answering from irrelevant documents?
The three verdicts
A grader — a small model, a cross-encoder, or a similarity threshold — labels the retrieved set:
Correct. At least one document is relevant. Proceed, often after stripping the irrelevant ones so they do not distract the generator.
Incorrect. Nothing is relevant. Discard everything and take a corrective action rather than generating.
Ambiguous. Something is partly relevant. Combine what you have with an external source.
What correction actually means
Query rewriting is the cheapest: the user's phrasing may simply not match the corpus vocabulary. See query rewriting and HyDE.
Fallback to another source — web search, a second index, a keyword search when the dense one missed.
Abstain. The one people leave out, and often the correct behaviour. A system that answers "that is not in the documents I have" is more useful than one that invents a citation.
The cost is latency and tokens: every query pays for grading, and corrected queries pay for a second retrieval. Grade cheaply, and consider skipping it when the top similarity is already unambiguous.
Things to try
- Select capital of Peru. Nothing clears the threshold, and the pipeline declines — the answer naive RAG cannot give.
- Raise the relevance threshold to 0.8 on a query that was working. More queries take the corrective path; grading is a precision/recall trade like any other classifier.
- Lower the ambiguous floor to 0.05. Weak evidence is now treated as partial rather than absent, which changes which path a borderline query takes.
What to remember
Corrective RAG grades retrieved documents before generating, and takes a different path when they are poor: rewrite the query, fall back to another source, or decline. Retrieval never fails loudly — it always returns its nearest k — so without a grader a query with no answer still produces a confident, sourced-looking one.