What is Hit Rate@k?
The bluntest of the ranking metrics, and often the most useful one to start with. Hit Rate@k is 1 if at least one relevant document appears in the top k, and 0 otherwise. It does not care how many relevant documents there were, or where in the top k they landed. Averaged over a query set, it answers a single question: how often does retrieval put something useful in front of the generator?
Overview
The definition, and the averaging that hides in it
For a single query, Hit Rate@k is binary: 1 if any of the top k results is relevant, 0 if none is. There is no partial credit. A query whose top 3 contains five relevant documents and a query whose top 3 contains exactly one both score 1.
What people usually mean by "our hit rate is 0.82" is the mean of that binary value across an evaluation set: 82% of queries had something useful in the top k. That framing is worth saying out loud, because it makes the metric's shape obvious — it is a proportion of queries, not a proportion of documents.
You will also see it called recall@k with a single relevant document, and in recommender literature simply hit rate. When every query has exactly one correct answer, Hit Rate@k and Recall@k are the same number, which is why the two get confused.
Parameters
Visualisation
—Readout
What to watch
- Click any result to flip whether it is relevant.
- Hit Rate jumps to 1 the moment one relevant item enters the top k.
- Move k below the first relevant rank and it collapses to 0.
What is Hit Rate@k?: A Practical Guide
What does Hit Rate@k measure, and when is it the right metric for a retriever?
Why it is the right first metric for RAG
A RAG generator does not need every relevant document. It needs enough grounding to answer, and for most factual questions one good chunk is enough. If the answer is in the context, the model can use it; if it is not, no amount of prompt engineering will recover it.
That makes Hit Rate@k a measure of the ceiling on your whole pipeline. A hit rate of 0.7 at your context budget means 30% of queries are unanswerable no matter how good the generator is, and every hour spent tuning prompts against those queries is wasted. It is the number to establish before anything else, because it tells you whether your problem is retrieval or generation.
It is also cheap to label. Deciding "is there anything useful here?" is much faster for a human annotator than grading every document on a five-point scale, so a hit-rate evaluation set can be built in an afternoon.
What it deliberately ignores
Position. A relevant document at rank 1 and at rank k score identically. That matters more than it sounds: models attend unevenly across a long context, and evidence buried at the bottom of ten chunks is measurably less likely to be used. Hit rate will not show you that; MRR will.
Quantity. One relevant document scores the same as five. For a question needing several sources — "compare our refund policy across regions" — hit rate can be 1.0 while the answer is hopelessly incomplete. Recall@k is the metric that notices.
Noise. Nine irrelevant results alongside one good one still scores 1. Irrelevant context measurably degrades generation and inflates cost, so a high hit rate with low precision is a real failure mode that this metric reports as success.
Choosing k, and reading the curve
k should be the number of chunks you actually put in the prompt. Evaluating at k=10 when you pass 3 to the model measures a system you are not running. This is the single most common mistake with @k metrics.
Plotting hit rate against k is more informative than any single value. If it climbs steeply from k=1 to k=5 and then flattens, your retriever finds the right documents but ranks them poorly — a reranker will help a lot. If it is flat and low from the start, the documents are not being retrieved at all, and the problem is your embeddings, your chunking, or the fact that the answer is not in the corpus.
That diagnostic split is the most valuable thing hit rate gives you, and it costs one evaluation run.
Things to try
- Set k to 1. Hit rate is 1 only if the very first result is relevant — this is the strictest possible reading, and the closest to what a single-chunk pipeline experiences.
- Click the top result to mark it irrelevant, then lower k to 2. Watch hit rate fall to 0 while recall and precision degrade gradually — the binary metric is far more brittle.
- Mark every result irrelevant, then mark just the last one relevant. Hit rate stays 0 until k reaches 10, then snaps to 1: no partial credit anywhere along the way.
What to remember
Hit Rate@k is the proportion of queries with at least one relevant result in the top k. It is binary per query, ignores position and quantity, and measures the ceiling on your pipeline — if the evidence never reaches the model, nothing downstream can fix it. Use it as the first number you establish, set k to the number of chunks you actually pass to the generator, and plot it against k to tell a ranking problem from a retrieval problem. Then reach for Recall@k when queries need several documents, and MRR when position matters.
All four of these read the same object: an ordered list of retrieved results, with each one labelled relevant or not by a human or a strong model. They differ only in what they choose to notice about it — which is why quoting one without saying which is close to meaningless, and why the visualisation above shows all four at once.
The labels are the expensive part. A relevance judgement per query-document pair is human work, and an evaluation set of fifty queries with judged results is worth more than any amount of metric sophistication on top of unjudged data. Build the set first.