Hybrid Search: Dense + Sparse
Two rankings of the same five documents, fused by rank rather than by score, because the two methods' raw scores live on entirely different scales.
Fusion
small k lets rank-1 dominate; large k (60 is the common default) smooths everything out
query: "python list methods"
Two Rankings
—Fused Ranking (Reciprocal Rank Fusion)
Reading It
Overview
The idea in brief
Dense retrieval and BM25 fail in different, mostly non-overlapping ways. A document phrased differently from the query but on the same topic can score well under a dense method and poorly under exact keyword match; a document with an unusual acronym or exact code can score well under BM25 and be embedded ambiguously. Hybrid search runs both and combines the results, so a failure in one is not by itself a failure of the whole system. The combination method matters: averaging raw scores is meaningless because the two scoring functions live on incomparable scales. Reciprocal Rank Fusion (RRF) solves this by discarding the scores fully and using only each document's position in each ranked list. The formula RRF(d) = sum(1 / (k + rank(d))) across all lists gives each document a score that rewards being ranked highly by multiple methods. The constant k (usually 60) dampens the gap between adjacent ranks. That way, a document at rank 1 does not overwhelm one at rank 3. A document that both methods agree on builds up contributions from both lists and rises to the top; a document that only one method likes gets a single contribution and ranks lower. RRF requires no tuning, no normalisation, and no learned weights. This is why it became the default fusion method in systems like Elasticsearch and Weaviate. This page runs dense and sparse retrieval on the same query, prints both ranked lists side by side, and shows the RRF scores and the fused ranking.
Two rankings of the same five documents, fused by rank rather than by score, because the two methods' raw scores live on entirely different scales.
In this guide
- The idea in brief
- Why fuse ranks, not raw scores
- Two retrievers with opposite strengths
- Why scores cannot simply be added
- Worked through
- Fusing two rankings without comparing their scores
- Try it yourself
- Summing up
- Implementing it
- The options
- Where hybrid search matters most
- Questions people ask
- Recap in one screen
Hybrid Search: A Practical Guide
Neither method alone, combined by their agreement.
The idea in brief
Dense retrieval and BM25 fail in different, mostly non-overlapping ways. A document phrased differently from the query but on the same topic can score well under a dense method and poorly under exact keyword match; a document with an unusual acronym or exact code can score well under BM25 and be embedded ambiguously. Hybrid search runs both and combines the results, so a failure in one is not by itself a failure of the whole system. The combination method matters: averaging raw scores is meaningless because the two scoring functions live on incomparable scales. Reciprocal Rank Fusion (RRF) solves this by discarding the scores fully and using only each document's position in each ranked list. The formula RRF(d) = sum(1 / (k + rank(d))) across all lists gives each document a score that rewards being ranked highly by multiple methods. The constant k (usually 60) dampens the gap between adjacent ranks. That way, a document at rank 1 does not overwhelm one at rank 3. A document that both methods agree on builds up contributions from both lists and rises to the top; a document that only one method likes gets a single contribution and ranks lower. RRF requires no tuning, no normalisation, and no learned weights. This is why it became the default fusion method in systems like Elasticsearch and Weaviate. This page runs dense and sparse retrieval on the same query, prints both ranked lists side by side, and shows the RRF scores and the fused ranking.
Why fuse ranks, not raw scores
A cosine similarity lives between -1 and 1. A BM25 score is an unbounded sum that depends on corpus size and term rarity. Averaging the two numbers right is meaningless — a BM25 score of 8 is not "worth" anything notably next to a cosine of 0.6. Reciprocal Rank Fusion sidesteps this by throwing the scores away and using only each document's position in each list.
RRF(d) = Σlist 1 / (k + ranklist(d))
A document ranked highly by both methods builds up a large score from both terms. A document ranked #1 by one method but unranked or low by the other only gets a large contribution from the one list —. This lets a document that both methods agree is decent beat a document only one method loves.
Two retrievers with opposite strengths
Dense retrieval compares embedding vectors. It finds documents that mean the same thing in different words. A query for "holiday allowance" retrieves a page about "annual leave entitlement".
Sparse retrieval (BM25) matches terms. It finds exact strings — product codes, error numbers, surnames, version identifiers — that embeddings blur into their neighbours.
Neither covers the other's ground:
| Query | Dense | Sparse |
|---|---|---|
| "how do I reset my password" | Finds "credential recovery" | Misses it — no shared terms |
| "error INV-2024-8871" | Blurs into other invoice numbers | Exact hit |
| "cheapest way to ship" | Finds "lowest-cost delivery" | Misses it |
| "section 4.2.1" | Approximate | Exact |
The pattern is consistent enough that combining them is the standard recommendation. And it is usually the single largest quality improvement available to a retrieval system.
Why scores cannot simply be added
The obvious combination — add the two scores — does not work, because they are not comparable.
BM25 scores are unbounded and depend on term rarity and document length; a good match might score 8.4 or 31.2 depending on the corpus. Cosine similarities sit between −1 and 1, and in practice cluster in a narrow band. Adding them means the BM25 score rules fully.
Normalising helps and is fragile: min-max normalisation depends on the highest score in this particular result set. So the same document scores differently depending on what else was retrieved.
Reciprocal rank fusion avoids the problem by discarding the scores and using only the ranks:
RRF(d) = Σlists 1 / (k + ranklist(d))
A document at rank 1 adds 1/61, at rank 2 adds 1/62. And so on with k = 60. A document appearing in both lists builds up from both.
Because only ordering is used, no normalisation is needed and the method works with any number of retrievers. Dense, sparse, a graph traversal, a metadata filter. Without tuning their relative scales.
Worked through
Query: "annual leave policy for new starters".
| Document | Dense rank | Sparse rank | RRF score |
|---|---|---|---|
| A: Leave entitlement handbook | 1 | 3 | 1/61 + 1/63 = 0.0323 |
| B: New starter checklist | 4 | 1 | 1/64 + 1/61 = 0.0320 |
| C: Holiday booking process | 2 | 8 | 1/62 + 1/68 = 0.0308 |
| D: Annual report 2024 | 9 | 2 | 1/69 + 1/62 = 0.0306 |
The final ordering is A, B, C, D. Note what happened: document A was top for dense and third for sparse. And its agreement across both lists put it first. Document D, second on keyword match alone, ranked last — "annual" matched a document about annual reports. And the dense retriever correctly disagreed.
That is the method in one table: agreement between different retrievers is evidence, and RRF rewards it.
The constant k controls how much rank position matters. Small k sharply favours top ranks; large k flattens the differences. 60 is the value from the first paper and is a fair default.
Fusing two rankings without comparing their scores
Hybrid search runs a keyword retriever and a vector retriever and combines the results. The hard part is that their scores are not on the same scale and never will be. This is exactly the problem reciprocal rank fusion sidesteps.
Just adding the two retrievers' raw scores looks like the obvious fix. The next block tries it and shows why it silently collapses into one ranking.
Min-max normalizing each list before summing looks like the real fix. The next block tries that and looks at what the normalization itself destroys.
Reciprocal rank fusion sidesteps both problems by dropping scores fully and working from rank alone. The next block sets up the formula and the constant k that keeps any single top result from ruling.
With the formula in hand, the next block really fuses the two ranked lists from the running example.
It is worth putting a number on how much that fusion buys over the naive and normalized options above. The next block scores all three approaches at once.
Beyond the numbers on this one query, RRF has a handful of properties that make it a default choice in production.
RRF is also widely said to be very hard to make worse than its inputs. A claim worth testing rather than repeating. The next block adds a third retriever that is pure noise and checks what survives.
A single query proves very little either way. The next block runs four queries where a different retriever wins each time, to see whether fusion's edge holds up on average.
Averaging across those four queries is only half the story. The next block looks at the mean and the worst query for each method. This is where fusion earns its keep.
That worst-case gap is the real argument for paying the cost of two retrievals per query.
Try it yourself
- Compare the two lists. They do not agree on the #1 result — dense and sparse are weighting the same five documents by truly different rules.
- Read the fused ranking. The winner is not always #1 in either single list — it is the document both methods placed respectably. This is exactly what "fused by rank" is built to surface.
- Push k up toward 60. The gaps between fused scores shrink and consensus rules even more strongly — this is the standard default, chosen to be forgiving of exactly how far down a list something sits.
- Pull k down to 1. Rank 1 in either list is now worth sharply more than rank 2 — a document has to really top one of the lists to compete.
Summing up
Hybrid search runs dense and sparse retrieval on its own and fuses their rankings rather than their scores,. This is because the two methods' raw numbers are not on comparable scales. Reciprocal Rank Fusion rewards documents both methods rank well, which makes the combined system more reliable than either retrieval method alone — a document only one method loves can be outranked by one both methods merely like.
Implementing it
def rrf(rankings, k=60, top_n=10):
"""rankings: list of lists of doc ids, each ordered best-first."""
scores = {}
for ranked in rankings:
for rank, doc_id in enumerate(ranked, start=1):
scores[doc_id] = scores.get(doc_id, 0) + 1 / (k + rank)
return sorted(scores, key=scores.get, reverse=True)[:top_n]
results = rrf([dense_ids, bm25_ids])Twelve lines, no tuning, no normalisation. That ease is a large part of why RRF is used so widely in practice despite more complex options existing.
Two practical details:
Retrieve more from each branch than you need. Ask each retriever for 50–100 candidates so documents ranked mid-list in one branch can still be rescued by the other.
Weight the branches if you have evidence. Multiplying one list's contribution by a factor lets you favour the retriever that measures better on your data. Do it from measurements, not intuition.
The options
| Method | How | Needs tuning? |
|---|---|---|
| RRF | Combine ranks | No |
| Weighted score fusion | Normalise scores, then weight and sum | Yes |
| Cross-encoder rerank | Rescore candidates with a model | No, but costs inference |
| Learned fusion | Train a model on click data | Yes, needs labels |
Weighted score fusion can beat RRF when carefully tuned on a specific corpus. And it is brittle: the normalisation depends on the result set and the weights need retuning as the corpus changes.
Cross-encoder reranking is not an other so much as the next stage. The standard production shape is: hybrid retrieval with RRF to get 50 good candidates cheaply. Then a cross-encoder to order them accurately. Then the top 5 into the prompt.
That three-stage pipeline — cheap and broad, then expensive and precise — is the shape almost every serious retrieval system converges on.
Where hybrid search matters most
- Technical documentation, full of identifiers, function names and version numbers that embeddings blur.
- Legal and regulatory text, where section references and exact phrasing carry the meaning.
- Support and ticketing, where users paste error codes alongside natural-language descriptions.
- E-commerce, where a query mixes a brand name (exact) with a description (semantic).
- Any corpus with jargon the embedding model was not trained on.
Where it matters least: short, clean, natural-language corpora with no identifiers. Where dense retrieval alone already performs well.
Questions people ask
Is hybrid search always better? Almost always at least as good. And the gain is largest where exact tokens matter. The cost is running two retrievers.
What value of k? 60, from the first paper. It is not sensitive — anything from 20 to 100 behaves similarly.
Can I fuse more than two retrievers? Yes. And that is a strength — RRF extends to any number without recalibration.
Does RRF need scores at all? No, only ranks. That is exactly what makes it reliable.
Should I rerank after fusing? Yes, if latency allows. Fusion improves the candidate set; a cross-encoder improves the ordering.
How do I know it helped? Measure recall@k and MRR on a question set with known answers, before and after. Do not judge by anecdote.
Recap in one screen
- Dense retrieval finds paraphrase; sparse retrieval finds exact terms. Their failures do not overlap.
- Their scores are on incomparable scales, so adding them does not work.
- RRF combines ranks only —
1/(k+rank)summed across lists — needing no normalisation or tuning. - Agreement between retrievers is evidence, and RRF rewards it by itself.
- The standard pipeline is hybrid retrieval, then RRF. Then a cross-encoder rerank.