Modules/Gen AI/ Hybrid Search Lab

Hybrid Search: Dense + Sparse

Generative AI & LLMsBy Ashish JangraUpdated 4 September 2026

Two rankings of the same five documents, fused by rank rather than by score, because the two methods' raw scores live on entirely different scales.

Overview

The idea in brief

Dense retrieval and BM25 fail in different, mostly non-overlapping ways. A document phrased differently from the query but on the same topic can score well under a dense method and poorly under exact keyword match; a document with an unusual acronym or exact code can score well under BM25 and be embedded ambiguously. Hybrid search runs both and combines the results, so a failure in one is not automatically a failure of the whole system.

Fusion

RRF constant k4

small k lets rank-1 dominate; large k (60 is the common default) smooths everything out

query: "python list methods"

Two Rankings

—

Dense (cosine)

Sparse (BM25)

Fused Ranking (Reciprocal Rank Fusion)

Reading It

Hybrid Search: A Practical Guide

Neither method alone, combined by their agreement.

Why fuse ranks, not raw scores

A cosine similarity lives between -1 and 1. A BM25 score is an unbounded sum that depends on corpus size and term rarity. Averaging the two numbers directly is meaningless — a BM25 score of 8 is not "worth" anything in particular next to a cosine of 0.6. Reciprocal Rank Fusion sidesteps this by throwing the scores away and using only each document's position in each list.

RRF(d) = Σ_list 1 / (k + rank_list(d))

A document ranked highly by both methods accumulates a large score from both terms. A document ranked #1 by one method but unranked or low by the other only gets a large contribution from the one list — which lets a document that both methods agree is decent beat a document only one method loves.

Two retrievers with opposite strengths

Dense retrieval compares embedding vectors. It finds documents that mean the same thing in different words — a query for "holiday allowance" retrieves a page about "annual leave entitlement".

Sparse retrieval (BM25) matches terms. It finds exact strings — product codes, error numbers, surnames, version identifiers — that embeddings blur into their neighbours.

Neither covers the other's ground:

Query	Dense	Sparse
"how do I reset my password"	Finds "credential recovery"	Misses it — no shared terms
"error INV-2024-8871"	Blurs into other invoice numbers	Exact hit
"cheapest way to ship"	Finds "lowest-cost delivery"	Misses it
"section 4.2.1"	Approximate	Exact

The pattern is consistent enough that combining them is the standard recommendation, and it is usually the single largest quality improvement available to a retrieval system.

Why scores cannot simply be added

The obvious combination — add the two scores — does not work, because they are not comparable.

BM25 scores are unbounded and depend on term rarity and document length; a good match might score 8.4 or 31.2 depending on the corpus. Cosine similarities sit between −1 and 1, and in practice cluster in a narrow band. Adding them means the BM25 score dominates entirely.

Normalising helps and is fragile: min-max normalisation depends on the highest score in this particular result set, so the same document scores differently depending on what else was retrieved.

Reciprocal rank fusion avoids the problem by discarding the scores and using only the ranks:

RRF(d) = Σ_lists 1 / (k + rank_list(d))

A document at rank 1 contributes 1/61, at rank 2 contributes 1/62, and so on with k = 60. A document appearing in both lists accumulates from both.

Because only ordering is used, no normalisation is needed and the method works with any number of retrievers — dense, sparse, a graph traversal, a metadata filter — without tuning their relative scales.

Worked through

Query: "annual leave policy for new starters".

Document	Dense rank	Sparse rank	RRF score
A: Leave entitlement handbook	1	3	1/61 + 1/63 = 0.0323
B: New starter checklist	4	1	1/64 + 1/61 = 0.0320
C: Holiday booking process	2	8	1/62 + 1/68 = 0.0308
D: Annual report 2024	9	2	1/69 + 1/62 = 0.0306

The final ordering is A, B, C, D. Note what happened: document A was top for dense and third for sparse, and its agreement across both lists put it first. Document D, second on keyword match alone, ranked last — "annual" matched a document about annual reports, and the dense retriever correctly disagreed.

That is the mechanism in one table: agreement between different retrievers is evidence, and RRF rewards it.

The constant k controls how much rank position matters. Small k sharply favours top ranks; large k flattens the differences. 60 is the value from the original paper and is a reasonable default.

Fusing two rankings without comparing their scores

Hybrid search runs a keyword retriever and a vector retriever and combines the results. The hard part is that their scores are not on the same scale and never will be, which is exactly the problem reciprocal rank fusion sidesteps.

example_01.pyNumPy

import numpy as np

DOCS = ["A", "B", "C", "D", "E", "F", "G", "H"]

# what each retriever returns for one query: (doc, score), best first
BM25 = [("C", 18.4), ("A", 12.1), ("F", 9.7), ("B", 4.2), ("H", 1.1)]
VEC = [("D", 0.91), ("G", 0.88), ("B", 0.86), ("C", 0.71), ("E", 0.64)]
RELEVANT = {"B", "C", "D"}

print("TWO RETRIEVERS, ONE QUERY. keyword search returns:")
for r, (d, s) in enumerate(BM25):
    print("   %d. doc %s   score %6.2f  %s"
          % (r + 1, d, s, "(relevant)" if d in RELEVANT else ""))
print("vector search returns:")
for r, (d, s) in enumerate(VEC):
    print("   %d. doc %s   score %6.2f  %s"
          % (r + 1, d, s, "(relevant)" if d in RELEVANT else ""))
print()
print("   they disagree, and neither is complete. BM25 put C first and")
print("   buried B at rank 4. vectors put D first and never returned F or")
print("   A at all. each one missed something the other found, which is")
print("   the entire reason to run both.")
print()

print("THE OBVIOUS APPROACH, AND WHY IT FAILS. just add the scores:")
naive = {}
for d, s in BM25:
    naive[d] = naive.get(d, 0.0) + s
for d, s in VEC:
    naive[d] = naive.get(d, 0.0) + s
print("%-8s %12s %12s %12s %s" % ("doc", "bm25", "vector", "sum", ""))
for d in sorted(naive, key=lambda x: -naive[x]):
    b = dict(BM25).get(d, 0.0)
    v = dict(VEC).get(d, 0.0)
    print("%-8s %12.2f %12.2f %12.2f %s"
          % (d, b, v, naive[d], "(relevant)" if d in RELEVANT else ""))
print("   BM25 scores run to %.1f and cosine scores run to 1.0, so the sum"
      % max(s for _, s in BM25))
print("   is just the BM25 ranking with a rounding error added. the vector")
print("   retriever contributed nothing at all: doc D, its top result, is")
print("   ranked %d of %d."
      % (sorted(naive, key=lambda x: -naive[x]).index("D") + 1, len(naive)))
print()

print("NORMALISING FIRST DOES NOT REALLY FIX IT. min-max each list to 0..1:")
def minmax(pairs):
    s = np.array([x[1] for x in pairs], float)
    lo, hi = s.min(), s.max()
    rng = hi - lo if hi > lo else 1.0
    return {d: (v - lo) / rng for (d, _), v in zip(pairs, s)}

nb, nv = minmax(BM25), minmax(VEC)
norm = {}
for d in set(nb) | set(nv):
    norm[d] = nb.get(d, 0.0) + nv.get(d, 0.0)
print("%-8s %12s %12s %12s %s" % ("doc", "bm25 norm", "vector norm", "sum", ""))
for d in sorted(norm, key=lambda x: -norm[x]):
    print("%-8s %12.4f %12.4f %12.4f %s"
          % (d, nb.get(d, 0.0), nv.get(d, 0.0), norm[d],
             "(relevant)" if d in RELEVANT else ""))
print("   better, but look at what min-max normalisation just did: the")
print("   LAST result in each list is now scored exactly 0.0, and the")
print("   first exactly 1.0, regardless of whether the query was answered")
print("   well or badly. it destroys the one thing the raw scores knew --")
print("   how good the results actually were -- and it is unstable: add")
print("   one more result to either list and every number changes.")
print()

print("RECIPROCAL RANK FUSION ignores the scores entirely and uses only")
print("the POSITION:")
print("      RRF(d) = sum over retrievers of  1 / (k + rank(d))")
K = 60
print("with k = %d, the value everyone uses. the contribution by rank:" % K)
print("%-10s %16s %16s" % ("rank", "1/(60+rank)", "share of rank 1"))
for r in (1, 2, 3, 5, 10, 20, 100):
    print("%-10d %16.6f %15.0f%%"
          % (r, 1.0 / (K + r), 100 * (1.0 / (K + r)) / (1.0 / (K + 1))))
print("   the curve is deliberately FLAT. rank 10 is still worth %.0f%% of"
      % (100 * (1.0 / (K + 10)) / (1.0 / (K + 1))))
print("   rank 1. that is the point of k: without it (k=0) rank 1 would be")
print("   worth %.0fx rank 10 and a single retriever's top hit would" % 10)
print("   dominate every fusion. k flattens the curve so that AGREEMENT")
print("   across retrievers matters more than any one retriever's")
print("   confidence.")
print()

def rrf(lists, k=60):
    out = {}
    for lst in lists:
        for r, (d, _) in enumerate(lst):
            out[d] = out.get(d, 0.0) + 1.0 / (k + r + 1)
    return out

fused = rrf([BM25, VEC], K)
print("FUSED:")
print("%-8s %12s %12s %14s %s"
      % ("doc", "bm25 rank", "vec rank", "RRF score", ""))
br = {d: i + 1 for i, (d, _) in enumerate(BM25)}
vr = {d: i + 1 for i, (d, _) in enumerate(VEC)}
for d in sorted(fused, key=lambda x: -fused[x]):
    print("%-8s %12s %12s %14.6f %s"
          % (d, br.get(d, "-"), vr.get(d, "-"), fused[d],
             "(relevant)" if d in RELEVANT else ""))
print("   doc C is first because BOTH retrievers ranked it highly. that is")
print("   the behaviour RRF is designed for: a document two independent")
print("   methods agree on is a better bet than one that either method")
print("   loves alone.")
print()

print("SCORING THE THREE APPROACHES on this query:")
def prec_at(order, k):
    return sum(1 for d in order[:k] if d in RELEVANT) / float(k)
def rec_at(order, k):
    return sum(1 for d in order[:k] if d in RELEVANT) / float(len(RELEVANT))
orders = {
    "BM25 alone": [d for d, _ in BM25],
    "vectors alone": [d for d, _ in VEC],
    "raw score sum": sorted(naive, key=lambda x: -naive[x]),
    "min-max sum": sorted(norm, key=lambda x: -norm[x]),
    "RRF": sorted(fused, key=lambda x: -fused[x]),
}
print("%-18s %14s %14s %s" % ("method", "precision@3", "recall@3", "top 3"))
for name, o in orders.items():
    print("%-18s %14.4f %14.4f %s"
          % (name, prec_at(o, 3), rec_at(o, 3), " ".join(o[:3])))
print()

print("AND THE PROPERTIES THAT MATTER IN PRODUCTION:")
print("%-34s %s" % ("no score calibration needed", "ranks are comparable"))
print("%-34s %s" % ("no per-retriever tuning", "one constant, k"))
print("%-34s %s" % ("works with any number of lists", "just add more terms"))
print("%-34s %s" % ("robust to one bad retriever", "it contributes rank only"))
print("%-34s %s" % ("cannot express confidence", "the cost of ignoring scores"))
print("   that last row is the honest limitation. if one retriever returns")
print("   a result it is certain about and the other returns junk, RRF")
print("   cannot tell -- it sees rank 1 and rank 1. a learned reranker can")
print("   use the scores, and does better when you have training data to")
print("   fit one. RRF's appeal is that it needs none and works")
print("   immediately.")
print()
print("   IT IS ALSO WIDELY SAID to be very hard to make worse than its")
print("   inputs. that claim is worth testing rather than repeating -- add")
print("   a third retriever that is pure noise and see what survives:")
NOISE = [("H", 0.9), ("G", 0.8), ("F", 0.7), ("E", 0.6), ("A", 0.5)]
with_noise = rrf([BM25, VEC, NOISE], K)
o2 = sorted(with_noise, key=lambda x: -with_noise[x])
print("      without noise: %s   precision@3 %.4f"
      % (" ".join(orders["RRF"][:3]), prec_at(orders["RRF"], 3)))
print("      with noise:    %s   precision@3 %.4f"
      % (" ".join(o2[:3]), prec_at(o2, 3)))
print("   and that is NOT the resilience the previous paragraph promised.")
print("   precision fell from %.4f to %.4f. look at why: the noise list is"
      % (prec_at(orders["RRF"], 3), prec_at(o2, 3)))
print("   not disjoint from the real ones. it ranks G second, and the")
print("   vector retriever also ranks G second, so two votes accumulate on")
print("   a document neither relevant nor agreed upon for any good reason.")
print("   the honest statement is narrower than the folklore: RRF is")
print("   robust to a retriever that returns UNRELATED documents, because")
print("   those votes never meet. it is NOT robust to a bad retriever that")
print("   returns plausible ones, because RRF cannot tell a coincidence")
print("   from a corroboration -- it has no notion of how much any")
print("   retriever should be trusted. that is the same limitation as the")
print("   row above, arriving with consequences: every list you fuse gets")
print("   an equal vote, so add a list only when you would be willing to")
print("   act on its top result.")
print()

print("ONE LAST HONEST POINT, because a single query proves very little.")
print("fusion is not guaranteed to beat the best single retriever -- what")
print("it guarantees is that you need not know which one that is. four")
print("queries where the winner is different each time:")
QUERIES = [
    ("Q1 keyword-friendly", ["C", "A", "F", "B", "H"],
     ["D", "G", "B", "C", "E"], {"B", "C", "D"}),
    ("Q2 paraphrased", ["E", "H", "G", "B", "F"],
     ["B", "A", "C", "G", "D"], {"A", "B", "C"}),
    ("Q3 exact product code", ["F", "B", "A", "C", "G"],
     ["G", "E", "H", "D", "B"], {"A", "B", "F"}),
    ("Q4 both work", ["B", "C", "A", "F", "H"],
     ["C", "B", "D", "G", "E"], {"B", "C"}),
]
print("%-24s %10s %10s %10s" % ("query", "BM25", "vectors", "RRF"))
tot = {"BM25": [], "vectors": [], "RRF": []}
for name, bl, vl, rel in QUERIES:
    def p3(order, rel=rel):
        return sum(1 for d in order[:3] if d in rel) / 3.0
    f = rrf([[(d, 0) for d in bl], [(d, 0) for d in vl]], K)
    fo = sorted(f, key=lambda x: -f[x])
    row = (p3(bl), p3(vl), p3(fo))
    tot["BM25"].append(row[0])
    tot["vectors"].append(row[1])
    tot["RRF"].append(row[2])
    print("%-24s %10.4f %10.4f %10.4f" % (name, row[0], row[1], row[2]))
print("%-24s %10.4f %10.4f %10.4f"
      % ("MEAN", float(np.mean(tot["BM25"])), float(np.mean(tot["vectors"])),
         float(np.mean(tot["RRF"]))))
print("%-24s %10.4f %10.4f %10.4f"
      % ("WORST QUERY", float(np.min(tot["BM25"])),
         float(np.min(tot["vectors"])), float(np.min(tot["RRF"]))))
wins = sum(1 for i in range(len(QUERIES))
           if tot["RRF"][i] >= max(tot["BM25"][i], tot["vectors"][i]))
print("   RRF matched or beat the better of the two on %d of %d queries."
      % (wins, len(QUERIES)))
print("   read the WORST QUERY row, though -- that is where fusion earns")
print("   its place. each single retriever has a query it fails badly, and")
print("   fusion's floor is higher than either, because a query has to")
print("   defeat BOTH methods before it defeats the combination.")
print("   in production you never know which query is coming next, so the")
print("   floor and the mean are what you are buying. the price is two")
print("   retrievals and two indexes on every query instead of one.")

Output

Try it yourself

Compare the two lists. They do not agree on the #1 result — dense and sparse are weighting the same five documents by genuinely different criteria.
Read the fused ranking. The winner is not necessarily #1 in either individual list — it is the document both methods placed respectably, which is exactly what "fused by rank" is built to surface.
Push k up toward 60. The gaps between fused scores shrink and consensus dominates even more strongly — this is the standard default, chosen to be forgiving of exactly how far down a list something sits.
Pull k down to 1. Rank 1 in either list is now worth dramatically more than rank 2 — a document has to actually top one of the lists to compete.

Summing up

Hybrid search runs dense and sparse retrieval independently and fuses their rankings rather than their scores, because the two methods' raw numbers are not on comparable scales. Reciprocal Rank Fusion rewards documents both methods rank well, which makes the combined system more robust than either retrieval method alone — a document only one method loves can be outranked by one both methods merely like.

Implementing it

def rrf(rankings, k=60, top_n=10):
    """rankings: list of lists of doc ids, each ordered best-first."""
    scores = {}
    for ranked in rankings:
        for rank, doc_id in enumerate(ranked, start=1):
            scores[doc_id] = scores.get(doc_id, 0) + 1 / (k + rank)
    return sorted(scores, key=scores.get, reverse=True)[:top_n]

results = rrf([dense_ids, bm25_ids])

Twelve lines, no tuning, no normalisation. That simplicity is a large part of why RRF is used so widely in practice despite more sophisticated alternatives existing.

Two practical details:

Retrieve more from each branch than you need. Ask each retriever for 50–100 candidates so documents ranked mid-list in one branch can still be rescued by the other.

Weight the branches if you have evidence. Multiplying one list's contribution by a factor lets you favour the retriever that measures better on your data. Do it from measurements, not intuition.

The alternatives

Method	How	Needs tuning?
RRF	Combine ranks	No
Weighted score fusion	Normalise scores, then weight and sum	Yes
Cross-encoder rerank	Rescore candidates with a model	No, but costs inference
Learned fusion	Train a model on click data	Yes, needs labels

Weighted score fusion can beat RRF when carefully tuned on a specific corpus, and it is brittle: the normalisation depends on the result set and the weights need retuning as the corpus changes.

Cross-encoder reranking is not an alternative so much as the next stage. The standard production shape is: hybrid retrieval with RRF to get 50 good candidates cheaply, then a cross-encoder to order them accurately, then the top 5 into the prompt.

That three-stage pipeline — cheap and broad, then expensive and precise — is the shape almost every serious retrieval system converges on.

Where hybrid search matters most

Technical documentation, full of identifiers, function names and version numbers that embeddings blur.
Legal and regulatory text, where section references and exact phrasing carry the meaning.
Support and ticketing, where users paste error codes alongside natural-language descriptions.
E-commerce, where a query mixes a brand name (exact) with a description (semantic).
Any corpus with jargon the embedding model was not trained on.

Where it matters least: short, clean, natural-language corpora with no identifiers, where dense retrieval alone already performs well.

Questions people ask

Is hybrid search always better? Almost always at least as good, and the gain is largest where exact tokens matter. The cost is running two retrievers.

What value of k? 60, from the original paper. It is not sensitive — anything from 20 to 100 behaves similarly.

Can I fuse more than two retrievers? Yes, and that is a strength — RRF extends to any number without recalibration.

Does RRF need scores at all? No, only ranks. That is precisely what makes it robust.

Should I rerank after fusing? Yes, if latency allows. Fusion improves the candidate set; a cross-encoder improves the ordering.

How do I know it helped? Measure recall@k and MRR on a question set with known answers, before and after. Do not judge by anecdote.

Recap in one screen

Dense retrieval finds paraphrase; sparse retrieval finds exact terms. Their failures do not overlap.
Their scores are on incomparable scales, so adding them does not work.
RRF combines ranks only — 1/(k+rank) summed across lists — needing no normalisation or tuning.
Agreement between retrievers is evidence, and RRF rewards it automatically.
The standard pipeline is hybrid retrieval, then RRF, then a cross-encoder rerank.