Maximal Marginal Relevance

The formula

MMR = argmax_{d ∈ remaining} [ λ · relevance(d, query) − (1−λ) · max_{s ∈ selected} similarity(d, s) ]

Selection happens one document at a time, not all at once. The first pick is whatever is most relevant, exactly like ordinary retrieval. Every pick after that is penalised by how similar it is to whatever has already been chosen — so a document that would have ranked highly on relevance alone can lose to a less relevant document that covers new ground.

The problem with taking the top five

A vector search returns the five chunks closest to the query. On a corpus with any redundancy, those five are frequently near-duplicates of each other.

Ask "what is the refund policy?" of a corpus containing the policy in the handbook, in the FAQ, in a support macro and in two versions of the terms page, and the top five results are five statements of the same thing. Four slots wasted, and any nuance — the exceptions, the timescales, the process — is not retrieved at all.

Maximal marginal relevance fixes this by selecting results one at a time, each chosen to be relevant to the query and different from what has already been selected:

MMR = argmax_d [ λ · sim(d, q) − (1 − λ) · max_{s ∈ selected} sim(d, s) ]

The first term is relevance. The second is a penalty for similarity to anything already chosen. λ sets the balance.

The lambda dial

0.5–0.7 is the range that works for most retrieval. Below about 0.3 the algorithm starts selecting documents chosen mainly for being unlike the others, which is not what you want.

The algorithm itself is greedy and short:

def mmr(query_vec, cand_vecs, lam=0.7, k=5):
    sim_q = cand_vecs @ query_vec                  # relevance to the query
    selected = [int(sim_q.argmax())]

    while len(selected) < k:
        best, best_score = None, -1e9
        for i in range(len(cand_vecs)):
            if i in selected:
                continue
            redundancy = max(cand_vecs[i] @ cand_vecs[j] for j in selected)
            score = lam * sim_q[i] - (1 - lam) * redundancy
            if score > best_score:
                best, best_score = i, score
        selected.append(best)
    return selected

Note that it operates on a candidate set, not the whole corpus. Retrieve 30–50 by plain similarity first, then apply MMR to select the final 5. Running it over a million vectors would be quadratic and pointless.

A worked selection

Query: "refund policy". Suppose five candidates with these similarities to the query, and high mutual similarity among the first three:

Plain top-3 returns A, B, C — three statements of the same policy.

MMR with λ = 0.6 picks A first (highest relevance). For the second slot, B scores 0.6×0.87 − 0.4×0.93 = 0.15, while D scores 0.6×0.71 − 0.4×0.35 = 0.29. D wins despite being less relevant, because it adds information.

The result is A, D, E — the policy, the timescales and the exceptions. That is a far better context for answering the question, and it came from the same candidate set.

Running the selection, and watching lambda move it

The formula and the dial are described above. Here they are executed -- the greedy loop written out, every candidate's score at every step, and the point where turning lambda stops adding diversity and starts discarding relevance.

import numpy as np

# 3 axes: how much each document is about refunds, about PROCESS, and
# about EXCEPTIONS. A and B are near-duplicates on purpose.
DOCS = {
    "A refund window is 30 days": np.array([1.00, 0.05, 0.02]),
    "B refunds within thirty days": np.array([0.98, 0.06, 0.03]),
    "C how to request a refund": np.array([0.90, 0.35, 0.05]),
    "D shipping takes 5-7 days": np.array([0.20, 0.95, 0.05]),
    "E warranty lasts 24 months": np.array([0.15, 0.10, 0.92]),
    "F refund exceptions apply": np.array([0.85, 0.05, 0.40]),
}
QUERY = np.array([1.0, 0.15, 0.10])

def unit(v):
    return v / np.linalg.norm(v)

def sim(a, b):
    return float(unit(a) @ unit(b))

names = list(DOCS)
rel = {n: sim(QUERY, DOCS[n]) for n in names}

print("SIX CANDIDATES, scored against the query by plain similarity:")
print("%-32s %14s" % ("document", "relevance"))
for n in sorted(names, key=lambda x: -rel[x]):
    print("%-32s %14.4f" % (n, rel[n]))
print()
print("TAKE THE TOP 3 AND LOOK AT WHAT YOU GET:")
top3 = sorted(names, key=lambda x: -rel[x])[:3]
for n in top3:
    print("   %s" % n)
print("   three ways of saying the same thing. every one is relevant and")
print("   the set is nearly useless: it answers one question three times")
print("   and the others not at all.")
print("   measure the redundancy -- pairwise similarity within the set:")
for i in range(len(top3)):
    for j in range(i + 1, len(top3)):
        print("      %-28s vs %-28s %.4f"
              % (top3[i][:26], top3[j][:26], sim(DOCS[top3[i]], DOCS[top3[j]])))
print()

def mmr(lmb, k=3):
    chosen, log = [], []
    pool = list(names)
    for step in range(k):
        scores = {}
        for n in pool:
            red = max([sim(DOCS[n], DOCS[c]) for c in chosen], default=0.0)
            scores[n] = lmb * rel[n] - (1 - lmb) * red
        # ties break by relevance, as every implementation does -- at
        # lambda = 0 the step-1 scores are all exactly 0 otherwise
        pick = max(scores, key=lambda x: (scores[x], rel[x]))
        log.append((step, dict(scores), pick))
        chosen.append(pick)
        pool.remove(pick)
    return chosen, log

LMB = 0.6
chosen, log = mmr(LMB)
print("MMR, STEP BY STEP, at lambda = %.1f:" % LMB)
print("      score = lambda*relevance - (1-lambda)*max similarity to")
print("              anything already chosen")
for step, scores, pick in log:
    print("   step %d -- already chosen: %s"
          % (step + 1, ", ".join(c.split()[0] for c in chosen[:step]) or "nothing"))
    print("%-32s %12s %12s %12s"
          % ("candidate", "relevance", "redundancy", "MMR score"))
    for n in sorted(scores, key=lambda x: -scores[x]):
        red = max([sim(DOCS[n], DOCS[c]) for c in chosen[:step]], default=0.0)
        print("%-32s %12.4f %12.4f %12.4f%s"
              % (n, rel[n], red, scores[n], "   <- picked" if n == pick else ""))
print("   at step 1 nothing is chosen, so redundancy is 0 for everyone and")
print("   MMR picks the most relevant document -- MMR's first choice is")
print("   ALWAYS the plain top result, whatever lambda is.")
print("   at step 2 the near-duplicates are penalised by their similarity")
print("   to what was just taken, and something different wins.")
print()

print("NOW TURN THE DIAL. the set MMR selects at each lambda:")
print("%-10s %-34s %14s %14s"
      % ("lambda", "selected", "mean relevance", "max pair sim"))
for lmb in (0.0, 0.3, 0.5, 0.6, 0.65, 0.7, 0.9, 1.0):
    sel, _ = mmr(lmb)
    pairs = [sim(DOCS[a], DOCS[b]) for i, a in enumerate(sel)
             for b in sel[i + 1:]]
    print("%-10.2f %-34s %14.4f %14.4f"
          % (lmb, " ".join(s.split()[0] for s in sel),
             float(np.mean([rel[s] for s in sel])),
             float(max(pairs)) if pairs else 0.0))
print("   lambda = 1.0 is pure relevance -- the same redundant top 3 from")
print("   the beginning. lambda = 0.0 ignores the query entirely after the")
print("   first pick and selects for difference alone, which is why it")
print("   reaches for the least relevant documents in the pool.")
print("   the two columns move in opposite directions, which is the whole")
print("   trade: you cannot raise diversity without spending relevance,")
print("   and the only question is the exchange rate.")
print("   note that the change is not smooth. the selected set is")
print("   identical across whole stretches of lambda and then flips at a")
print("   threshold, because MMR makes a discrete choice at each step --")
print("   lambda moves a score, and the score only matters when it crosses")
print("   another one. so tuning lambda is not a dial you turn until it")
print("   feels right; it is a search for the few values where the answer")
print("   actually changes.")
print()

print("WHERE THE EXCHANGE RATE IS WORTH IT. compare the extremes:")
for lmb in (1.0, 0.6, 0.0):
    sel, _ = mmr(lmb)
    print("   lambda %.1f: %s" % (lmb, " | ".join(sel)))
mid, _ = mmr(0.6)
pairs_mid = [sim(DOCS[a], DOCS[b]) for i, a in enumerate(mid)
             for b in mid[i + 1:]]
top_pairs = [sim(DOCS[a], DOCS[b]) for i, a in enumerate(top3)
             for b in top3[i + 1:]]
print("   at lambda 0.6 the most similar pair in the set is %.4f, against"
      % max(pairs_mid))
print("   %.4f for the plain top 3 -- and mean relevance fell only from"
      % max(top_pairs))
print("   %.4f to %.4f."
      % (float(np.mean([rel[x] for x in top3])),
         float(np.mean([rel[x] for x in mid]))))
print("   that is the exchange rate, stated plainly: a large drop in")
print("   redundancy for a small drop in relevance. whether it is a good")
print("   trade depends entirely on whether the query has several aspects")
print("   -- and this one does, since refunds have a window, a process and")
print("   exceptions.")
print()
print("WHEN MMR IS WRONG:")
print("%-38s %s" % ("the query has one correct answer", "diversity adds noise"))
print("%-38s %s" % ("the corpus has genuine duplicates", "dedupe instead"))
print("%-38s %s" % ("the user asked for a list", "MMR helps a lot"))
print("%-38s %s" % ("a summarisation context", "MMR helps a lot"))
print("   for a factual lookup, the second-best document is usually the")
print("   same fact stated again, and that redundancy is CONFIRMATION")
print("   rather than waste. penalising it hands the model a weaker set.")
print("   MMR earns its place when the question has parts -- and when it")
print("   does not, lambda = 1 is the correct setting, which is to say")
print("   MMR turned off.")

Guided experiments

1. Read the pairwise similarity grid. Three of the six candidates are near-duplicates of each other — high similarity across that block — while the rest cover different ground.
2. Leave MMR off. The top 3 by relevance alone are the query-planning chunk plus two of the three near-duplicate B-tree chunks — the hash-index sub-topic is crowded out entirely, covering only 2 of 3 sub-topics with two of three slots spent restating the same B-tree point.
3. Turn MMR on at λ=0.5. The first pick does not change — it is still the most relevant document, since nothing has been selected yet to be similar to. The second and third picks do change: instead of a second B-tree chunk, MMR reaches for the hash-index one, and coverage goes from 2 of 3 sub-topics to all 3.
4. Push λ to 1.0. The similarity penalty vanishes entirely and the result is identical to MMR being off — λ=1 is pure relevance by construction.
5. Pull λ to 0.0. Relevance stops mattering at all; only spreading out matters, which can surface a barely-related document purely because nothing already picked resembles it.

What to remember

MMR selects documents one at a time, and after the first pick, every subsequent choice is weighed against how similar it is to what has already been selected — trading some relevance for coverage. λ controls the trade directly: 1.0 is ordinary top-k relevance ranking, 0.0 ignores relevance and maximises spread, and everything between balances the two. It costs one similarity comparison against the selected set per candidate per step, which is cheap next to embedding or reranking and is standard in any RAG system retrieving more than a couple of chunks per query.

Where diversity helps and where it hurts

It helps when the question has several facets. "What are the risks of this approach?" wants several distinct risks, not five phrasings of the biggest one.

It helps on redundant corpora. Documentation that repeats itself, support content duplicated across channels, versioned documents.

It helps summarisation and overview tasks, where coverage matters more than precision.

It hurts on narrow factual questions. "What is the notice period?" has one answer, and the best three chunks are probably all about it. Diversity here means introducing less relevant material.

It hurts when the corpus is already diverse. If there is no redundancy, the penalty term does nothing except distort the ordering slightly.

That split suggests treating λ as query-dependent rather than fixed: high for lookup questions, lower for exploratory ones. Some systems classify the query type and set it accordingly.

The alternatives

Threshold filtering is the simpler cousin of MMR and often sufficient: go down the ranked list and skip anything with cosine similarity above 0.95 to something already kept. It handles the near-duplicate case without changing the ordering of genuinely distinct results.

Parent deduplication is worth doing regardless if you use parent-child retrieval, and it addresses a specific and common source of redundancy.

The pragmatic recommendation: start with threshold-based near-duplicate filtering, which is cheap and rarely harmful. Add MMR when the queries are genuinely multi-faceted.

Practical notes

It needs the candidate vectors, not just their ids — the redundancy term compares candidates with each other. Retrieval must return the embeddings, or they must be re-fetched.

Order matters after selection. MMR chooses a set; the order in which it is presented to the model still matters, because attention favours the ends of a long context. Keep the most relevant first.

It interacts with reranking. A cross-encoder gives more accurate relevance scores; running MMR on reranked candidates using cross-encoder scores for the relevance term and embedding similarity for the redundancy term works well.

Cost is negligible on 30–50 candidates — a few thousand dot products.

Questions people ask

What λ should I use? 0.5–0.7 as a default. Higher for factual lookup, lower for exploratory questions.

Is MMR better than simple deduplication? More principled, and more likely to change the result set. Threshold deduplication is simpler and adequate when the problem is exact repeats.

Does it improve answer quality? On multi-faceted questions and redundant corpora, measurably. On narrow factual questions it can reduce it slightly.

How many candidates should I pass it? 30–50. Too few and there is nothing diverse to choose; too many and the penalty term starts favouring outliers.

Can I use it with hybrid search? Yes — fuse the ranked lists first, then apply MMR to the fused candidates.

Does it help with the "lost in the middle" problem? Indirectly, by making every retrieved chunk carry distinct information, so fewer slots are wasted.

Recap in one screen

• Plain top-k on a redundant corpus returns several statements of the same thing.
• MMR selects greedily, scoring each candidate for relevance minus similarity to what is already selected.
• λ balances the two; 0.5–0.7 suits most retrieval, higher for narrow factual questions.
• Apply it to a candidate set of 30–50, not to the whole corpus.
• Simpler near-duplicate threshold filtering is often enough; MMR earns its place on multi-faceted queries.