Maximal Marginal Relevance

Context first

Plain top-k retrieval picks the k highest-scoring documents on its own. If a document collection has a few near-the same passages about the same popular sub-topic, all of them can legitimately score near the top — and a naive top-k fills the context with repetition instead of coverage. This wastes the retrieval budget on saying the same thing three times. Maximal Marginal Fit (MMR) fixes this by selecting documents iteratively. At each step, it picks the document that boosts a weighted combination of fit to the query and diversity from the documents now selected. The formula is MMR = argmax[ lambda * sim(doc, query) - (1 - lambda) * max(sim(doc, selected)) ]. Lambda controls the trade-off: lambda=1 is pure fit (same as top-k), lambda=0 is pure diversity (maximally unlike what you now have). Values between 0.5 and 0.7 are typical. The "marginal" in the name means the incremental value of adding one more document given what has now been chosen. MMR is used in RAG pipelines (LangChain, LlamaIndex) where the fetched passages turn context for a language model, and redundant context wastes the model's limited context window. It also appears in extractive summarisation, where selecting diverse sentences gives a better summary. This page fetches documents with plain top-k and with MMR, shows the likeness matrix, and lets you adjust lambda to see the fit-diversity trade-off.

The formula

MMR = argmax_{d ∈ left} [ λ · fit(d, query) − (1−λ) · max_{s ∈ selected} likeness(d, s) ]

Choice happens one document at a time, not all at once. The first pick is any is most apt, just like ordinary retrieval. Each pick after that is penalised by how close it is to any has now been chosen — so a document that would have ranked highly on fit alone can lose to a less apt document that covers new ground.

The problem with taking the top five

A vector search returns the five chunks closest to the query. On a corpus with any redundancy, those five are often near-copies of each other.

Ask "what is the refund policy?" of a corpus holding the policy in the handbook, in the FAQ, in a support macro and in two versions of the terms page. And the top five results are five statements of the same thing. Four slots wasted, and any nuance — the errors, the timescales, the process — is not fetched at all.

Maximal marginal fit fixes this by selecting results one at a time, each chosen to be apt to the query and unlike what has now been selected:

MMR = argmax_d [ λ · sim(d, q) − (1 − λ) · max_{s ∈ selected} sim(d, s) ]

The first term is fit. The second is a cost for likeness to a thing now chosen. λ sets the balance.

The lambda dial

0.5–0.7 is the range that works for most retrieval. Below about 0.3 the algorithm starts selecting documents chosen mainly for being unlike the others. This is not what you want.

The algorithm itself is greedy and short:

def mmr(query_vec, cand_vecs, lam=0.7, k=5):
    sim_q = cand_vecs @ query_vec                  # relevance to the query
    selected = [int(sim_q.argmax())]

    while len(selected) < k:
        best, best_score = None, -1e9
        for i in range(len(cand_vecs)):
            if i in selected:
                continue
            redundancy = max(cand_vecs[i] @ cand_vecs[j] for j in selected)
            score = lam * sim_q[i] - (1 - lam) * redundancy
            if score > best_score:
                best, best_score = i, score
        selected.append(best)
    return selected

Note that it runs on a candidate set, not the whole corpus. Retrieve 30–50 by plain likeness first, then apply MMR to select the final 5. Running it over a million vectors would be quadratic and pointless.

A worked choice

Query: "refund policy". Suppose five candidates with these similarities to the query, and high mutual likeness among the first three:

Plain top-3 returns A, B, C — three statements of the same policy.

MMR with λ = 0.6 picks A first (highest fit). For the second slot, B scores 0.6×0.87 − 0.4×0.93 = 0.15. While D scores 0.6×0.71 − 0.4×0.35 = 0.29. D wins despite being less apt, because it adds data.

The result is A, D, E — the policy, the timescales and the errors. That is a far better context for answering the question. And it came from the same candidate set.

Running the choice, and watching lambda move it

The formula and the dial are described above. Here they are executed -- the greedy loop written out, each candidate's score at each step. And the point where turning lambda stops adding diversity and starts discarding fit.

import numpy as np

# 3 axes: how much each document is about refunds, about PROCESS, and
# about EXCEPTIONS. A and B are near-duplicates on purpose.
DOCS = {
    "A refund window is 30 days": np.array([1.00, 0.05, 0.02]),
    "B refunds within thirty days": np.array([0.98, 0.06, 0.03]),
    "C how to request a refund": np.array([0.90, 0.35, 0.05]),
    "D shipping takes 5-7 days": np.array([0.20, 0.95, 0.05]),
    "E warranty lasts 24 months": np.array([0.15, 0.10, 0.92]),
    "F refund exceptions apply": np.array([0.85, 0.05, 0.40]),
}
QUERY = np.array([1.0, 0.15, 0.10])

def unit(v):
    return v / np.linalg.norm(v)

def sim(a, b):
    return float(unit(a) @ unit(b))

names = list(DOCS)
rel = {n: sim(QUERY, DOCS[n]) for n in names}
print("done")

Building on the same setup, the next block shows the left cases.

import numpy as np

print("SIX CANDIDATES, scored against the query by plain similarity:")
print("%-32s %14s" % ("document", "relevance"))
for n in sorted(names, key=lambda x: -rel[x]):
    print("%-32s %14.4f" % (n, rel[n]))
print()
print("TAKE THE TOP 3 AND LOOK AT WHAT YOU GET:")
top3 = sorted(names, key=lambda x: -rel[x])[:3]
for n in top3:
    print("   %s" % n)
print("   three ways of saying the same thing. every one is relevant and")
print("   the set is nearly useless: it answers one question three times")
print("   and the others not at all.")
print("   measure the redundancy -- pairwise similarity within the set:")
for i in range(len(top3)):
    for j in range(i + 1, len(top3)):
        print("      %-28s vs %-28s %.4f"
              % (top3[i][:26], top3[j][:26], sim(DOCS[top3[i]], DOCS[top3[j]])))
print()

def mmr(lmb, k=3):
    chosen, log = [], []
    pool = list(names)
    for step in range(k):
        scores = {}
        for n in pool:
            red = max([sim(DOCS[n], DOCS[c]) for c in chosen], default=0.0)
            scores[n] = lmb * rel[n] - (1 - lmb) * red
        # ties break by relevance, as every implementation does -- at
        # lambda = 0 the step-1 scores are all exactly 0 otherwise
        pick = max(scores, key=lambda x: (scores[x], rel[x]))
        log.append((step, dict(scores), pick))
        chosen.append(pick)
        pool.remove(pick)
    return chosen, log

The below block covers the left steps.

import numpy as np

LMB = 0.6
chosen, log = mmr(LMB)
print("MMR, STEP BY STEP, at lambda = %.1f:" % LMB)
print("      score = lambda*relevance - (1-lambda)*max similarity to")
print("              anything already chosen")
for step, scores, pick in log:
    print("   step %d -- already chosen: %s"
          % (step + 1, ", ".join(c.split()[0] for c in chosen[:step]) or "nothing"))
    print("%-32s %12s %12s %12s"
          % ("candidate", "relevance", "redundancy", "MMR score"))
    for n in sorted(scores, key=lambda x: -scores[x]):
        red = max([sim(DOCS[n], DOCS[c]) for c in chosen[:step]], default=0.0)
        print("%-32s %12.4f %12.4f %12.4f%s"
              % (n, rel[n], red, scores[n], "   <- picked" if n == pick else ""))
print("   at step 1 nothing is chosen, so redundancy is 0 for everyone and")
print("   MMR picks the most relevant document -- MMR's first choice is")
print("   ALWAYS the plain top result, whatever lambda is.")
print("   at step 2 the near-duplicates are penalised by their similarity")
print("   to what was just taken, and something different wins.")
print()

The next block goes on with the left demonstrations.

import numpy as np

print("NOW TURN THE DIAL. the set MMR selects at each lambda:")
print("%-10s %-34s %14s %14s"
      % ("lambda", "selected", "mean relevance", "max pair sim"))
for lmb in (0.0, 0.3, 0.5, 0.6, 0.65, 0.7, 0.9, 1.0):
    sel, _ = mmr(lmb)
    pairs = [sim(DOCS[a], DOCS[b]) for i, a in enumerate(sel)
             for b in sel[i + 1:]]
    print("%-10.2f %-34s %14.4f %14.4f"
          % (lmb, " ".join(s.split()[0] for s in sel),
             float(np.mean([rel[s] for s in sel])),
             float(max(pairs)) if pairs else 0.0))
print("   lambda = 1.0 is pure relevance -- the same redundant top 3 from")
print("   the beginning. lambda = 0.0 ignores the query entirely after the")
print("   first pick and selects for difference alone, which is why it")
print("   reaches for the least relevant documents in the pool.")
print("   the two columns move in opposite directions, which is the whole")
print("   trade: you cannot raise diversity without spending relevance,")
print("   and the only question is the exchange rate.")
print("   note that the change is not smooth. the selected set is")
print("   identical across whole stretches of lambda and then flips at a")
print("   threshold, because MMR makes a discrete choice at each step --")
print("   lambda moves a score, and the score only matters when it crosses")
print("   another one. so tuning lambda is not a dial you turn until it")
print("   feels right; it is a search for the few values where the answer")
print("   actually changes.")
print()

print("WHERE THE EXCHANGE RATE IS WORTH IT. compare the extremes:")
for lmb in (1.0, 0.6, 0.0):

Building on the same setup, the next block shows the left cases.

    sel, _ = mmr(lmb)
    print("   lambda %.1f: %s" % (lmb, " | ".join(sel)))
mid, _ = mmr(0.6)
pairs_mid = [sim(DOCS[a], DOCS[b]) for i, a in enumerate(mid)
             for b in mid[i + 1:]]
top_pairs = [sim(DOCS[a], DOCS[b]) for i, a in enumerate(top3)
             for b in top3[i + 1:]]
print("   at lambda 0.6 the most similar pair in the set is %.4f, against"
      % max(pairs_mid))
print("   %.4f for the plain top 3 -- and mean relevance fell only from"
      % max(top_pairs))
print("   %.4f to %.4f."
      % (float(np.mean([rel[x] for x in top3])),
         float(np.mean([rel[x] for x in mid]))))
print("   that is the exchange rate, stated plainly: a large drop in")
print("   redundancy for a small drop in relevance. whether it is a good")
print("   trade depends entirely on whether the query has several aspects")
print("   -- and this one does, since refunds have a window, a process and")
print("   exceptions.")
print()
print("WHEN MMR IS WRONG:")
print("%-38s %s" % ("the query has one correct answer", "diversity adds noise"))
print("%-38s %s" % ("the corpus has genuine duplicates", "dedupe instead"))
print("%-38s %s" % ("the user asked for a list", "MMR helps a lot"))
print("%-38s %s" % ("a summarisation context", "MMR helps a lot"))
print("   for a factual lookup, the second-best document is usually the")
print("   same fact stated again, and that redundancy is CONFIRMATION")
print("   rather than waste. penalising it hands the model a weaker set.")
print("   MMR earns its place when the question has parts -- and when it")
print("   does not, lambda = 1 is the correct setting, which is to say")
print("   MMR turned off.")

Guided tests

1. Read the pairwise likeness grid. Three of the six candidates are near-copies of each other — high likeness across that block — while the rest cover different ground.
2. Leave MMR off. The top 3 by fit alone are the query-planning chunk plus two of the three near-copy B-tree chunks — the hash-index sub-topic is crowded out fully. This covers only 2 of 3 sub-topics with two of three slots spent restating the same B-tree point.
3. Turn MMR on at λ=0.5. The first pick does not change — it is still the most apt document. Since nothing has been selected yet to be close to. The second and third picks do change: instead of a second B-tree chunk, MMR reaches for the hash-index one, and coverage goes from 2 of 3 sub-topics to all 3.
4. Push λ to 1.0. The likeness cost vanishes fully and the result is the same to MMR being off — λ=1 is pure fit by construction.
5. Pull λ to 0.0. Fit stops mattering at all; only spreading out matters. This can surface a barely-tied document purely because nothing now picked resembles it.

What to recall

MMR selects documents one at a time, and after the first pick, each later choice is weighed against how close it is to what has now been selected — trading some fit for coverage. λ controls the trade right: 1.0 is ordinary top-k fit ranking, 0.0 ignores fit and boosts spread. And all of it between balances the two. It costs one likeness check against the selected set per candidate per step. This is cheap next to embedding or reranking and is standard in any RAG system fetching more than a couple of chunks per query.

Where diversity helps and where it hurts

It helps when the question has a few facets. "What are the risks of this approach?" wants a few distinct risks, not five phrasings of the biggest one.

It helps on redundant corpora. Documentation that repeats itself, support content duplicated across channels, versioned documents.

It helps summarisation and overview tasks, where coverage matters more than precision.

It hurts on narrow factual questions. "What is the notice period?" has one answer. And the best three chunks are probably all about it. Diversity here means adding less apt material.

It hurts when the corpus is now diverse. If there is no redundancy, the cost term does nothing except distort the sorting slightly.

That split suggests treating λ as query-dependent rather than fixed: high for lookup questions, lower for exploratory ones. Some systems classify the query type and set it so.

The options

Threshold sifting is the simpler cousin of MMR and often enough. Go down the ranked list and skip a thing with cosine likeness above 0.95 to a thing now kept. It handles the near-copy case without changing the sorting of truly distinct results.

Parent deduplication is worth doing regardless if you use parent-child retrieval. And it fixes a set and common source of redundancy.

The pragmatic recommendation: start with threshold-based near-copy sifting, which is cheap and rarely harmful. Add MMR when the queries are truly multi-faceted.

Real notes

It needs the candidate vectors, not just their ids. The redundancy term compares candidates with each other. Retrieval must return the embeddings, or they must be re-fetched.

Order matters after choice. MMR chooses a set; the order in which it is presented to the model still matters,. This is because attention favours the ends of a long context. Keep the most apt first.

It interacts with reranking. A cross-encoder gives more exact fit scores; running MMR on reranked candidates using cross-encoder scores for the fit term and embedding likeness for the redundancy term works well.

Cost is tiny on 30–50 candidates — a few thousand dot products.

Questions people ask

What λ should I use? 0.5–0.7 as a default. Higher for factual lookup, lower for exploratory questions.

Is MMR better than simple deduplication? More principled, and more likely to change the result set. Threshold deduplication is simpler and adequate when the problem is exact repeats.

Does it improve answer grade? On multi-faceted questions and redundant corpora, measurably. On narrow factual questions it can reduce it slightly.

How many candidates should I pass it? 30–50. Too few and there is nothing diverse to choose; too many and the cost term starts favouring outliers.

Can I use it with hybrid search? Yes — fuse the ranked lists first. Then apply MMR to the fused candidates.

Does it help with the "lost in the middle" problem? Indirectly, by making each fetched chunk carry distinct data. So fewer slots are wasted.

Recap in one screen

• Plain top-k on a redundant corpus returns a few statements of the same thing.
• MMR selects greedily, scoring each candidate for fit minus likeness to what is now selected.
• λ balances the two; 0.5–0.7 suits most retrieval, higher for narrow factual questions.
• Apply it to a candidate set of 30–50, not to the whole corpus.
• Simpler near-copy threshold sifting is often enough; MMR earns its place on multi-faceted queries.