Permission filtering in RAG retrieval

Filter inside the search, not around it. Post-filtering — retrieve k, then drop what the user cannot see — is the obvious approach and it silently returns fewer results, sometimes none, because the permitted documents were never in the top k. And filtering after generation is not filtering: the model already read the text.

Overview

Filter inside the search, not around it.

In this guide
  1. Why post-sifting quietly fails
  2. Pre-sifting, and why it is harder than it looks
  3. Where the permissions live
  4. Why pre-sifting fights the index
  5. Getting the model right, and failing closed
  6. The failure mode that matters most
  7. Where the filter has to be applied
  8. Modelling permissions in metadata
  9. Post-sifting, measured, and why it fails quietly
  10. Things to try with Permission filtering in RAG retrieval
  11. What to recall
  12. Late-binding permissions
  13. Failure modes to test for by name
  14. What else leaks
  15. Questions people ask about Permission filtering in RAG retrieval
  16. Permission filtering in RAG retrieval: recap in one screen

Parameters

Visualisation

—

Readout

What to watch

  • Post-filtering removes results after the top-k is chosen.
  • The permitted documents may never have entered the top-k at all.
  • Filtering after generation is too late — the model already saw it.

Permission filtering in RAG retrieval: A Practical Guide

How do you stop a RAG system retrieving documents the user is not allowed to see?

Why post-sifting quietly fails

Retrieve the top 10 by likeness, then remove what the user may not see. If eight were restricted you return two, and nothing reports that the result set collapsed. The generator answers from thin evidence and sounds no less confident.

Over-fetching — retrieve 100, filter, keep 10 — makes it less likely and never fixes it. A user with access to 0.1% of the corpus needs an enormous k. And the failure is worst just for the most restricted users.

The core problem is that vector likeness search and access control run on else axes. The index ranks by semantic fit; the access layer filters by who is allowed to see what. Combining them correctly is a systems problem, not a modelling problem, and there are three approaches.

Pre-sifting restricts the search space before retrieval: just documents the user is allowed to see are candidates. This guarantees the result set is clean and the right size. But it needs the vector index to support metadata sifting fast. Not all indexes do — a naive implementation scans the full corpus and checks permissions per document. This is as slow as brute-force search.

Post-sifting fetches by fit first and removes forbidden documents afterwards. It is simple to implement but unreliable: the result set shrinks unpredictably. And the generator must work with any remains. Raising k helps probabilistically but never guarantees enough permitted results.

Hybrid approaches retrieve a large candidate set, filter it. Then backfill from a wider search if too few documents survive. This is more complex but gives a steady result size. The trade-off is latency: two retrieval rounds instead of one. And the backfill query may not be as well-targeted as the first.

Whichever approach you choose, the permission model it must be correct. A document tagged with the wrong access group will be served to unauthorised users silently and confidently. Permission metadata must be propagated from the source system at indexing time and kept in sync when access rules change.

Pre-sifting, and why it is harder than it looks

Pre-sifting restricts the candidate set before ranking, so the top k is k permitted results. That is the correct behaviour and it fights the index. An HNSW graph is built over all vectors, and walking it while skipping most nodes can disconnect the search. You traverse into a region where all of it is sifted out and the walk stalls.

Vector stores handle this else, which is one of the real gaps between them. Options include sifted graph traversal, per-tenant sub-indexes (clean, and costly at many tenants), and falling back to a flat scan when the filter is quite selective —. This is much truly fastest,. This is since the permitted set is small.

Where the permissions live

Baking an access list into each chunk's metadata at index time is fast and goes stale the moment someone leaves a group. And stale permissions fail open, which is the wrong way.

The usual compromise: index a stable group identifier with each chunk, resolve the user's groups per request from the identity system, and filter on the intersection. Group membership changes are then reflected at once without re-indexing a thing.

Two more things to say. Deleted documents must leave the index, not just be sifted out. And the filter belongs on the server, derived from the authenticated session. Never from a client-supplied user id, which is trivially forged.

Why pre-sifting fights the index

Pre-sifting is the correct behaviour and it is truly hard to implement,. This is since an ANN index was built over the whole corpus and does not know about your predicate.

Walking an HNSW graph while skipping most nodes can disconnect the traversal. The search reaches a region where each neighbour is sifted out and stalls. This returns far fewer results than asked for still though plenty of permitted documents exist elsewhere in the graph.

The plans stores use, all with trade-offs: sifted traversal that keeps searching past left out nodes; per-tenant sub-indexes. This are clean and costly once tenants count in the thousands; and falling back to a flat scan when the filter is selective —. This is much truly fastest. Since a user with access to 0.1% of a million documents is a thousand-vector brute force.

Getting the model right, and failing closed

Two design rules cover most of the damage.

Derive the filter from the session, never the request. A client-supplied user or group id is trivially forged. The filter must be built server-side from the authenticated identity.

Index groups, resolve membership per query. Baking a user list into each chunk means a departure leaves the index granting access until the next re-index. And stale permissions fail open, which is the wrong way. Indexing a stable group id and resolving the user's groups per request makes revocation immediate.

Too: sifting is not deletion. A document removed for legal or privacy reasons must leave the index and any cached answers derived from it. Or it remains recoverable by anyone who can still see it — which is a else and worse problem than a missing result.

The failure mode that matters most

A RAG system fetches documents and puts them in a prompt. If the retrieval does not respect who is asking, the model will faithfully summarise a document the user was never allowed to see. And it will do so fluently, with a citation.

That is a data breach with a helpful tone. And it is the most consequential thing that can go wrong in an enterprise retrieval system. Unlike a hallucination, it is not a grade problem; it is a security one.

The requirement is straightforward to state and easy to implement incorrectly: the set of documents thought of must be just the set this user may read.

Where the filter has to be applied

There are three places it could go, and just one is correct.

Past generation — check the answer for leaked content. Hopeless: the content is now in the output, and paraphrase makes detection unreliable.

Past retrieval, before the prompt — retrieve k documents, discard the ones the user cannot see, pass the rest. Better, and still wrong in two ways: the unauthorised documents taken slots. So fewer authorised ones were thought of; and their content has now been loaded into the use's memory.

Inside the search — the index just ever thinks of documents matching the user's permissions. This is the correct answer, and it needs the store to support sifted search.

results = store.search(
    query_vector=q,
    k=10,
    filter={"acl": {"$in": user.group_ids}},   # applied during the search
)

The gap between the second and third options is not theoretical. With post-sifting, a user in a small group querying a corpus ruled by else groups' documents may get zero results while apt authorised documents exist —. This is since all ten fetched slots went to documents they cannot see.

Modelling permissions in metadata

The filter can just express what the metadata records, so the model has to be decided at indexing time.

ModelMetadataFilter
Group-basedacl: [group_ids]user's groups intersect the list
Role-basedmin_role: "manager"user's role rank is sufficient
Owner-basedowner_idowner matches, or is shared with
Attribute-basedregion, department, classificationseveral predicates combined
Row-level via sourceReference to the source systemResolve at query time

Group-based ACLs cover most cases and are what most source systems now provide. Store the list of group ids permitted to read each document, and filter on intersection with the requesting user's groups.

Two hard parts:

Inherited permissions. A document in a restricted folder inherits its restriction. The indexing pipeline must resolve inheritance and write the effective ACL, not the document's own.

Permission changes. When someone leaves a group, or a document is reclassified, the index is stale and will over-share until it is updated. Permissions change far more much than document content, so they need their own refresh path. Ideally event-driven, and at min on a short schedule.

Post-sifting, measured, and why it fails quietly

The article says post-sifting quietly fails. That is a claim with a count, and the count is worse than most people expect. This measures it, then measures the pre-sifting approach that fixes it and the new problem that one creates.

example_01.pyNumPy
Output

That average hides how deep the true results are really sitting. Gauging how far down the ranked list a low-privilege user's real results fall shows why "just retrieve more" is not a free patch.

example_02.pyNumPy
Output

Sifting before ranking instead of past removes that whole failure mode. The search just ever sees documents the user is allowed to have.

example_03.pyNumPy
Output

Pre-sifting is correct, so the question is why anyone still post-filters. And the answer is that an approximate index does not tolerate having nodes yanked out from below its search.

example_04.pyNumPy
Output

None of this is about ranking at all once permissions them go stale. And an index that was correct when it was built can turn wrong the moment an access grant changes.

example_05.pyNumPy
Output

Things to try with Permission filtering in RAG retrieval

  1. Drop corpus visible to 5%. Post-sifting returns almost nothing while pre-sifting still returns k. The control fails worst for the most restricted user.
  2. Raise over-fetch to 500. The shortfall shrinks and never disappears, which is why over-fetching is a mitigation rather than a fix.
  3. Set visibility to 100%. All three plans agree, which is just why this bug survives testing on an admin account.

What to recall

Permission sifting has to happen inside the search, not around it. Post-sifting ranks first and drops afterwards, so it silently returns fewer results than requested and degrades worst for the most restricted users. Sifting past generation is not a control at all. The model has now read the text. Index a group id and resolve membership per request, since stale permissions fail open.

Late-binding permissions

An else to storing ACLs in the index is to resolve them at query time against the source system.

Store just a guide — the source id — and past fetching candidates, ask the source system which of them this user may read.

 ACLs in the indexResolve at query time
StalenessPossible, needs refreshNone — always current
LatencyNone extraAn extra call per query
Correctness during changesWindow of over-sharingCorrect immediately
Filtering inside the searchYesNo — post-filter only

The trade is sharp: resolving at query time is always correct and forces post-sifting. This brings back the slot-wastage problem.

The pragmatic combination used in practice is both: filter inside the search with indexed ACLs to get a correct-enough candidate set fast. Then verify the final choice against the source system before showing a thing. The index filter does the heavy lifting; the verification catches the staleness window.

Failure modes to test for by name

Permission bugs do not announce them, so they have to be tested on purpose:

  • A user with no groups should retrieve nothing, not all of it. A missing or empty filter that matches all documents is the classic catastrophic bug.
  • A filter on a field that does not exist should fail closed, not open. Some stores silently ignore unknown fields.
  • Inherited restrictions on documents in restricted containers.
  • Deleted users and revoked groups.
  • Documents with no ACL recorded — decide whether the default is deny (correct) or allow (dangerous), and make it explicit.
  • Cached results. A cache keyed just on the query text will serve one user's authorised results to one more. Cache keys must include the permission context.

That last one is worth stating plainly since caching is added later, as a speed improvement, by someone who is not thinking about permissions.

What else leaks

Retrieval is the main channel and not the just one.

Conversation history carries content forward. If a user's permissions change mid-session, earlier fetched content is still in the context.

Logs and traces. Prompt logging captures fetched document content, so log storage inherits the documents' classification.

Embeddings them. Research has shown that text can be partially reconstructed from its embedding. So a vector store holding sensitive content is it sensitive and needs the same protections as the documents.

Aggregate answers. A user may be permitted to see single documents but not a synthesis that reveals a pattern across them. This is a policy question rather than a technical one, and worth asking.

Questions people ask about Permission filtering in RAG retrieval

Can I filter past retrieval? It is not secure enough and it wastes retrieval slots. Filter inside the search.

What if my vector store does not support sifted search? Over-retrieve heavily and post-filter as an interim measure, and treat it as a reason to change store.

How do I keep permissions fresh? Event-driven updates from the source system, with a scheduled reconciliation as a backstop. Permissions change more much than content.

Should documents with no ACL be readable? Default deny. An explicit allow-all marker is safer than an absent field.

Does the embedding leak content? Partially reconstructable, so treat the vector store as holding the data it.

How do I cache safely? Include the user's permission context in the cache key, or cache just at the query-embedding level rather than the results level.

Permission filtering in RAG retrieval: recap in one screen

  • An unfiltered retrieval will faithfully summarise documents the user may not read.
  • Apply the filter inside the search; post-sifting both leaks and wastes retrieval slots.
  • Store effective ACLs at indexing time, resolving inheritance, and refresh them on its own of content.
  • Default to deny for documents with no recorded permissions, and test the no-groups case by name.
  • Caches, logs, conversation history and the vectors them are all extra leak channels.

Recall check

0 of 3

Say the answer out loud before you reveal it — recalling it is what makes it stick, and rereading it is not.

  1. What does this module say about “Why post-sifting quietly fails”?

  2. What does this module say about “Pre-sifting, and why it is harder than it looks”?

  3. What does this module say about “Where the permissions live”?

Cheat sheet

Permission filtering in RAG retrieval

Stop a RAG system from surfacing documents the user cannot see. Compare pre-filter and post-filter approaches and weigh speed against safety.

GEN AI · vizlearn.in/gen_ai/permission_filtering_in_rag.html

About the author

Ashish Jangra builds and maintains VizLearn. Every module here is written and the visualisation behind it hand-built, so the numbers in a readout come from the same code that draws the picture. Corrections are genuinely welcome and get priority over everything else — if a page states something wrong, or an animation misrepresents what the algorithm does, get in touch.

Teaching a course with this? Individual teachers can use it freely; for a whole department or programme, see institutional licences.