Permission filtering in RAG retrieval
Filter inside the search, not around it. Post-filtering — retrieve k, then drop what the user cannot see — is the obvious approach and it silently returns fewer results, sometimes none, because the permitted documents were never in the top k. And filtering after generation is not filtering: the model already read the text.
Overview
Why post-filtering quietly fails
Retrieve the top 10 by similarity, then remove what the user may not see. If eight were restricted you return two, and nothing reports that the result set collapsed. The generator answers from thin evidence and sounds no less confident.
Over-fetching — retrieve 100, filter, keep 10 — makes it less likely and never fixes it. A user with access to 0.1% of the corpus needs an enormous k, and the failure is worst exactly for the most restricted users.
Parameters
Visualisation
—Readout
What to watch
- Post-filtering removes results after the top-k is chosen.
- The permitted documents may never have entered the top-k at all.
- Filtering after generation is too late — the model already saw it.
Permission filtering in RAG retrieval: A Practical Guide
How do you stop a RAG system retrieving documents the user is not allowed to see?
Pre-filtering, and why it is harder than it looks
Pre-filtering restricts the candidate set before ranking, so the top k is k permitted results. That is the correct behaviour and it fights the index: an HNSW graph is built over all vectors, and walking it while skipping most nodes can disconnect the search — you traverse into a region where everything is filtered out and the walk stalls.
Vector databases handle this differently, which is one of the real differences between them. Options include filtered graph traversal, per-tenant sub-indexes (clean, and costly at many tenants), and falling back to a flat scan when the filter is very selective — which is often genuinely fastest, because the permitted set is small.
Where the permissions live
Baking an access list into each chunk's metadata at index time is fast and goes stale the moment someone leaves a group — and stale permissions fail open, which is the wrong direction.
The usual compromise: index a stable group identifier with each chunk, resolve the user's groups per request from the identity system, and filter on the intersection. Group membership changes are then reflected immediately without re-indexing anything.
Two more things to say. Deleted documents must leave the index, not just be filtered out. And the filter belongs on the server, derived from the authenticated session — never from a client-supplied user id, which is trivially forged.
Things to try
- Drop corpus visible to 5%. Post-filtering returns almost nothing while pre-filtering still returns k — the control fails worst for the most restricted user.
- Raise over-fetch to 500. The shortfall shrinks and never disappears, which is why over-fetching is a mitigation rather than a fix.
- Set visibility to 100%. All three strategies agree, which is exactly why this bug survives testing on an admin account.
What to remember
Permission filtering has to happen inside the search, not around it. Post-filtering ranks first and drops afterwards, so it silently returns fewer results than requested and degrades worst for the most restricted users. Filtering after generation is not a control at all — the model has already read the text. Index a group id and resolve membership per request, because stale permissions fail open.