Self-Query Retriever
Some of what a question asks for is a subject, and some of it is a constraint. Similarity search can only answer the first half — unless you pull the constraint out first.
Overview
The idea in brief
Similarity search compares meaning, and that is all it does. Ask it for "papers about attention published after 2020" and the words published after 2020 are treated as more subject matter to match, not as a rule to enforce. A 2017 paper that is squarely about attention will score well and be returned, because nothing in a cosine score knows what a year is.
Ask
off = plain similarity search over the whole corpus
Top result
1 — The question, split in two
—2 — Filter, then rank what survives
Where the parse comes from
In a real self-query retriever an LLM turns the sentence into that filter, given a description of the metadata schema. The three parses here are written by hand as a labelled example of what it would emit — no model runs in this page. The filtering and the ranking below are really computed.
Self-Query Retrieval: A Practical Guide
Half the sentence is a subject. Half of it is a WHERE clause.
Two questions in one sentence
A self-query retriever hands the sentence to an LLM along with a description of the metadata each document carries, and asks it to return two things: the semantic query — the part that should be embedded and compared — and a structured filter over the metadata, in the form of real comparisons like year > 2020. The filter runs first as an ordinary database-style predicate; similarity then ranks only the documents that survived it. The constraint becomes a guarantee rather than a hint.
Turning a question into a filter plus a search
Many questions contain two different kinds of requirement, and a plain vector search can only handle one of them.
"What did the 2023 security reports say about phishing?"
"Phishing" is a semantic requirement — find passages about it. "2023" and "security reports" are structured requirements — filter on a date field and a document-type field.
Embedding the whole question and searching gives poor results: the year and the document type become part of a blended vector, so documents from 2019 with the word "2023" in the body may rank above the right ones.
A self-query retriever uses a language model to split the question:
"What did the 2023 security reports say about phishing?"
→ query: "phishing" + filter: year = 2023 AND type = "security_report"
Then it runs a filtered vector search: the filter constrains the candidate set, and the semantic query orders what remains.
What it needs to work
Two things, and the first is the part that is easy to under-invest in.
Metadata on every document, populated at indexing time. The filter can only reference fields that exist. Adding a date field later means re-indexing.
A schema description given to the model, so it knows which fields exist and what values they take:
metadata_fields = [
{"name": "year", "type": "integer", "description": "Publication year"},
{"name": "doc_type", "type": "string",
"description": "One of: policy, report, guide, minutes"},
{"name": "department", "type": "string",
"description": "Owning department, e.g. Security, HR, Finance"},
]The model is prompted with that schema and the user's question, and asked to return a structured object with a query string and a filter expression. Constraining the output to a JSON schema is worth doing — it converts a class of malformed-filter failures into a validation error.
The failure modes
This is a language model producing a filter, so it fails in language-model ways, and each failure needs a specific guard.
Invented field names. The model filters on publication_date when the schema says year. Validate the filter against the schema and drop or repair unknown fields.
Invented values. A filter for doc_type = "reports" when the actual value is "report". Enumerate allowed values in the schema, and fuzzy-match or reject.
Over-filtering. The model adds a constraint the user did not intend, and the result set is empty. The standard remedy is to fall back to an unfiltered search when the filtered one returns nothing, and to say so in the response.
Under-filtering. The model ignores a constraint the user did state, and irrelevant results come back. Harder to detect automatically; it shows up in evaluation.
Latency. An extra model call before retrieval begins, adding several hundred milliseconds. Cache the parse for repeated queries.
The practical shape that results:
parsed = parse_query(question, schema) # LLM call
parsed.filter = validate(parsed.filter, schema) # drop unknown fields/values
results = index.search(parsed.query, filter=parsed.filter, k=10)
if not results:
results = index.search(parsed.query, k=10) # graceful degradationExploration guide
- Read the parse for the first query. Note what was removed from the semantic half: only attention is left to match on, because the date has become a filter instead.
- Turn the filter off. This is plain similarity search over everything. A 2017 paper takes the top spot — it is genuinely the best match for "attention", and it breaks the constraint the reader stated.
- Turn it back on. The 2017 paper is struck out with the reason it failed, and the top result is now both about attention and inside the requested date range.
- Try "short surveys on retrieval". Two constraints this time, on different fields and different types — a number and a category — combined with AND.
- Try "anything about attention". No constraint in the sentence, so the filter is empty and the retriever degrades gracefully into ordinary similarity search.
The short of it
Embeddings compare meaning and cannot enforce a rule, so any constraint left inside the query text is at best a weak hint. Self-querying separates the sentence into the part worth embedding and the part worth executing as a filter, which turns "after 2020" from a phrase competing for cosine similarity into a predicate that simply removes the documents that fail it. The trade is an extra LLM call before retrieval, and a hard dependency on documents actually carrying the metadata the filter names.
Where it earns its place
Time-bounded questions. "Last quarter", "since the policy changed", "in 2023". Dates are exactly the kind of constraint embeddings handle badly and filters handle perfectly.
Faceted corpora. Documents with meaningful categories — department, product line, region, document type, author. Users naturally phrase questions that reference them.
Numeric constraints. "Products under £50", "incidents affecting more than 100 users". A vector search cannot express an inequality.
Mixed catalogues. E-commerce is the canonical case: "waterproof jacket under £80 in medium" is one semantic term and three filters.
Where it is unnecessary: a homogeneous corpus with no meaningful metadata, or an interface where the user already selects filters in the UI. If the filters are available as controls, use the controls — they are free and cannot be hallucinated.
Related retrieval strategies
| Strategy | What it adds |
|---|---|
| Self-query | Extracts filters from the question |
| Multi-query | Rewrites the question several ways and unions the results |
| HyDE | Generates a hypothetical answer and searches with that |
| Parent document | Retrieves small, returns large |
| Reranking | Reorders candidates with a cross-encoder |
These compose. A realistic pipeline: self-query to extract filters, multi-query to broaden the semantic search, hybrid retrieval over the filtered set, reciprocal rank fusion, then a cross-encoder rerank.
Each stage adds latency, so the order to add them in is by measured benefit. In most systems, hybrid search and reranking are the largest gains; self-query matters specifically when the corpus has metadata users refer to.
Splitting a sentence into a filter and a search
A self-query retriever reads one sentence and produces two things -- a metadata filter and a semantic query. This runs both halves over a small collection, and shows what happens at each of the failure modes the article names.
Questions people ask
Does it need a large model? No — extracting a filter from a question is a small structured task, and a fast, cheap model handles it well. Use structured output constraints.
What if the model produces an invalid filter? Validate against the schema and fall back to an unfiltered search. Never pass an unvalidated filter to the store.
How much latency does it add? One model call, typically 200–600ms. Cache by query text.
Can I use it without a vector database? The filtering needs a store that supports metadata predicates. Most do; plain in-memory vector arrays generally do not.
Should filters be hard or soft? Hard for correctness constraints such as permissions and dates. Consider soft (a score boost) for preferences, where over-filtering to zero results is worse than imperfect ordering.
Does it replace hybrid search? No — it constrains the candidate set. Hybrid search still decides the ordering within it.
Recap in one screen
- Questions often mix a semantic requirement with structured constraints; embeddings handle only the first.
- A model parses the question into a query string plus a metadata filter, which is then applied at search time.
- It requires metadata populated at indexing time and a schema description in the prompt.
- Validate the generated filter against the schema, and fall back to unfiltered search when it returns nothing.
- Most valuable for dates, numeric constraints and faceted corpora; unnecessary when the UI already offers filters.