Modules/Gen AI/ Self-Query Lab

Self-Query Retriever

Some of what a question asks for is a subject, and some of it is a constraint. Similarity search can only answer the first half — unless you pull the constraint out first.

Overview

The idea in brief

Similarity search compares meaning, and that is all it does. Ask it for "papers about attention published after 2020" and the words published after 2020 are treated as more subject matter to match, not as a rule to enforce. A 2017 paper that is squarely about attention will score well and be returned, because nothing in a cosine score knows what a year is.

Ask

off = plain similarity search over the whole corpus

Top result

Document
Obeys the ask

 

1 — The question, split in two

2 — Filter, then rank what survives

Where the parse comes from

In a real self-query retriever an LLM turns the sentence into that filter, given a description of the metadata schema. The three parses here are written by hand as a labelled example of what it would emit — no model runs in this page. The filtering and the ranking below are really computed.

Self-Query Retrieval: A Practical Guide

Half the sentence is a subject. Half of it is a WHERE clause.

Two questions in one sentence

A self-query retriever hands the sentence to an LLM along with a description of the metadata each document carries, and asks it to return two things: the semantic query — the part that should be embedded and compared — and a structured filter over the metadata, in the form of real comparisons like year > 2020. The filter runs first as an ordinary database-style predicate; similarity then ranks only the documents that survived it. The constraint becomes a guarantee rather than a hint.

Exploration guide

  1. Read the parse for the first query. Note what was removed from the semantic half: only attention is left to match on, because the date has become a filter instead.
  2. Turn the filter off. This is plain similarity search over everything. A 2017 paper takes the top spot — it is genuinely the best match for "attention", and it breaks the constraint the reader stated.
  3. Turn it back on. The 2017 paper is struck out with the reason it failed, and the top result is now both about attention and inside the requested date range.
  4. Try "short surveys on retrieval". Two constraints this time, on different fields and different types — a number and a category — combined with AND.
  5. Try "anything about attention". No constraint in the sentence, so the filter is empty and the retriever degrades gracefully into ordinary similarity search.

The short of it

Embeddings compare meaning and cannot enforce a rule, so any constraint left inside the query text is at best a weak hint. Self-querying separates the sentence into the part worth embedding and the part worth executing as a filter, which turns "after 2020" from a phrase competing for cosine similarity into a predicate that simply removes the documents that fail it. The trade is an extra LLM call before retrieval, and a hard dependency on documents actually carrying the metadata the filter names.

Predict, then reveal

About to run: Try "short surveys on retrieval". Before it does — what happens to the readout?

Committing to an answer first is the point — the reveal runs the experiment on the visualisation above and reads the real value back, so nothing here is scripted.

Check yourself

0 of 3

Answer without scrolling back up.

  1. Why can plain similarity search not honour "published after 2020"?

  2. A self-query retriever splits the question into:

  3. In what order do the two halves run?

Cheat sheet

Self-Query Retriever

Some of what a question asks for is a subject, and some of it is a constraint. Similarity search can only answer the first half — unless you pull the constraint out first.

GEN AI · vizlearn.in/gen_ai/self_query_retriever.html

About the author

Ashish Jangra builds and maintains VizLearn. Every module here is written and the visualisation behind it hand-built, so the numbers in a readout come from the same code that draws the picture. Corrections are genuinely welcome and get priority over everything else — if a page states something wrong, or an animation misrepresents what the algorithm does, get in touch.