Retrieval, and its three failure points

What "it searched for that" actually means

Most serious AI products no longer answer from the model alone. They retrieve first. The pattern has a name — retrieval-augmented generation, usually shortened to RAG — and it is worth understanding at the level of the plumbing, because it is now behind most enterprise assistants, most "chat with your documents" tools, and the search summaries you meet daily.

The pipeline has four steps.

1. Chunking. Your documents are cut into pieces, typically a few hundred words each.
2. Embedding. Each chunk is converted into a list of numbers — an embedding — positioned so that pieces of text with similar meaning sit close together in that space.
3. Retrieval. Your question is embedded the same way, and the chunks nearest to it are pulled out. Usually three to ten of them.
4. Generation. Those chunks are pasted into the model's context along with your question and an instruction like "answer using only the provided material".

That instruction is a request, not a constraint. Nothing in the machinery enforces it.

Failure one: it retrieved the wrong thing

Similarity of meaning is not relevance. Ask about the 2024 refund policy and the retriever may return the 2019 refund policy, because the two chunks are nearly identical in meaning — that is precisely why they score as similar.

This is the single most common cause of confidently wrong answers from document assistants, and it produces a distinctive symptom: the answer is coherent, plausible, and describes a real policy that is not the current one. Anything where recency, version or jurisdiction matters is exposed. Embeddings capture topic well and capture "which of these near-identical documents applies to me" poorly.

The second version of this failure is the chunk boundary. Cut a document into 400-word pieces and a table's heading ends up in one chunk and its numbers in another. The retrieved piece is then technically correct and missing the thing that made it meaningful.

Failure two: nothing relevant existed and it answered anyway

If the answer is simply not in the corpus, the retriever still returns its best matches — retrieval returns the nearest chunks, not the sufficient ones. The model then has irrelevant material and a question, and its trained instinct is to be helpful.

A well-built system checks the similarity score and refuses below a threshold. Most systems do not, because refusing looks worse in a demo than answering. The observable sign is an answer that cites documents which turn out to be about a neighbouring topic.

Failure three: the citation does not support the sentence

This is the one that catches careful people, because the interface has already reassured them. The system attaches a source link to each claim. The link works. The document is genuine. And the sentence overstates it, merges two chunks into a claim neither made, or attaches a number from one paragraph to a subject from another.

Studies of commercial search assistants have repeatedly found meaningful rates of citations that do not support the statement attached to them. The presence of a footnote is a design element, not a verification.

What to do as a user

Click the citation and find the sentence. Not the document — the sentence. If you cannot locate text supporting the claim, treat the claim as unsupported regardless of how reasonable it sounds.

Ask what it did not find. "Which parts of my question were not covered by the retrieved material?" often produces a much more honest answer than the original question did, because you have given it permission to report a gap.

Watch for version and date. In any corpus containing successive editions of the same document, ask explicitly for the date of the source and check it.

Give it the document directly when you can. Pasting the actual contract in beats retrieval over a corpus containing that contract, because you have removed the step that can pick the wrong one. The privacy cost of doing so is the subject of the next module.

What to do as a builder

Surface the retrieved chunks, not just the citation. Set a relevance floor and let the system say it does not know. Include document dates and versions in the chunk text itself, so the model can see them. And measure the two things separately: how often the right chunk was retrieved, and how often the answer was faithful to what was retrieved. A system that is failing at retrieval and a system that is failing at faithfulness need completely different fixes, and one aggregate accuracy number hides which you have.