Retrieval gives an answer generator relevant evidence from a document collection. The pipeline has two distinct opportunities to fail: retrieving the wrong passages and making unsupported claims from the right passages. Evaluate both rather than treating the final answer as a single opaque result.
Build the evidence trail
Store a document identifier, source location, access scope, and text. Split documents at meaningful boundaries so a passage retains enough context. Compare a simple keyword baseline with semantic retrieval on your own questions before introducing more components.
For a question about refunds, retrieving a cancellation heading without its eligibility conditions may produce a misleading answer. Inspect passages, not just similarity scores. Apply authorization filters before evidence is exposed to the model.
Return citations using identifiers from the retrieved set:
{"answer":"Refunds require an unused booking.","source_ids":["policy-4-p2"]}
Validate that each identifier exists. Then check that its passage supports the claim. Citation existence alone is not grounding. If the supplied documents do not answer the question, return an explicit unsupported state.
Exercise
Prepare ten questions with known supporting passages and five questions the collection cannot answer. Try two passage sizes. Compare retrieval hits, answer support, and response time.
Check: every cited passage is accessible to the requesting user; unsupported questions are handled explicitly; instructions embedded in a document cannot override application policy.