The promise and the boundary
A document assistant has an appealing promise: ask a question in ordinary language and receive a useful answer grounded in your information. Retrieval-augmented generation, or RAG, is one approach to building that experience. It retrieves relevant material and supplies it to a generative model as part of answering a question.
The foundational paper by Lewis and colleagues combined a language model with a retrievable document memory and evaluated the approach on knowledge-intensive tasks. [1] It is a research basis for the approach, not a guarantee that any application connected to a document store will answer reliably.
Our argument is that a useful RAG product should be designed around evidence. That changes the questions a team asks: what was retrieved, why was it relevant, which claims does it support, and what should happen when it is not enough?
Separate finding from answering
Consider a fictional company handbook assistant asked whether a particular expense is reimbursable. It needs the applicable policy, the relevant version, and any conditions that affect this employee’s situation. An answer can fail because the right policy was not retrieved, because the retrieved policy was outdated, or because the model misread a correct passage.
These failures call for different fixes. Rewriting the answer prompt does little if the system never receives the right document. Improving retrieval does not guarantee the model will preserve a qualification in the evidence. Evaluate the two stages separately.
For retrieval, create questions paired with the passages needed to answer them. Check whether those passages appear in the retrieved set. For generation, supply the necessary passages directly and inspect the resulting answer. This isolates where useful evidence is being lost.
Treat a citation as a claim to check
A link at the end of a paragraph can look authoritative without establishing much. Ask whether the source actually supports the sentence, whether its context changes the meaning, and whether the answer quietly adds facts the source does not contain.
For a practical interface, place citations near the relevant claims and let the reader inspect the underlying passage. Show useful document metadata, such as the version or effective date when the source provides it. Avoid presenting a document’s retrieval time as though it were its publication date.
Our suggested evaluation records unsupported claims, inaccurate citations, and omitted qualifications separately. This gives a team more information than a single “helpful answer” rating. It also makes a polished but weakly supported answer harder to mistake for a success.
Respect the evidence boundary
A document collection is not the whole world. The correct response may be that the available material does not answer the question. That response should explain what is missing, and, where useful, identify the additional document or clarification needed.
Build unanswerable questions into the evaluation set. Include questions that sound similar to covered topics but depend on absent information. A system should not stretch an adjacent passage into evidence for a different claim.
Permissions belong in this design too. A document assistant should retrieve only material the requesting person is allowed to access. The generated answer must not become a route around the source system’s restrictions. Apply access rules before the material reaches the model and test those boundaries as part of the application.
Make maintenance visible
Documents change. A policy may be superseded, a product specification corrected, or an owner replaced. Decide how updates and deletions reach the retrieval system, and how the application handles conflicting versions. There is no single ideal refresh interval; it should reflect how quickly the underlying information changes and what mistakes would cost.
Our proposed launch checklist is deliberately concrete: known-answer questions, unanswerable questions, permission checks, citation checks, and a documented update path. Record failures against those categories so that future changes can be tested against the same evidence.
A RAG system earns trust by making its support inspectable and its limits legible. Connecting documents supplies an opportunity to ground an answer. The rest of the product must make that opportunity count.
Sources & further reading
- Lewis et al. — Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks
2020 paper describing and evaluating the foundational RAG approach.
Sources checked 5 October 2026. Links support the specific research claims identified in the article; recommendations and examples are ReasoningReport analysis.
