Start with the knowledge contract
Define which sources are authoritative, how quickly updates must appear, who may retrieve each document, and what the answer should do when evidence is missing. These rules determine chunking, indexing, access filters, and freshness jobs.
Store source identity and version with every chunk. Without lineage, citations cannot be trusted and stale answers are hard to debug.
Retrieval is a pipeline
A robust pipeline often combines lexical and semantic search, metadata filters, reranking, deduplication, and diversity control. More chunks do not automatically produce a better answer; they can dilute the evidence and increase input cost.
Measure retrieval recall separately from answer quality. If the required passage never reaches the model, changing the generator cannot fix the problem.
Construct context deliberately
Allocate a context budget by source importance, remove boilerplate, preserve document boundaries, and give the model stable citation identifiers. Include instructions for conflicts and missing evidence.
Use a smaller model for query rewriting or document classification only after it passes a workload-specific evaluation. The retrieval step is often high-volume and a strong cost-optimization candidate.
Observe the full chain
Trace query transformation, retrieved sources, reranker scores, context size, generation route, latency, cost, citations, and user outcome. Privacy rules may require storing identifiers and scores without storing the full source text.
Build an evaluation set with answerable, unanswerable, conflicting, stale, and access-restricted questions. A RAG system is trustworthy when it knows when not to answer.