Retrieval is the product, not the model
Teams obsess over which model to use and under-invest in retrieval, then wonder why answers are confidently wrong. In practice the quality of what you put into the context window matters far more than the last few points on a model benchmark.
We treat retrieval as the core of the product. If the right chunk is in context, almost any capable model gives a good answer. If it is not, no amount of prompt engineering saves you.
Chunking with intent
Naive fixed-size chunking shreds tables, code blocks and lists, and the model inherits the damage. We chunk along the document's natural structure — headings, sections, semantic boundaries — and attach metadata so we can filter before we ever embed.
Overlap is tuned per corpus, not guessed once. We measure retrieval recall directly and let the data set the window.
Grounding and guardrails
Every answer cites the sources it used, and we hold the model to them. When confidence or retrieval scores fall below a threshold, the system says it does not know instead of inventing an answer.
Hybrid search — dense vectors plus keyword — catches the cases pure embeddings miss, especially exact identifiers, error codes and product names.
Evaluate continuously
We ship a golden evaluation set with every pipeline and run it on each change. Regressions in retrieval recall or answer faithfulness block the deploy, the same way a failing test would.
That discipline is what keeps a RAG system accurate in the wild long after the impressive demo.