RAG System Development & Consulting

Most retrieval-augmented generation projects do not fail at the model. They fail at the boring parts: a PDF parser that silently drops tables, a chunking strategy that splits a clause from its heading, an index that returns plausible-looking passages that do not actually contain the answer.

I build the boring parts properly.

What I actually do

Start with the corpus, not the model

The first week is spent on your documents. I measure what your corpus really looks like: how many documents are scanned rather than digital, how much of the meaning lives in tables, and how often a single answer requires stitching two documents together. Those facts determine the architecture far more than the choice of model does.

Build a labelled evaluation set before writing retrieval code

I sit with your subject-matter experts and assemble a set of real questions with known-correct source passages. It is unglamorous work and it is the single most valuable thing in the project: without it, every subsequent decision is guesswork, and every “this feels better” is unfalsifiable.

Tune retrieval against that set

Chunk size, overlap strategy, embedding model, hybrid weighting between semantic and keyword search, reranking, metadata filters. Each of these is an experiment with a number attached. I typically run thirty to sixty configurations and report recall@k and mean reciprocal rank for each, so the final architecture is a decision you can audit rather than a preference you have to trust.

Constrain generation and measure groundedness

The generation layer cites the passages it used and declines to answer when the retrieved context does not support one. Refusal behaviour is a feature, and in regulated settings it is often the most important one. I measure groundedness and citation accuracy on every build.

Hand it over properly

You get the evaluation harness, the architecture decision records explaining why each choice was made, a runbook for the failure modes I found, and working sessions with your engineers. The goal is that the next change to the system is made by your team, not by me.

Where this pays back

Retrieval works best where the answer already exists in writing but is expensive to find: regulatory and policy libraries, technical documentation and service manuals, contract portfolios, historical support tickets, clinical or research literature, and internal knowledge that currently lives with three people who are always busy.

It works poorly where the answer requires calculation, judgement under uncertainty, or data that was never written down. I will tell you which one you have during discovery, before you have spent a budget finding out.