Evaluation & quality
How do you reduce hallucinations in an LLM system?
Short answer
Most hallucinations in production systems are retrieval failures rather than model failures. The model was asked to answer from context that did not contain the answer. Fix retrieval first and measure recall@k, because no prompt change compensates for the right passage never arriving. Then constrain generation to cite retrieved passages and to refuse when context is insufficient, add a verification pass that checks each claim against the source for high-stakes answers, and measure groundedness on every change so regressions are caught in CI rather than by users.
Last reviewed
Diagnose before you treat
The first question is always: was the correct information in the context window?
Log the retrieved passages alongside every answer and check a sample of failures. Teams that do this are usually surprised. In most RAG systems I audit, the majority of wrong answers had wrong or missing context. That is a retrieval bug wearing a hallucination costume, and no amount of prompt tuning fixes it.
Only once retrieval is measurably good does it make sense to work on generation.
Retrieval-side fixes
- Hybrid search. Pure vector search misses exact terms: product codes, error numbers, proper nouns. Combining it with BM25 keyword search closes a surprising share of failures.
- Reranking. A cross-encoder reranker over the top 50 candidates consistently improves precision at the top of the list, and it is one of the highest-return changes available.
- Chunking that respects structure. A clause split from its heading loses the context that made it findable.
- Query rewriting. Users ask questions that do not match how documents are written. Rewriting the query before search, or generating several variants and merging the results, helps materially on conversational systems.
Generation-side fixes
- Instruct for grounding and refusal explicitly. The system prompt should require answers to come only from the provided context and require an explicit “the provided documents do not cover this” when they do not.
- Require citations per claim. Attribution pressure measurably reduces invention, and it makes errors visible when they happen.
- Lower temperature for factual tasks. Not a cure, but free.
- Do not overfill the context. Stuffing thirty marginal passages in makes the answer worse, not better, because relevant material competes with noise.
Verification for high-stakes answers
Where an error is expensive, add a second pass: take each claim in the generated answer and check it against the cited passage. This roughly doubles cost and latency, so reserve it for the paths that warrant it. It catches the residual errors that grounding alone does not.
Make refusal a first-class outcome
A system that says “I do not know” is more useful than one that guesses, and users calibrate their trust accordingly. Build a subset of deliberately unanswerable questions into your evaluation set and measure the refusal rate on them. If it is low, your system is confidently wrong on exactly the questions where being wrong matters most.
The honest limit
Hallucination cannot be reduced to zero with current models. It can be reduced to a measured, acceptable rate for a defined task, with a known failure mode and a human checkpoint where the stakes require one. Any vendor claiming elimination is describing a demo, not a production system.
The useful goal is not perfection. It is knowing your groundedness number, having it under regression control, and designing the surrounding process for the residual error rate.
People also ask this as
- Why does our RAG chatbot make things up?
- Can hallucinations be eliminated completely?
- How do you make an LLM say we do not know?