---
title: What is retrieval-augmented generation (RAG)?
source: https://rubradigital.com/answers/what-is-rag
site: Rubra Digital
description: A plain explanation of retrieval-augmented generation: what RAG is, how the pipeline works step by step, when to use it, and where it fails.
updated: 2026-07-14
topic: rag
also_asked: What does RAG stand for in AI? | How does retrieval-augmented generation work? | What is a RAG pipeline? | Why use RAG instead of just prompting a model?
---

# What is retrieval-augmented generation (RAG)?

**Summary:** Retrieval-augmented generation (RAG) is a technique where a language model answers a question using documents fetched from your own data at the moment the question is asked, rather than relying only on what it learned during training. A retrieval step searches your corpus for relevant passages, those passages are placed into the model prompt as context, and the model generates an answer grounded in them. RAG lets a model use private, current or proprietary information it was never trained on, and makes answers citable back to a source document.

## The pipeline, step by step

A RAG system has two phases: one that runs ahead of time, and one that runs per
question.

**Indexing (ahead of time)**

1. **Ingest.** Documents are collected from their sources: file shares, a CMS, a
   ticketing system, a database.
2. **Parse.** Each document is converted to text. This is where most quality is
   won or lost, particularly with PDFs, tables and scanned pages.
3. **Chunk.** Text is split into passages small enough to be useful as context
   but large enough to remain self-contained.
4. **Embed.** Each chunk is converted into a vector that represents its meaning.
5. **Index.** Vectors and their source text are stored in a searchable index.

**Retrieval and generation (per question)**

6. **Embed the question** using the same model used for the chunks.
7. **Search** the index for the most similar chunks, usually combining vector
   similarity with keyword search.
8. **Rerank** the candidates with a model that scores relevance more precisely
   than the initial search can.
9. **Assemble the prompt.** The question plus the top passages, with
   instructions to answer only from them.
10. **Generate** the answer, with citations pointing back to the source
    passages.

## Why it exists

A language model's knowledge is fixed at training time and contains nothing
private to your organisation. Three problems follow: it does not know your
internal information, it does not know what changed last week, and it cannot
show you where an answer came from.

RAG addresses all three by moving the knowledge out of the model's weights and
into a retrievable store. Update the store and the system's knowledge updates
immediately, with no retraining.

## Where RAG works well

Questions whose answers already exist in writing but are expensive to find:
policy and regulatory libraries, technical manuals, contract portfolios, support
histories, research literature.

## Where RAG struggles

- **Aggregation.** "How many contracts expire this quarter?" is a database
  query, not a retrieval problem. RAG will find some contracts, not all of them.
- **Answers spread thinly across many documents.** Retrieval returns a handful
  of passages; if the answer needs fifty, it will not fit.
- **Information that was never written down.** No retrieval strategy recovers
  what does not exist.
- **Reasoning over structure.** Comparisons, calculations and trend analysis are
  better served by giving the model a tool that queries structured data.

The most common production architecture is a hybrid: retrieval for the
document-shaped questions, tool calls for the structured ones, and a routing
step that decides which is which.

---

Source: https://rubradigital.com/answers/what-is-rag
Rubra Digital. Independent LLM and RAG consulting for regulated and document-heavy organisations in Europe and North America.
Contact: hello@rubradigital.com