How much does it cost to build a RAG system?

Build cost: what the range actually depends on

The two things that move build cost most are not the ones teams expect.

Document complexity. A corpus of clean, digital, text-first documents is a straightforward ingestion job. A corpus of scanned PDFs, engineering drawings, documents where the meaning lives in tables, or files exported from a system that lost the structure can double or triple the parsing effort. This is the single most common source of budget overrun, and it is knowable in week one if anyone looks.

Number of source systems. One SharePoint library is one integration. Six systems with different auth models, permission schemes and update cadences is a different project, and permission-aware retrieval across all of them is harder still.

Rough bands, for delivery with an experienced partner:

Scope	Typical range
Prototype on a clean single corpus	€15,000 – €30,000
Production system, one corpus, internal users	€60,000 – €95,000
Production system, several sources, permission-aware	€95,000 – €180,000
Customer-facing, regulated, multilingual	€180,000+

An internal build is not free: the same work is roughly three to six months of two capable engineers, plus the learning curve if this is their first such system.

Running cost: usually the smaller number

For a system handling around 3,000 queries a day over a corpus of a few hundred thousand chunks, a representative monthly bill in 2026:

• Generation. €300 to €2,500, depending heavily on model choice and how much context is stuffed into each prompt
• Embeddings. €20 to €150. Re-embedding on document change is minor
• Vector storage. €0 if you use pgvector on a database you already run, up to €700 for a managed vector service at this scale
• Reranking. €50 to €400
• Hosting and observability. €100 to €500

Teams routinely over-provision here. The most common waste is sending 8,000 tokens of context when 2,000 would answer the question equally well, and routing every request to the largest available model regardless of difficulty.

What reduces cost without hurting quality

• Caching. Exact-match plus semantic caching typically removes 25–45% of calls in assistant-style workloads.
• Right-sizing the model per request. Route simple lookups to a small model, hard synthesis to a large one. Verify with evaluation, because this quietly degrades some task types.
• Tighter retrieval. Better reranking means fewer passages needed, which means shorter prompts and lower cost per call.
• Prompt-prefix caching where the provider supports it, for the static instruction block.

The cost nobody budgets

Evaluation. Building the labelled set and the harness is typically 15–25% of the project, and it is the line most often cut. It is also the line that determines whether you can safely change anything after launch. Cutting it does not save money, it defers a larger cost to month six.