Retrieval & RAG

Which vector database should we use for RAG?

Short answer

If your corpus is under roughly one million chunks and you already run PostgreSQL, use pgvector. It removes a system from your stack, keeps vectors transactionally consistent with your metadata, and performs well at that scale. A dedicated vector database earns its place when you need very large scale, sub-50ms retrieval at high concurrency, complex metadata filtering, or built-in hybrid search and reranking you would otherwise build yourself. Choose on operational fit rather than benchmark charts: retrieval quality is determined far more by chunking, embedding choice and reranking than by which index you store the vectors in.

Last reviewed 15 June 2026

The uncomfortable answer

The vector store is rarely what determines whether your RAG system works. Chunking strategy, embedding model choice, hybrid search and reranking all move retrieval quality far more than the choice of index. Teams spend weeks on this decision and days on chunking, which is exactly backwards.

Pick the option that fits your operations and move on to the things that matter.

Start with pgvector if you can

If you already run PostgreSQL, pgvector is the default answer:

No new system to operate, monitor, back up, secure or pay for.
Vectors and metadata in one transaction. No consistency gap between your documents table and your index, which is a real source of bugs in two-system setups.
Filtering is just SQL. Permission checks and metadata filters compose naturally, which matters a lot for access-controlled retrieval.
Performance is fine for corpora into the low millions of chunks with HNSW indexing.

The limits are real but arrive later than most teams expect: index build time grows awkward at very large scale, and heavy vector query load competes with your transactional workload unless you use a read replica.

When a dedicated vector database earns its place

Scale beyond a few million chunks, where purpose-built index management and sharding matter.
Tight latency at high concurrency. Consistent sub-50ms retrieval under sustained load.
Complex metadata filtering at scale, where naive pre- or post-filtering degrades recall badly.
Built-in hybrid search and reranking, saving you from assembling BM25 and a cross-encoder yourself.
You do not run PostgreSQL and do not want to start.

What actually differentiates the options

Ignore recall benchmarks. At typical configurations the mature options are within noise of each other. Judge on:

Filtering semantics. How does filtering interact with the approximate index? Some options degrade recall sharply under selective filters. Test this with your real filter patterns.
Update behaviour. How costly are inserts and deletes? If your corpus changes hourly this dominates.
Operational model. Managed or self-hosted, and does the managed region list include the jurisdiction your data must stay in? For European clients this frequently eliminates otherwise-attractive options.
Hybrid support. If it does not do keyword search, you will run a second system anyway.
Cost at your scale. Per-vector pricing looks cheap until you re-embed a large corpus.

How I choose in practice

I benchmark two candidates against the client’s actual corpus and their actual filter patterns, using the labelled evaluation set built earlier in the project. It takes about two days and replaces an argument with a number. Nine times out of ten for European mid-market clients the answer is pgvector, and the two days are still worth spending, because they close the question for good.