Evaluation & quality
Your evaluation set is the real product
Short answer
Models change, frameworks are replaced, architectures get rewritten. A well-built labelled evaluation set survives all of it and makes every future decision cheaper. It is the highest-leverage artefact in an AI project and the one most often cut.
Ask a team what their AI project produced and they will describe an application. Ask them in eighteen months what still has value and the honest answer is usually: the evaluation set.
What outlasts what
In three years the model you launched on will be several generations old. The framework will have been replaced or abandoned. The retrieval architecture will have been rewritten at least once.
The set of 250 real questions, each paired with the passage that answers it and validated by someone who knows the domain, is still correct. It was correct before your architecture and it will be correct after it.
It is the only artefact in the project that does not depreciate.
What it makes possible
Model migration becomes an afternoon. A new model appears. Run the suite, compare the numbers, decide. Without it, migration is a multi-week exercise in subjective comparison that nobody trusts.
More people can safely change things. With a gate in CI, a junior engineer can adjust a prompt. Without one, changes are bottlenecked on whoever has the best intuition, which does not scale and does not survive that person leaving.
Vendor claims become checkable. Someone will tell you their platform is more accurate. You can find out in an hour.
Regulators and auditors get a real answer. “How do you know it performs as claimed?” has a document behind it.
Why it gets cut
It is expensive in the currency projects are shortest on: subject-matter expert attention. Getting a specialist to label which passage correctly answers each of 250 questions is days of a busy person’s time, and it produces nothing demonstrable.
Meanwhile the prototype already looks impressive. The pressure to move to features is real, and evaluation is the easiest thing to defer.
Build it early, deliberately
Real questions. From search logs, tickets, or a week of asking users what they actually want to know. Invented questions cluster on the easy cases.
Stratified. Simple lookups, multi-document synthesis, near-miss distractors, ambiguous phrasings, and deliberately unanswerable questions. Roughly balanced, not naturally distributed.
Expert-validated ground truth. The expensive, unavoidable part.
Version controlled. It changes as the product changes; you want the history.
150 to 300 cases. Curated beats scraped by a wide margin.
The test
If your team cannot answer “did last week’s change make the system better?” with a number, you do not have an evaluation set. You have opinions with a deployment pipeline attached.
Frequently asked questions
How much of an AI project budget should go to evaluation?
Between 15 and 25 percent for a first build. It is consistently the line item that gets cut when budgets tighten, and consistently the one that determines whether the team can safely change anything after launch. Cutting it does not save money. It defers a larger cost to the point where someone needs to change a prompt and nobody can tell whether it helped.