Evaluation Datasets

This unlocks the rest of LLM Evaluation. Without a fixed dataset you cannot compare two versions of a model, you cannot run a regression gate, and you cannot trust any metric. Master this, and you can build the gold, adversarial and slice-based sets that the rest of the parent concept stands on, and you can understand how offline evaluation, benchmarking and deterministic validators all depend on a stable reference.

Think of a recipe you know by heart. If you change one ingredient — say, swap sugar for honey — you cannot tell whether the cake is better or worse unless you bake the same cake twice, once with each, and compare. If you changed the recipe AND the oven AND the pan, you would blame the oven when it was the honey all along. An evaluation dataset is that fixed recipe: a set of inputs you never change while you are testing. You run your old model on it, you run your new model on it, and the only difference is the model. The dataset is the measuring stick. It does not need to be perfect; it needs to be stable. A slightly imperfect measuring stick that never changes tells you which model is better. A perfect measuring stick that you swap halfway tells you nothing.

A schoolteacher giving a test. The students (models) take the same paper (dataset) on different days. If the teacher changed the questions between students, they could not say who knew more — only who happened to see the easier version. A good test is fixed, but it is also curated: the teacher includes easy questions to check basics, hard ones to stretch, and a couple of trick ones to catch students who memorised instead of understood. That is a gold set (questions with correct answers), an adversarial set (questions designed to trip you), and a slice-based set (questions grouped by topic, so you can see where each student is weak). The test does not have to cover everything — no test does — but it has to stay the same for everyone. Where the analogy breaks down: a school test is written by one teacher and fixed for years. In AI, your dataset must evolve as your product changes. When you change the questions, you must version them — call it Test v2 — and keep the old questions around so you can still compare old results. Another break: a test is judged by a human, but evaluation datasets are often judged by metrics or models, and those judges drift. So the dataset must be frozen not just during a test run, but across months, so you can detect when the judge has shifted.

An evaluation dataset is a curated collection of inputs with expected outputs or expected properties, fixed and versioned, used as the unchanging reference against which model changes are measured.

You have notes on Offline vs Online Evaluation: this dataset is the core of offline evaluation — you run it before release against fixed inputs. It also powers Training-Time vs Inference-Time Evaluation: at training time you use a held-out slice of the dataset to prevent overfitting; at inference time you run the same dataset to catch regressions. You have notes on Benchmarking: a benchmark is a shared, public evaluation dataset — but it gets contaminated when models train on it, so your internal dataset must be yours. You have notes on Deterministic Validators and Gold vs Adversarial Sets: both are types of data within your dataset. The dataset is the substrate for all of them.