Eval-Driven Development

Concept of the skill

Eval-driven development is the practice of building LLM-integrated systems by writing evaluations before and alongside the system, with each eval defining a behavioral criterion the system must satisfy on a representative input set, and the suite's aggregated signal gating every change to prompt, model, retrieval, context assembly, scaffolding, tooling, or orchestration. Five primitives: (1) eval dataset — curated input examples representing production plus edge cases, prior failures, and held-out slices, typically JSONL of {input, reference} records checked into version control; (2) evaluation function — per-example grader producing a score, with mechanism chosen from programmatic, model-graded, human-graded, preference comparison, trace check, or hybrid; (3) aggregation — statistical summary across the dataset (pass-rate, weighted pass-rate, stratified slice scores, distribution) with sample size and either a confidence interval or a defined minimum-detectable-change threshold, plus cost/latency side metrics; (4) iteration loop — eval → diagnose → change one meaningful variable → re-eval → compare, with the suite stable while the system changes; (5) regression budget — per-eval/per-slice merge policy: gating (no regression allowed), optimizing (improvement gates merge), watchful (tracked, not gated), human-signoff, or cost-aware threshold (a pass-rate gain only counts if it clears the eval's cost/latency budget — a 1pp quality gain at 3× inference cost is a regression).

Replaces "ship LLM systems based on how the model feels at the desk" (and benchmark shopping) with empirical measurement that distinguishes "the new prompt works better" from "the new prompt works better on the five examples I happened to try." Solves the problem that LLM system behavior is stochastic (every run is a sample, not a deterministic pass/fail), the input space is open-ended (the developer's pet examples are not a representative sample of what users will throw at it), model behavior changes across releases (a feature that failed last quarter may now be viable), and traditional binary unit tests do not capture the distributional nature of LLM outputs. The eval suite is the empirical measurement instrument that makes shipping decisions defensible and changes gateable: each change has a baseline, a representative private eval set, a comparison method, and a documented decision policy. The discipline's hard part is not writing evals — it is choosing what to measure, encoding the choice into a grader the team agrees with, sampling a dataset that represents production, distinguishing a real improvement from sampling noise, tracking high-risk slices separately from the average, calibrating model-graders against humans, feeding sanitized production failures back into the next eval version, and resisting Goodhart's Law as the eval suite becomes the optimization target. Teams that get this right ship systems whose quality matches their stated definition of "good"; teams that get this wrong ship systems that ace evals and disappoint users.

Distinct from testing-strategy, which owns deterministic-software testing where every run is binary pass/fail and the main decision is the cheapest honest test level — this skill owns LLM evaluation where every run is a sample from a distribution and pass-rate (or a score distribution) is the unit of judgment; the disciplines share vocabulary (suite, gate, regression) but the math underneath differs. The exact construction of eval task sets, rubrics, graders, hard negatives, trace schemas, and thresholds is separate from this skill; this skill owns the development discipline that uses those artifacts to gate whether prompt/model/retrieval/tool/orchestration changes ship. Distinct from evaluation, which owns general scoring frameworks and result interpretation — this skill owns the repeated LLM-system change gate. Distinct from tool-call-flow, which owns the protocol cycle by which a model invokes tools — this skill owns the discipline of measuring whether that cycle produces correct behavior; tool-call and trajectory evals are a specialization of the general pattern. Distinct from prompt-injection-defense, which owns the security property and threat model — this skill owns the measurement of whether the property holds (red-team evals against an injection corpus are one application). Distinct from error-tracking and observability-modeling, which own runtime measurement of deployed systems — this skill owns offline pre-deployment measurement while using production signals as inputs for dataset refresh; one is not used as a substitute for the other. Eval-driven development is to LLM system engineering what crash-test ratings are to automotive safety — you do not ship a car based on how well it parked in your driveway; you ship it after a battery of standardized tests on representative crash scenarios, with the pass-rate against named criteria as the gating signal. A score of 4.3 stars across the suite is the only defensible claim of 'safer'; a developer's intuition that 'the new model feels smarter' is the unmeasured equivalent of 'I drove it home, it seemed fine.' And just as a crash lab that the manufacturer trains its cars to detect stops measuring real safety, an eval the model can recognize as a test stops measuring real behavior. The wrong mental model is that LLM evals are "unit tests for prompts" — binary pass/fail, written once, checked in CI, ship on green. They are not. The fundamental property is distributional and lifecycle-based: every run is a sample from a stochastic system, and the unit of judgment is pass-rate over a sampled population with a sample size and confidence interval. A test that runs once and "passes" tells you nothing about the distribution; pass-rate at n=20 has huge uncertainty; pass-rate at n=500 starts to be informative. Adjacent misconceptions: that a higher headline pass-rate is a real improvement (it is not until the delta's confidence interval excludes zero — a paired bootstrap on a shared dataset is the cheap default, McNemar's test is the right check for paired pass/fail outcomes, and halving the effect you want to detect costs four times the samples); that a single headline number is sufficient (it is not — it can hide a high-risk slice regressing while the average improves; a panel of independent measures and slices is harder to over-fit, and HELM's multi-metric framing exists precisely as a counter to single-metric Goodharting); that public benchmarks (MMLU, HumanEval, BIG-bench, GAIA, MT-Bench, τ-bench) gate system-specific shipping decisions (they do not — benchmarks predict how a model will do on the exact tasks they contain, not on your system's user inputs; use them for model-selection grounding, and treat a sudden benchmark jump as a contamination hypothesis, not an automatic capability win); that higher eval scores are always improvements (they are not — Goodhart's Law: when the eval becomes the optimization target, it ceases to be a good measure; a 100% pass-rate is a measurement failure on a capability/gating suite, not a victory, because a suite the system aces has stopped discriminating — though a regression suite near 100% is doing its job, see the carve-out in § The 100% Pass-Rate Trap); that model-graded evals are reliable without calibration (they are not — model-graders have verbosity bias, position bias in pairwise comparisons, self-preference bias toward their own family's outputs, and correlated error with the system being graded; the fix is mechanical — judge from a different top-tier family, swap positions, ensemble across families — not a "be objective" instruction in the grader prompt, because the bias lives in the embedding distribution, not the surface style); that you can trust a metric without reading the transcripts (you cannot — error analysis on the actual traces is how you learn whether the grader itself is working and what the failures really are); that final-output grading is sufficient for multi-step agents (it is not — an agent that reaches the right answer through a broken or unsafe trajectory will pass a final-output eval and fail in production; multi-step systems need trajectory-aware evals that score planning, tool selection, and execution); that the model under test is fixed (it is not — every model upgrade or vendor switch changes the system under test and must be re-gated against the private suite, including high-risk slices, cost, and latency); and that offline evals replace production telemetry (they do not — production telemetry measures the actual user-facing system under actual load, and every production incident is a candidate eval case after sanitization; the two are complementary, not substitutable). Public benchmarks, vendor dashboards, and LLM judges are useful inputs, not shipping authority by themselves.

Coverage

The practice of building language-model-integrated systems by writing evaluations before and alongside the system, then using the eval suite's measured signal to gate every meaningful change. Covers:

• The statistical (not binary) nature of LLM evaluation: pass-rate, score distribution, repeated runs, slice variance, confidence intervals, and minimum detectable change.
• The five primitives: dataset, evaluation function, aggregation, iteration loop, regression budget.
• Where eval datasets come from (production logs, user-feedback capture, hand-authoring, prior failures, adversarial generation, synthetic generation) and how origin determines representativeness.
• How a suite starts small and grows — a handful of real-failure cases is a legitimate beginning; statistical power is earned by accretion, not required on day one.
• The bootstrap-vs-certification distinction: a 20–50 case early suite can drive coarse development feedback, but it cannot certify small deltas unless its MDE supports the claim; every suite is labeled with the posture (bootstrap, regression, capability, launch gate) that says which claim it warrants.
• Repeated-trial metrics for stochastic agents: pass@k ("at least one success in k attempts") versus pass^k ("all k attempts succeed"), and why the product promise — one good candidate vs reliable-every-time behavior — decides which one gates.
• Trace-first debugging versus repeatable eval runs: traces diagnose workflow failures; versioned datasets and eval runs gate comparable changes.
• Benchmark retirement and deprecation hygiene: public benchmarks, shared evals, and private suites must be marked stale, contaminated, saturated, or retired when they stop measuring the intended capability.
• The judgment-mechanism taxonomy (programmatic / model-graded / human-graded / preference comparison / trace-hybrid) and its cost dimension.
• Error analysis — reading transcripts/traces before trusting any metric — as the diagnostic engine of the loop.
• The eval surface stack: final response, trajectory/tool-use, retrieval/context use, safety/boundary, side effects, cost/latency — and which layer must block merge.
• RAG-specific eval surfaces: context precision, context recall, faithfulness/groundedness, answer relevance, citation support.
• The discipline of distinguishing a real improvement from sampling noise: paired difference tests, McNemar's test for paired binary outcomes, bootstrap confidence intervals on the delta, minimum-detectable-effect, clustered standard errors, and high-risk slice vetoes.
• The distinction between final-output and trajectory evaluation for multi-step agents.
• Continuous evaluation and model/vendor-upgrade gates: rerunning private evals before adopting a new model, API feature, tool runtime, or judge model.
• The distinction between system-specific evals and canonical public benchmarks (MMLU, HumanEval, BIG-bench, GAIA, MT-Bench, HELM, SimpleQA, SWE-bench, τ-bench).
• Why higher scores are not always improvements (Goodhart's Law, including suite saturation at 100%), and the eval-awareness/contamination risk that erodes benchmark validity — including the structural worsening of eval-awareness with model capability.
• The pre-registration / spec-first discipline that keeps the iteration loop from degrading into post-hoc p-hacking.
• Cost-aware regression budgets: when a quality gain justifies higher latency, token spend, tool calls, or human-review load.
• Offline evals vs production telemetry: the graded offline→shadow→canary→A/B→full rollout spectrum, and how live traces, feedback, incidents, and A/B tests feed the next eval version without replacing pre-deployment gates.
• The eval-lifecycle archetypes (acceptance, regression, calibration, red-team, cross-model, drift-refresh).

Philosophy of the skill

Building LLM-integrated systems without evals is shipping airplanes based on how good the model feels at the desk. The system's behavior is stochastic, the input space is open-ended, the model may change under you, and the developer's pet examples are not a representative sample of what users will throw at it. An eval suite is the empirical measurement instrument that lets a team distinguish "the new system is better" from "the new system is better on the five examples I happened to inspect."

The discipline's hard part is not writing evals. It is choosing what to measure, encoding the choice into a grader the team agrees with, sampling a dataset that represents production, distinguishing a real improvement from sampling noise, calibrating subjective graders, holding the suite stable while the system changes, and resisting the gravitational pull of Goodhart's Law as the eval suite becomes the optimization target. Teams that get this right ship systems whose quality matches their team's stated definition of "good." Teams that get this wrong ship systems that ace evals and disappoint users.

Eval-driven development is not test-driven development with extra noise. It is empirical engineering applied to systems whose behavior is a distribution rather than a value. The vocabulary overlaps with testing — suite, gate, regression, CI — but the unit of judgment differs. A deterministic unit test asks "did this exact behavior happen?" An LLM eval asks "how often, under which slices, at what cost, with what uncertainty, and against whose definition of quality?"

Start small; do not wait for a perfect suite. The barrier to entry is low and the cost of delay is high: a starter set of 20–50 tasks drawn from real failures is a legitimate beginning, not a placeholder (Anthropic's agent-eval guidance). Early in development the effect sizes of changes are large, so even a small suite separates good from bad; the statistical machinery below matters most later, when the easy wins are gone and you are arguing over single percentage points. The failure mode is not "started with too few cases" — it is "never grew the suite, and never read the transcripts."

The Five Primitives In Practice

Three invariants keep the loop honest:

• Keep the eval suite stable while the system changes. If the dataset, grader, and system all change at once, the comparison anchor is gone and the team is doing parallel experiments.
• When the eval itself changes, version it as a new measurement instrument. Re-baseline the old and new system on the new eval if you need continuity; a delta is only comparable across runs on the same dataset version.
• Pre-register the comparison before seeing candidate results. At minimum, write the hypothesis, dataset version, primary metric, slice vetoes, judge version, sample size or repeat policy, cost/latency budget, and merge rule. This is the structural defense against post-hoc rationalization (see § Iteration Loop Discipline).

Where Eval Datasets Come From

The first primitive — the dataset — is only as good as its representativeness, and origin determines what population the result can speak for and which blind spots the suite will have. Six common origins, ordered roughly from closest-to-deployment to furthest:

A healthy suite blends origins and labels each case with provenance: production-sampled inputs anchor representativeness, hand-authored + adversarial cases pin the spec and the boundaries, prior-failure cases prevent regressions, and synthetic generation fills volume — with the mix and each stratum's weight chosen deliberately, not by whichever data was easiest to collect.

Bootstrap Suites: Start Small, But Label The Claim

The "start small" point from § Philosophy has a discipline attached: the claim a suite can support must match its sample. A starter set of 20–50 well-chosen tasks from real failures, manual smoke checks, support issues, and product requirements is enough to expose coarse regressions and make the team's definition of success concrete — it is a development steering wheel, not a certification instrument. A bootstrap suite can justify "this change fixed the obvious failure class" or "the new model is worth deeper evaluation"; it cannot justify "a 2pp lift is real" unless the design's minimum detectable effect supports that claim (see § The Statistics of "Is This Better?"). Effects worth shipping shrink as the system improves, so mature systems need larger, harder, stratified suites, repeated trials, or paired comparisons.

The operational rule is to label every suite with its use posture — this is the epistemic-warrant companion to the cadence/gating view in § Eval Lifecycle Archetypes:

What To Evaluate — The Eval Surface Stack

"Run the evals" is underspecified until you name which surface each eval scores. For simple LLM features, final-output quality may be enough; for agents it is usually too shallow. A complete LLM system has several distinct evaluable surfaces, and grading only the final response — the most common mistake — leaves the others unmeasured.

A suite that covers only the first row reports a single number that hides failures on every other surface. Decide which surfaces your system actually has, and put at least one eval on each that matters. This skill decides how those surfaces become a change gate; constructing the exact rubrics and trace schemas is a task-specific artifact concern.

RAG Eval Surfaces

Retrieval-augmented systems have a distinct, well-named set of eval dimensions because a RAG answer can fail in two independent places — the retriever fetched the wrong passages, or the generator ignored/contradicted the right ones. A green final answer can hide an unreliable retriever, and a faithful answer can still be useless if the retriever missed the necessary source. Naming the layers separately localizes the failure instead of blaming "the model" (RAGAS framing):

Report the failure location: retrieval miss, ranking/noise problem, context-packing loss, synthesis hallucination, bad refusal, or citation mismatch. The regression budget should say which layer blocks merge and which only triggers diagnosis. These are eval surfaces to name and measure, not a RAG-metric construction manual — building the retrieval corpus, the rubric, and the graders is the task-specific work of building the retrieval corpus, rubric, and graders.

Safety And Red-Team Eval Gates

Safety evals need named gates, not just a generic "safety score." Separate attack success from false-positive refusal and severity.

This skill owns making those signals part of the change gate. The threat model, attack taxonomy, and defense mechanism belong to prompt-injection-defense or the relevant safety/security skill.

Judgment Mechanism Selection

A practical default: programmatic checks for invariants, model judges for semantic scale, preference comparison for variants, and human review for calibration and high-risk decisions.

Cost is a design axis, not an afterthought. Per-grade cost multiplied by suite size multiplied by run frequency is a real budget, and it shapes which mechanism goes where. A 500-example suite graded by a model at ~$0.05/grade is ~$25 per run — runnable on every PR; the same suite under human grading at ~$5/grade is ~$2,500 per run — a quarterly-calibration activity, not a CI gate. The design rule that falls out: put the highest-frequency runs (per-commit gates) on the cheapest adequate mechanism (programmatic, then model-graded), and reserve human grading for the low-frequency calibration and ambiguous-case sampling that only humans can do. Cost also belongs in the gating decision itself — see the "cost-aware threshold" regression-budget policy and § Cost-Aware Evaluation.

Read The Transcripts — Error Analysis Is The Engine

A metric is a summary, and a summary you have not traced back to specific transcripts is a number you do not yet understand. Before trusting any aggregate — and especially before trusting a model-graded aggregate — read a sample of the actual traces: the inputs, the full output, the tool calls and intermediate state, and the grade the grader assigned. This is error analysis, and it is the diagnostic engine of the iteration loop, not an optional polish step. "You won't know if your graders are working well unless you read the transcripts and grades from many trials" (Anthropic's agent-eval guidance).

Error analysis answers questions a pass-rate cannot:

• Is the grader actually right? A model-graded eval can report 85% while silently mis-grading a recurring failure shape; only reading graded transcripts catches a broken grader before it pollutes every downstream decision.
• *What are the failures really?* Reading a dozen failing traces usually clusters into 2–3 named failure modes (wrong tool, dropped context, over-refusal, hallucinated citation) far faster than staring at the scalar score — and those named modes become the next eval cases and the next fix.
• Is a "pass" a real pass? A trace that reached the right answer through a broken or unsafe path passes a final-output grader; only the trace reveals it.

The loop is: read traces → name failure modes → encode them as cases/graders → then optimize the metric. Skipping straight to "the number went up" is how teams optimize a metric they have never grounded in reality.

The Statistics of "Is This Better?"

A pass-rate is an estimate, not a measurement. "Version B scored 3 points higher" is not a shipping argument until you have asked whether the gap survives sampling noise. This is the single most common place eval-driven development degrades into eval-flavored vibes: a team reads two numbers, declares a winner, and ships a change the data does not actually support. (The statistics matter most once the suite is past its starter phase — early, large-effect changes are visible even at n=20; single-percentage-point arguments are where the machinery below earns its keep.)

Report sample size with every pass-rate, mean score, or win rate. A bare number with no n is not interpretable.

Compare the delta, not two separate intervals. The decision rule is whether the *confidence interval on the difference (B − A) excludes zero* (or clears the predeclared regression budget) — not whether A's and B's individual intervals fail to overlap (the overlap check is needlessly conservative and throws away power).

Prefer a paired test on a shared dataset. Run both system versions on the same eval examples and analyze the per-example score differences. Because scores on the same question are positively correlated across versions, the paired estimator removes that shared variance "for free" — a paired t-test or Wilcoxon signed-rank produces a substantially tighter confidence interval on the delta than the unpaired equivalent, at the cost of roughly 2× inference (you run both arms). How much tighter is not a fixed quantity: it scales with the within-example correlation between the two arms, so the gain is large when versions agree on most examples and modest when they do not — there is no universal "orders of magnitude" multiplier to quote. When you cannot pair (different datasets), fall back to the unpaired interval and expect to need far more data.

For paired binary pass/fail outcomes, name the test. McNemar's test (or an exact binomial test on the discordant pairs) is the usual check for whether B fixes more cases than it breaks. The interesting count is not total wins; it is A fail / B pass versus A pass / B fail.

Bootstrap the delta when assumptions are shaky. For small, skewed, or non-normal datasets, compute delta_i = score_B_i − score_A_i for each shared example, resample the delta vector with replacement (e.g. 1,000–10,000 resamples), recompute the mean/pass-rate delta, and use the 2.5th and 97.5th percentiles as the approximate 95% interval. Ship only when that CI sits entirely on one side of zero. When central-limit assumptions hold and n is a few hundred or more, a closed-form paired interval (Δ ± 1.96 × SE(difference)) is sufficient and bootstrapping is optional.

Know your minimum detectable effect (MDE) before you run. Sample size scales quadratically with the inverse of the effect you want to detect: detecting a gap half as large requires 4× the samples. There is no universal "N examples per 1% gap" constant — the requirement depends on the design and the pass-rate. For an unpaired one-sample 95% CI with ±1pp half-width, the normal approximation needs roughly 1.96² · p(1−p) / 0.01² examples: ≈9,600 at p=0.5, ≈6,100 at p=0.8, ≈3,500 at p=0.9 — thousands, not hundreds, to pin a single percentage point. As a rough feel for smaller suites, a binary pass-rate at n=100 has a 95% half-width of about ±8pp near 80% pass-rate and about ±10pp near 50%; n=400 cuts those roughly in half. *These figures are the unpaired case — and they are why teams that actually ship on small eval suites run paired comparisons, which need far fewer examples for the same MDE (how many fewer depends entirely on the arm correlation, so no fixed per-1pp number is defensible for either design). The practical consequence: a small eval resolves only coarse effects — treat any delta below your design's MDE as "no signal," not "small improvement," and compute that MDE for your* design and pass-rate rather than trusting a rule of thumb. For finer MDE you need more examples, repeated runs, or token-probability scoring.

Account for correlated examples (clustered standard errors). If your dataset contains groups of related examples — multiple questions from one document, paraphrases, translations, repeated tasks from one user — the effective sample size is the number of clusters, not the number of rows. Naively treating correlated rows as independent can understate the true standard error by ~3×, turning noise into a false "significant" result. Cluster the standard error on the grouping variable, or treat the cluster count as the effective n.

Shrink within-question noise with repeats. A stochastic system gives a different score on the same input across runs. Generating K samples per question and averaging reduces the within-question variance by a factor of K; when token-probability scores are available, that within-question noise can be eliminated entirely. Repeat runs whenever the system, tools, environment, or judge is stochastic enough that one run can flip the result.

Choose the repeated-trial metric that matches the product promise (`pass@k` vs `pass^k`). For stochastic agents, repeated runs can answer two opposite questions. pass@k asks whether at least one of k attempts succeeds; it rises as k increases and is appropriate when the product can cheaply try multiple candidates and keep the first valid one. pass^k asks whether all k attempts succeed; it falls as k increases and is the right consistency metric when a user expects the agent to work every time. A change can improve pass@10 while making pass^3 worse — it found more lucky successes but became less reliable. Record which repeated-trial metric gates the change before running the eval, because the two reward opposite behaviors.

Report slices separately, and treat high-risk slices as veto gates. Break results out by task type, difficulty, user segment, language, tool path, safety class, context source, model family, and known prior failures. A +5-point average does not justify a critical privacy, safety, tool-side-effect, or grounding regression on a slice.

Reporting standard. A defensible eval result reports: mean score, standard error (CLT-based or clustered), the 95% CI on the delta, sample size n, cluster count, the slice table, the MDE or threshold, the repeat-run policy, the cost/latency delta, and — for a paired comparison — the correlation between the paired arms. A bare "B scored higher" with none of these is an unverified claim.

Boundary note: constructing the rubric, hard negatives, and acceptance thresholds is the task-specific work of constructing the rubric, hard negatives, and acceptance thresholds; interpreting whether the delta is real and gating the change on it is this skill's. The statistics live here because they are the gating decision.

Iteration Loop Discipline

The eval-driven iteration loop is the development cycle. Run the suite, diagnose failures and slices, choose one meaningful change, re-run the same suite, gate the change on the regression budget.

baseline eval
  -> diagnose failures and slices (READ THE TRANSCRIPTS, not just the score)
  -> choose one meaningful change
  -> re-run the same eval suite
  -> compare against regression budget
  -> merge, iterate, or revert

The "diagnose" step is error analysis, not metric-watching (see § Read The Transcripts). Allowed system changes include prompt wording, system/developer instructions, retrieval query strategy, context packing, tool schema, tool-call policy, model choice, reasoning effort, output format, guardrail policy, and orchestration. The suite should remain stable during the comparison. Change one thing per iteration so the eval delta is attributable; if the budget allows multiple changes, gate them as a batch but expect harder root-causing when the batch regresses.

Pre-register the decision rule before you run (spec-first, anti-p-hacking). The structural defense against fooling yourself is to write down what you will measure, the threshold, and the ship/no-ship rule before you see the result — not after. Before a candidate run, write the eval spec as if it were a test plan: baseline system, candidate change, dataset version, primary metric, secondary metrics, slice vetoes, judge/grader version, sample size or repeat count, MDE, cost/latency ceiling, and merge rule. After the run, record whether the predeclared rule passed. An iteration loop that decides the metric and cutoff after looking at the numbers is p-hacking with extra steps: with enough sliced views (this subset, that stratum, this rephrased grader) some comparison will always cross significance by chance, and a team that picks the favorable slice post-hoc ships noise. Treat any post-hoc slice that "looks better" as a hypothesis for the next pre-registered run, not as evidence for this one — that is a new experiment, not the same comparison.

*Iterating on the gate set silently over-fits it. A documented 2026 failure mode: when prompt edits are tuned against the same eval set that decides whether they ship, "better" eval scores can correspond to worse* production behavior — the prompt has learned the eval, not the task ("When Better Prompts Hurt," 2026). The defense is structural: keep a held-out slice the iteration never sees (see § Goodhart defenses), and connect offline scores to online outcomes so an offline gain that does not show up in production is caught.

When you must change both the system and the eval:

1. Freeze the old system and old eval result as the baseline.
2. Add or revise eval cases with a reason: new production failure, drifted product requirement, missing slice, judge bug, or stale benchmark assumption.
3. Run old and new system on the new eval if you need a fair comparison.
4. Record that future deltas are against the new eval version, not the old one.

Eval Lifecycle Archetypes — Different Jobs, Not One Number

"The eval suite" is usually several suites doing different jobs, and conflating them into a single headline number is a recurring failure: a regression suite and a red-team suite answer opposite questions, and averaging them tells you nothing. Name the archetype each eval belongs to and run it on its own cadence:

The point is not the exact list — it is that a single fragile number conflates jobs that need different datasets, cadences, and gating rules. Splitting the suite by archetype is how you stop a calibration drift from silently masquerading as a quality regression, and how you avoid reading a regression suite's healthy near-100% as a capability suite's saturation failure.

Calibrating Model-Graded Evals (LLM-as-Judge)

Model-graded evals are the workhorse of open-output evaluation, but the judge is itself a stochastic, biased instrument — another model with its own error distribution. Treat it as a measurement instrument that needs calibration, not as neutral authority. Naming the biases and fixing them mechanically is operational hygiene, not optional polish. (And calibration starts with reading graded transcripts — see § Read The Transcripts — because a bias you have not seen in a trace is a bias you cannot size.)

Known biases of an LLM judge. The effect-size column gives illustrative ranges — they vary widely by judge model, task, and study, so treat them as "big enough to flip a close call," not as fixed constants; measure the effect on your own setup before trusting any number here. (2026 systematic studies report frontier judges failing the majority of standardized bias probes, so assume the biases are present until your own calibration data says otherwise.)

Treat the bias table as a checklist, not a universal ranking. There is no fixed hierarchy of which bias dominates: recent LLM-as-judge debiasing work found style bias dominating position bias in its experimental setup, while other work emphasizes position, verbosity, and self-preference — and debiasing strategies are themselves model-dependent (a mitigation that helps one judge can be neutral or harmful for another). The safe operational move is local measurement: build calibration pairs that hold substance fixed while varying style, length, format, model family, and answer order, then estimate which bias is large enough to flip your gate. Record the calibration receipt rather than importing a generic bias hierarchy.

The fixes are mechanical, not prompt-level. Adding "evaluate objectively, ignore writing style" to the grader prompt does not remove these biases, because they live in the model's embedding distribution, not in surface style. What works:

• Judge from a different family than the candidate — but only if the cross-family judge is itself a top-tier model. Diversifying families breaks self-preference bias only when the alternate judge is at least as capable as the candidate; grading quality with a weaker model to "diversify families" trades one bias for a worse one (an unreliable judge) and is a false economy. If you cannot afford a top-tier judge from a different family, a same-family top-tier judge with position-swapping and human calibration is a legitimate fallback — not ideal, but valid when the mechanical mitigations are applied, and strictly better than dropping to a weaker cross-family judge. Never let a model grade its own family's output as the sole, un-mitigated judge.
• Give the judge a defined escape hatch. Let the grader return Unknown / cannot determine when the transcript does not contain enough information to grade the criterion, rather than forcing a pass/fail guess (Anthropic's agent-eval guidance). A forced binary on an under-specified case manufactures noise; an explicit abstain bucket both reduces that noise and surfaces under-specified cases for rubric repair. (This is a grader-contract fix, distinct from "be objective" prose — it changes the output space, not the instruction tone.)
• Use reference-guided grading where a correct reference exists. Anchoring the judge to a known-good answer is more reliable than open-ended absolute scoring (2026 mitigation literature).
• Swap positions (run each pairwise comparison both ways, A-then-B and B-then-A) and average, to cancel position bias. Inconsistent verdicts across the swap are themselves a signal of low-confidence judgments.
• Ensemble across families for high-stakes decisions — but prefer a meta-judge to a debate. For a launch gate, run a small multi-family judge panel and aggregate by majority or weighted vote; family-specific biases tend to cancel, at roughly N× the cost of a single judge. Note a 2026 caution: debate-style multi-model setups can amplify bias rather than cancel it, so aggregate independent verdicts (meta-judge / vote) rather than letting judges argue toward consensus.
• Detect self-preference empirically. Generate the same answer with two different families on a calibration set, have each judge score both; if a judge scores its own family's text consistently higher on equivalent content, you have measured the bias and can correct or down-weight that judge.
• Calibrate against humans on a sample every cadence (quarterly is a common default). The operational loop has named steps: (1) draw a stratified sample of graded examples (cover pass/fail and each criticality band, not just a random slice that under-represents rare-but-important cases); (2) have humans grade that sample blind to the model's score; (3) measure agreement with an inter-rater statistic — Cohen's κ or percent-agreement, not eyeballing — and set a re-anchor threshold below which the judge is not trusted; (4) re-run the calibration whenever the judge model version changes, because a silent provider-side update can shift scoring even with a byte-identical prompt. A judge that has not been calibrated since its model last changed is reporting an unverified number.

Estimate judge-bias magnitude on the local calibration slice in score points or win-rate points. Do not import universal bias ranges as if they were constants; the size depends on task, rubric, judge model, candidate model, answer length, and how close the variants are.

Boundary note: this section is the measurement-hygiene application of calibration inside the iteration loop. The design of the grader rubric and its anchors belongs with the task-specific work of constructing the grader rubric and anchors; the general result-interpretation framework is evaluation.

Evaluating Multi-Step Agents — Trajectory, Not Just Final Output

The five primitives describe the general pattern, but for a multi-step agent (one that plans, calls tools, reads results, and iterates) the unit being graded shifts. Grading only the final answer is insufficient and actively dangerous: an agent can reach a correct final answer through a broken, wasteful, or unsafe path — calling the wrong tool, fabricating an intermediate result, taking a destructive action that happened not to matter this time — and pass a final-output-only eval while being unfit for production.

Two complementary metric families:

A complete agent eval scores both, because a high outcome score with a poor trajectory score is a strong predictor of brittle production behavior (the agent succeeded by luck or memorization, not by sound process). The grading unit here is the transcript (also called a trace or trajectory): the complete record of a trial — outputs, tool calls, reasoning, intermediate results, and interactions — and grading it well requires reading a sample of those transcripts (see § Read The Transcripts), not only scoring the end state.

*Reference benchmarks for the shape of agent evals* (cited for grounding, never for gating your system): τ-bench and τ²-bench (multi-turn tool-use agents in policy-constrained environments with shared world-state), BFCL (function/tool-calling correctness across languages and modes), GAIA and Terminal-Bench (end-to-end multi-step assistant tasks), and the trajectory-aware benchmarks that explicitly score the sequence of tool calls rather than only the final state. Use these to understand what trajectory grading looks like; build your own trajectory eval against your agent's actual tools and policies.

Trace-First Debugging, Dataset-Backed Gates

§ Read The Transcripts establishes why you read traces (validate the grader, find the real failures); this subsection is the agent-specific workflow that turns that reading into a gate. For agents, the fastest path from "something is wrong" to "we know what to measure" is often trace grading: a trace captures the model calls, tool calls, handoffs, guardrails, intermediate state, and final output for one run. Grading traces is excellent for diagnosing whether the agent chose the wrong tool, skipped a handoff, violated a policy, looped wastefully, or reached the right answer through a fragile path (OpenAI's agent-evals guidance: start with trace grading while debugging, then move to datasets/eval runs when you need repeatability).

But a graded trace is not automatically a release gate. A single trace is evidence for a failure mode; a versioned dataset of traces or tasks is the comparable measurement instrument. The promotion loop:

inspect representative trace
  -> identify failure mode
  -> turn it into a task / trace expectation
  -> add it to a versioned dataset
  -> rerun baseline and candidate
  -> gate on the predeclared regression budget

Use trace tooling to find and localize failures; use datasets and eval runs to compare system versions. This keeps trace inspection from becoming another form of anecdotal "vibe eval."

Public Benchmarks — Cited For Grounding, Not For Gating

Benchmarks measure cross-system capability against a shared standard. They predict how a model will do on the exact tasks the benchmark contains. They do not prove your specific system — with your prompt, tools, retrieval, latency budget, and user population — is good enough to ship.

The right use: pick a model partly on benchmark performance (and to notice upstream movement), then build system-specific evals to gate the actual deployment. Note that several headline benchmarks are now saturated at the frontier