"The tests pass." "It scores 92% on our evals." "The model tops the benchmark." Each of these is offered as a reason to trust an AI agent, and they are often heard as the same reason. They are three different ones. A test, an eval and a benchmark examine different things, give different kinds of result and support different decisions, and much of what goes wrong in judging an agent comes from taking one for another.
This post sets them side by side. The example throughout is an agent that handles refund requests: it reads a request, looks up the order and the terms of sale, and refunds, refuses, or hands the request to a person.
Three questions
| Asks | Decides | |
|---|---|---|
| Test | Does the code do what we wrote it to do? | Whether a change to the code is merged |
| Eval | Does the agent do our work, and how often? | Whether a version is released |
| Benchmark | How does this model compare with others? | Which models to try first |
They also differ in what they examine and in the kind of answer they give. A test examines the code around the model, and its result is pass or fail. An eval examines your agent on your own tasks, and its result is a rate, held against a threshold. A benchmark examines a model on tasks that someone else chose and published, and its result is a score on a leaderboard.
The words are used less strictly than the table suggests. Anthropic's guide to agent evals defines an eval as "a test for an AI system", a benchmark is itself a kind of eval, and test suites turn up inside evals as graders. So the labels will go on overlapping. The three questions do not, and for any number you are shown, the useful thing to know is which of them it answers.
A test checks code
In software, a test has a settled meaning. Software Engineering at Google describes the simplest one as a single behaviour, a specific input, an observable output and a controlled environment. The same input gives the same output every time, so the result is pass or fail, and a failure is a defect: someone fixes the code, or the test, until it passes.
Testing treats variation as a fault. A test that passes on one run and fails on the next, with the code unchanged, is called flaky, and the same book reports from experience that "as you approach 1% flakiness, the tests begin to lose value".
Most of an agent is code of this kind. In the example: the function that works out the amount from the order, the parser that reads the model's structured output, the tool that issues the payment, and the rule that an amount above a limit goes to a person. All of it is tested like any other software, on every commit. That includes the controls that keep an agent within its rules. Our post on legal expertise argues that a rule that matters has to be enforced outside the model. What is outside the model is code, and code can be tested.
What a test cannot tell you is whether the agent's judgment is good: whether it understood the request, applied the right clause of the terms and gave the customer a correct answer. Those come from a language model, and a model does not give the same output for the same input. Atil et al. configured five models for deterministic output, and their accuracy still varied by up to 15% between runs of the same tasks. A test that asserts on what a model says in one run is either flaky, or loose enough to pass answers that are wrong.
An eval measures your agent on your tasks
An eval takes the measurement that a test cannot. What it examines is the whole system (the model, the prompt, the tools and the code around them), and its tasks are your own: real refund requests, each with a defined way to decide whether the outcome is acceptable.
Three things set it apart from a test.
It repeats. In the words of Anthropic's guide: "Because model outputs vary between runs, we run multiple trials to produce more consistent results." Each case is run several times, where a test is run once.
Its result is a rate. The answer is how often, and that can be far from the answer to whether. In tau-bench, an agent that solved more than 60% of customer service tasks in a single attempt solved fewer than 25% of them eight times in a row. Whether a given rate is good enough is a decision, made in advance as a threshold. A failing test means a defect. An eval result below its threshold means that a release waits, and that someone reads the failures.
Its grader may need judgment. Code grades whatever a rule can decide, such as the amount refunded. Whether the explanation given to the customer is accurate takes a language model as a judge, checked against a person who knows the work.
An eval suite that runs on every change and blocks a release looks much like a test suite, and the same guide says that its regression cases "should have a nearly 100% pass rate". That is where the two are most often confused. What differs is the thing measured: behaviour, sampled over cases and runs, with a margin of error. Our post on evals covers how such a suite is built.
A benchmark compares models on someone else's tasks
A benchmark is an eval that many parties share. A draft guideline from NIST describes it as a combination of a dataset and a metric, "used by a community of researchers as a shared framework for the comparison of methods". Because its tasks and its scoring are fixed and public, results can be compared across models and over time. Nothing else offers that: when a provider releases a new model, benchmark results are the first evidence of whether it is worth trying.
The same properties limit what a score says about your agent.
The tasks are not yours. tau-bench has an agent serve simulated customers of a shop and an airline, under policies written for the benchmark. Your refund requests come under your own terms of sale. Kapoor et al. call the choice of a model for an application "downstream evaluation", and warn that "benchmarks meant for model evaluation can be misleading when used for downstream evaluation". Whether a benchmark measures what it is said to measure is also less settled than a leaderboard suggests: a review of 445 LLM benchmarks by 29 experts found patterns in what they measure, in their tasks and in their scoring "which undermine the validity of the resulting claims".
Public tasks leak, and graders have faults. In February 2026 OpenAI stopped reporting its models' scores on SWE-bench Verified, a coding benchmark that it had published itself in 2024. Every frontier model it tested had seen at least some of the benchmark's problems and solutions in training. And in an audit of the problems that models often failed, "at least 59.4% of the audited problems have flawed test cases that reject functionally correct submissions". Its conclusion was that "improvements on SWE-bench Verified no longer reflect meaningful improvements in models' real-world software development abilities".
The score depends on the setup. A benchmark runs a model inside a harness, on somebody's infrastructure. Anthropic measured the effect of one such setting, the computing resources that each task is given: between the most and the least generous setups, the same model's score on Terminal-Bench 2.0 moved by 6 percentage points. It concluded that "leaderboard differences below 3 percentage points deserve skepticism until the eval configuration is documented and matched".
None of this makes benchmarks useless. It makes them a way to choose which models to try, and no more than that.
When one is taken for another
A benchmark taken for an eval. "The model leads the leaderboard, so the agent will work." The model is one part of the agent, and the leaderboard has none of your tasks on it.
Tests taken for an eval. "All tests pass" says that the code is right. It says nothing about how often the agent decides a request correctly. Even where the work itself is code, passing tests is a narrower thing than acceptable work. On SWE-bench Verified, a solution counts when the project's tests pass. METR had four maintainers of three of those projects review 296 pull requests written by agents, and roughly half of the ones that passed the tests would not have been merged. METR adds that the agents had no chance to respond to a review, as a human developer would.
An eval taken for a test. This one gets less attention. Suppose that no refund above the limit may be paid without a person's approval, and that the eval suite shows the agent handing such requests over in 97% of runs. The instrument is the wrong one. A rule that must hold every time is not measured as a rate: it is enforced in code, as a control, and the control is tested. Evals are for what is left to the model's judgment.
A number without its tasks. "Our agent is 95% accurate." On which tasks, graded how, and run how many times? Without those, the number is neither an eval that you can check nor a benchmark that you can compare.
How the three work together
In a build, each has its place.
- Benchmarks, to shortlist. Two or three candidate models, chosen with their price, their speed and their terms for data processing beside their scores.
- Tests, throughout. Every tool, parser, permission and control, on every commit.
- Evals, to decide. The candidates are compared on your own cases, inside your agent. After that, every change to a prompt, a model or a tool runs the suite, and a result below the threshold blocks the release.
The same order holds when a provider releases a new model. The benchmarks say whether it is worth trying. The eval suite says whether to move to it. The tests say that the code around it still does what it did.
Illustration of what a benchmark, tests and evals each say when a new model is released, for an agent that handles refund requests. On a public benchmark of shared tasks the new model takes first place, 2.8 points ahead of the current one. The agent's 212 tests are run, each once, and all pass, as before. The eval suite then runs ten of the company's own cases eight times each on the new model. Six of the eighty runs fail, which is 92.5%, under the 95% that a release requires and under the current model's 96.3%. The three are then set side by side. The benchmark says that the model is worth trying, the tests that the code holds, and the evals that the move waits.
Questions to ask
Whether you build an agent yourselves or buy one, these questions show which of the three a claim rests on.
- When someone says that the agent was tested, was code checked or behaviour measured?
- Whose tasks does this number come from, and can you see them?
- How many times was each task run, and how often did every run pass?
- Which rules are enforced in code and tested, and which are only measured?
- For a benchmark score: who ran it, in what setup, and is the benchmark still maintained?
- Which of the three decides whether a change is released?
A benchmark can tell you where to start, and tests that the parts work. Whether an agent does your work is something only evals on your own tasks can show. Every agent we build ships with its own, made from your real tasks.