Evaluations (evals)

1 · What it is

---

An eval, short for evaluation, checks an AI system’s work. You feed it an input, score the reply with a set of rules, and note whether it passed. Google compares its eval checklists to unit tests. A unit test is a small automatic check that programmers write to make sure their code still works.

Why guessing is not enough

A model is the AI program that writes the answers. OpenAI points out that without evals, it is slow and hard to learn what a new version does to your own app. Anthropic adds that teams without evals cannot tell real regressions from noise. A regression is something that used to work and now fails.

The three parts of an eval

Every eval needs a clear idea of success, a set of test cases, and a grader.

Success. Anthropic’s first tip is to spell out exactly what a good result looks like.

Test cases. Anthropic says your test cases should look like the mix of jobs the system will really face. It also reminds you not to forget edge cases, the odd inputs that rarely show up. Its examples include irrelevant or missing input and very long input.

A grader. This is the part that decides whether each output passes. In OpenAI’s evals, the graders are what decide whether an answer counts as right. There are several kinds:

• Exact match checks whether the output is the same as a correct answer you wrote down ahead of time.
• A rubric is the list of criteria for rating a response, such as “stays on topic” or “is polite”.
• A person can judge the answer. Anthropic rates people as the most adaptable, highest-quality graders, though they are slow and costly.
• Another model can act as a judge. Anthropic says this is quick, adaptable, easy to scale and good for tricky calls, but you should check that it judges well before you lean on it for lots of tests.

Anthropic’s advice is to write questions so a computer can mark them. Examples are matching exact text, multiple choice, marking by a small program, or marking by a model. Its view: lots of questions marked by a machine, even a bit less exactly, beat a few marked by hand.

A worked example

Imagine a school helpdesk bot that sorts student messages into four moods: negative, positive, mixed or neutral. You write 200 real-looking messages and label each one, including sarcastic ones like “Great, another surprise quiz.” Exact match grading gives the bot a score.

Now you rewrite its instructions (the prompt), or switch to a newer model. You run the same 200 messages again. If the sarcasm cases improve but the mixed messages now fail, you have found a regression before any student sees it. Say v1 scored 181 of 200 and v2 scores 188: better overall, yet check what changed.

Add odd messages too: a blank one, a pasted lunch menu, a page-long rant. Score the share handled with no errors, say 48 of 50, or 96%.

Running evals again and again

OpenAI describes a loop: run the eval, study what went wrong, change your prompt, and run it again.

Anthropic separates two kinds of evals for this loop. Capability evals ask what a system can do well. They aim at tasks the system still finds hard, so they start with a low pass rate. That gives the team something to climb. Regression evals check that old skills have not broken, so they should pass almost every time. Once a capability eval passes most of the time, it can graduate and join the set of regression evals.

Where evals fall short

In one study, strong model judges agreed with people’s choices over 80% of the time. That is about as often as people agree with each other. But the same study found the judges had built-in leanings. A judge can favour the first answer, the longer one, or its own. Anthropic separately advises testing a model judge for reliability before scaling it up.