Inspect is an open-source Python framework, made by the UK AI Security Institute, for building and running evaluations of large language models.
In plain words
In Inspect, every evaluation is a task made of three parts. A dataset holds the test questions and the ideal answers. A solver produces the model's answer, from a single call up to a multi-turn agent with tools. A scorer then grades that answer, by comparing text or by asking a model to judge. It can test coding, reasoning, knowledge, agent tasks and more, and it ships with a large set of ready-made evaluations.
A quick example
Imagine a capture-the-flag test: the model gets a shell inside a sandbox and must find a hidden flag. The solver is an agent loop that lets the model run commands, and the scorer simply checks whether the flag appears in the model's final answer.
Where you'll meet it
- Safety and capability testing of AI models.
- Running ready-made benchmarks against any model.
- Testing agents that use tools inside a sandbox.
Related
Sources
- DocsUKGovernmentBEIS/inspect_ai, UK AI Security Institute (GitHub) · read 29 Sept 2026
- DocsInspect, UK AI Security Institute · read 29 Sept 2026
Checked by WhatIsAI against the sources above · 29 Sept 2026. Spotted an error? Report it.