Humanity's Last Exam, or HLE, is a benchmark of 2,500 very hard expert-written questions, built because top AI models had maxed out older tests.
In plain words
When models score over 90% on a test, it stops telling you which one is better. HLE collects questions from subject experts, mostly professors and researchers, across maths, the humanities and the sciences. Answers are multiple choice or short, so a computer can grade them. Some questions are kept private to catch models trained on the test.
A quick example
A lab releases a new model and reports its HLE score next to older benchmarks. Because leading models still score low on HLE, a jump there says more about progress at the frontier than another point on a test that everyone already aces.
Where you'll meet it
- Comparing frontier AI models.
- Tracking how close models are to expert-level knowledge.
Related
Sources
- PaperHumanity's Last Exam, Phan et al. (arXiv) · read 29 Sept 2026
- DocsHumanity's Last Exam, Center for AI Safety and Scale AI · read 29 Sept 2026
Checked by WhatIsAI against the sources above · 29 Sept 2026. Spotted an error? Report it.