MATH is a benchmark of 12,500 competition maths problems, each with a full worked solution, used to test harder mathematical reasoning in AI models.
In plain words
MATH is built from hard problems taken from maths competitions. Every problem comes with a step-by-step solution that models can learn from. When it was released, even very large models scored low, and the authors argued that simply making models bigger would not be enough.
A quick example
A problem in MATH's style might ask for the number of ways to arrange letters in a word under some rule. The model has to work it out and give the exact final answer, which is then compared with the official one.
Where you'll meet it
- Reporting maths ability in model launches.
- Training reasoning models with checkable answers.
Related
Sources
- PaperMeasuring Mathematical Problem Solving With the MATH Dataset, Hendrycks et al. (arXiv) · read 29 Sept 2026
- PaperMeasuring Mathematical Problem Solving With the MATH Dataset, NeurIPS 2021 Datasets and Benchmarks · read 29 Sept 2026
Checked by WhatIsAI against the sources above · 29 Sept 2026. Spotted an error? Report it.