TruthfulQA is a benchmark of 817 questions designed to catch AI models repeating common myths and misconceptions instead of true answers.
In plain words
Models learn from human writing, which includes plenty of popular falsehoods. TruthfulQA asks questions where many people would give a wrong answer out of a false belief, across topics like health, law and money. A model scores well only if it avoids those learned mistakes. In the original paper, people were truthful 94% of the time and the best model 58%.
A quick example
One question is about people who often crack their knuckles. A model copying popular belief might say you will get arthritis. TruthfulQA rewards an answer that sticks to what is actually known.
Where you'll meet it
- Measuring how often models repeat misconceptions.
- Safety and honesty evaluations of chat models.
Related
Sources
- PaperTruthfulQA: Measuring How Models Mimic Human Falsehoods, Lin, Hilton and Evans (arXiv) · read 29 Sept 2026
- DocsTruthfulQA, Lin, Hilton and Evans (GitHub) · read 29 Sept 2026
- DocsTruthfulQA.csv, Lin, Hilton and Evans (GitHub) · read 29 Sept 2026
Checked by WhatIsAI against the sources above · 29 Sept 2026. Spotted an error? Report it.