Menu
Concepts

Hallucination

3 min readbeginnerUpdated 28 Sept 2026
1 · In one line

A hallucination is generated content that sounds plausible but is false, unsupported by evidence, or inconsistent with the given input.

1 · What it is

Hallucination is a common name for what NIST’s Generative AI Profile calls confabulation. NIST’s definition includes false content, divergence from the input and contradiction within the same context. Smooth grammar is no defence: models that write very fluent replies are still known to make up facts. A polished paragraph can therefore contain one supported sentence beside an unsupported one.

NIST treats confabulation as a side effect of how generative models are built. A language model writes by guessing each next token, one small piece at a time. The same guessing that yields true sentences can also yield false or self-contradicting ones. TruthfulQA showed that generated false answers can imitate popular human misconceptions.

FActScore evaluates long text by splitting it into atomic facts and checking each against a reliable knowledge source. Search grounding can add real-time information to the answer path. Sampling several answers offers a different warning signal when their factual statements diverge. OpenAI recommends matching safeguards such as human review or grounding to the use case, especially when stakes are high.

2 · Why it exists

Fluent language can hide errors, so confidence and polish are not proof.

False detailsA model can confidently present erroneous content as an answer.
Broken groundingAn answer can diverge from the prompt, supplied evidence or an earlier statement in the same conversation.
Invented supportGenerated citations or reasoning steps can appear to justify an answer even when the answer is wrong.
3 · How it works

Follow one unsupported citation from prompt to decision.

An unsupported citation is caught by an evidence check A question and an empty source packet feed a next-token generator. It writes a plausible citation. A highlighted evidence check finds no supporting record and routes the claim to an unsupported warning. INPUTS GENERATION DRAFT DECISION QUESTION “Give me the paper title and DOI.” SOURCE PACKET No matching publisher record Evidence available: none NEXT-TOKEN GENERATOR Predict a plausible continuation title → author → journal → DOI PLAUSIBLE CITATION Fluent title Plausible authors DOI-shaped string Looks right ≠ supported EVIDENCE CHECK Split into claims Find a reliable record Compare each field Result: no support OUTPUT UNSUPPORTED Flag, remove or review The hallucination is the unsupported claim, not its fluent style.
Plausible text becomes a hallucination when its claims lack support or conflict with the available evidence.
  1. 1 · promptThe system receives a request and any evidence supplied with it.
  2. 2 · predictThe model writes its reply one token at a time, each one a guess at what comes next.
  3. 3 · draftThose predictions can form a fluent claim even when the claim is factually inaccurate.
  4. 4 · verifyBreak the draft into atomic claims and check whether a reliable source supports each one.
  5. 5 · handleSend unsupported claims for human review rather than presenting them as fact.

A confident tone measures presentation, not evidence.

4 · Where it's used
WhoWhat they askWhat it works with
Research assistant“Give me the DOI for this paper”The publisher record and bibliographic index
Support agent“Does our policy promise a full refund?”The current policy text and revision date
Developer“Which method does this package expose?”Versioned API documentation and source code
Clinician“Does this report state that diagnosis?”The supplied report and reviewed medical evidence
5 · What it solves, and what it doesn't
solves
  • The term names confidently presented false content and answers that conflict with their input.
  • Claim-by-claim evaluation can measure how much generated text is supported by a reliable source.
  • Letting a model look things up in live search results can cut down how often it makes things up.
  • Comparing multiple sampled answers can help detect facts that diverge or contradict one another.
doesn't solve
  • Confident wording can still lead readers to believe a false answer and act on it.
  • Generated answers can repeat popular human misconceptions.
  • Grounding is described as reducing hallucinations, rather than guaranteeing their elimination.
  • A citation is not proof by itself because generative systems can confabulate citations too.
6 · Go deeper
GroundingRAGKnowledge cutoffLLMFine-tuning

Sources used

This explainer is written in original language. The links below support its factual claims.

  1. officialArtificial Intelligence Risk Management Framework: Generative Artificial Intelligence Profile, National Institute of Standards and Technology · read 27 Sept 2026
  2. officialGPT-4, OpenAI · read 27 Sept 2026
  3. paperTruthfulQA: Measuring How Models Mimic Human Falsehoods, Lin, Hilton and Evans · read 27 Sept 2026
  4. paperSelfCheckGPT: Zero-Resource Black-Box Hallucination Detection for Generative Large Language Models, Manakul, Liusie and Gales · read 27 Sept 2026
  5. paperFActScore: Fine-grained Atomic Evaluation of Factual Precision in Long Form Text Generation, Min et al. · read 27 Sept 2026
  6. docsGrounding with Google Search, Google AI for Developers · read 27 Sept 2026
Suggest a correction