What are hallucinations in LLMs and why do they occur?
LLM & AILevel: FoundationHallucinations
What are hallucinations in LLMs and why do they occur?
Hints
- Clarify the threat: an unsupported fact, an untrusted instruction or an unsafe action?
- Separate input checks, authority limits and output validation; consider false positives.
Answer
Hallucinations are confident, fluent, and factually incorrect outputs — the model generates text that sounds authoritative but is wrong or fabricated.
Types:
- Factual hallucination: incorrect facts ("The Eiffel Tower was built in 1892" — it was 1889)
- Source hallucination: fabricated citations, fake paper titles, non-existent URLs
- Entity hallucination: wrong names, dates, relationships between entities
- Reasoning hallucination: correct premises but invalid logical conclusions
Root causes:
- Training data contains conflicting or incorrect information
- Knowledge cutoff — model extrapolates about post-training events
- Models are trained to generate fluent text; they don't have an internal "I don't know" mechanism by default
- Sycophancy — models agree with users who state incorrect facts
- High temperature increases creativity and hallucination simultaneously
Source and licence: JustInCache & contributors — Awesome AI Interviews · Apache-2.0