Why LLMs Hallucinate

Fluent Is Not the Same as True

A hallucination is a confident answer that is wrong or made up: a fake policy, a non-existent API function, a wrong order status. It happens because the model is trained to produce likely-sounding text, not to check facts. When it does not know, the most likely continuation is still a fluent sentence.

The fact was never in the training data (your private refund policy, today's order status).
The fact is newer than the training cut-off.
The question assumes something false ("Why does ShopEase give 90-day returns?").
The prompt is vague, so the model fills gaps with guesses.

Why It Happens, Mechanically

Remember the loop: at every step the model must output a probability distribution and one token is chosen. There is no built-in "I am not sure" token that is forced when knowledge is missing. If the question is "When was ShopEase founded?", the tokens "ShopEase was founded in" are very likely, and then some year must come next. The model picks a plausible year, and the sentence reads perfectly.

Training objective: pre-training rewards plausible continuations, not verified truth.
Compressed memory: rare facts are stored weakly and can blend with similar facts (two companies, two dates).
Incentive to answer: training and many benchmarks reward giving an answer over admitting uncertainty, so guessing is learned behaviour unless training pushes against it.
Snowballing: once a wrong token is generated, later tokens build on it to stay consistent.

An analogy: a student in a viva who does not know an answer but speaks confidently anyway, because silence scores zero. Modern models are trained to say "I am not sure" more often, but the risk never fully disappears.

A Model That Always Answers

Example

This stand-in "model" learned sentence patterns from a few true facts. Asked about a store it never saw, it still produces a fluent sentence by mixing patterns: a hallucination.

facts = {
    "flipkart": "Flipkart was founded in 2007 in Bengaluru .",
    "zomato":   "Zomato was founded in 2008 in Gurugram .",
    "swiggy":   "Swiggy was founded in 2014 in Bengaluru .",
}

def fake_model(question):
    name = question.split()[2]     # "When was <name> founded?"
    if name.lower() in facts:
        return facts[name.lower()]
    # never seen it: copy the pattern of the "most similar" known answer
    pattern = facts["swiggy"]
    return pattern.replace("Swiggy", name)

for q in ["When was Zomato founded?", "When was ShopEase founded?"]:
    print(q, "->", fake_model(q))

Output

When was Zomato founded? -> Zomato was founded in 2008 in Gurugram .
When was ShopEase founded? -> ShopEase was founded in 2014 in Bengaluru .

The second answer sounds just as confident as the first, but it is invented. The fixes you will learn: give the model the real facts in the prompt (RAG), tell it to say "I don't know" when the context has no answer, ask for citations, and check outputs with code.

Types, Limits and Misconceptions

Type	Example
Factual error	Wrong founding year, wrong capital city
Fabrication	A made-up research paper, URL or library function
Unfaithful to context	The policy you gave says 7 days, the answer says 10
False premise accepted	Explains a 90-day return policy that does not exist

Note: Misconceptions. "RAG removes hallucination": it reduces it a lot, but the model can still misread or ignore the context, so you still check. "A confident tone means it is right": tone carries no information about accuracy. "Bigger models do not hallucinate": they do it less on common facts, not never.

Summary: hallucination = fluent but false output, caused by the next-token objective and imperfect memory.
Risk is highest for rare, private, recent or false-premise questions.
Reduce it with context (RAG), permission to say "I don't know", citations, and code checks.