Hallucination Reduction
What a Hallucination Is
A hallucination is output that sounds fluent and confident but is false or not supported by the given sources: a wrong return window, an invented order status, a made-up citation. It is not a rare bug. It follows from how LLMs work: a model is trained to produce the most plausible next token, and "plausible" is not the same as "true". When the model lacks the fact, the most plausible-looking continuation is often a confident guess, because training text rarely says "I don't know".
It helps to separate two kinds. An intrinsic (or unfaithful) hallucination contradicts the context you gave: the docs say 7 days, the answer says 10. An extrinsic one adds facts that are in no source at all. RAG mainly fights the second kind; post-generation checks catch both. Think of a student who did not revise a chapter: under exam pressure, they write something that looks like an answer rather than leave the page blank. Your job is to give them the textbook (grounding), permission to say "not covered" (abstention), and a checker who marks against the book (verification).
What Helps
| Technique | Why it works |
|---|---|
| Ground answers in retrieved docs (RAG) | the model copies facts instead of recalling them |
| Allow "I don't know" | say in the prompt: if the docs do not contain the answer, say so and offer a human |
| Ask for citations | every claim must point to a doc ID, which you can check |
| Use tools for live data and maths | order status, prices and dates come from code, not memory |
| Structured outputs | fewer places for invented text |
| Verify after generation | check citations, numbers and IDs before showing the answer |
Example
Example
A citation checker: every number in a sentence must appear in the document that sentence cites. The second sentence below is a hallucination.
import re
DOCS = {"1": "Refunds reach the original payment method within 5-7 working days.",
"2": "Cash on delivery is available for orders up to Rs 50000."}
answer = ("Refunds reach your original payment method in 5-7 working days [1]. "
"COD is available for orders up to Rs 100000 [2]. Thanks for shopping with us!")
for sentence in re.split(r"(?<=[.!?])\s+", answer):
cites = re.findall(r"\[(\d+)\]", sentence)
numbers = re.findall(r"\d+(?:-\d+)?", re.sub(r"\[\d+\]", "", sentence))
if not cites:
status = "no claim to check" if not numbers else "UNSUPPORTED (numbers without citation)"
else:
source = " ".join(DOCS.get(c, "") for c in cites)
missing = [n for n in numbers if n not in source]
status = "ok" if not missing else f"HALLUCINATION? {missing} not in doc {cites}"
print(f"{status:40} | {sentence}")Output
ok | Refunds reach your original payment method in 5-7 working days [1]. HALLUCINATION? ['100000'] not in doc ['2'] | COD is available for orders up to Rs 100000 [2]. no claim to check | Thanks for shopping with us!
The first sentence's "5-7" appears in doc 1, so it passes. The second sentence cites doc 2 but says Rs 100000 while the doc says 50000: a classic intrinsic hallucination, caught by plain code. The last sentence makes no factual claim. Numbers, IDs and dates are the easiest claims to verify automatically and also the most costly to get wrong.
Tip
Tip: When a check fails, do not show the answer. Retry once with the error ("the number 100000 is not in doc 2"), or fall back to "Let me connect you to a support agent".
Note: Misconceptions: RAG does not eliminate hallucinations; the model can still misread or blend retrieved text, so you still verify. A bigger or newer model hallucinates less but not never. And a model saying it is confident is not evidence: confidence in the wording is not calibrated to truth. Measure faithfulness with evals (see the Evaluation module).
- Hallucinations come from predicting plausible text, not from looking up facts.
- Ground with RAG and tools, allow "I don't know", and require citations.
- Verify numbers, IDs and citations in code before showing the answer.
- On failure: retry once with the error, then hand over to a human.