A hallucination (AI) is an output of a language model that is fluent and confident but factually wrong, invented or not supported by its input or sources. Examples are fabricated citations, nonexistent API functions, wrong figures or tool calls with made-up arguments. Hallucinations follow from how models generate text and can be reduced, but not fully eliminated.
What is an AI hallucination?
An AI hallucination occurs when a large language model produces content that reads well but is not true or not backed by the information it was given. The model does not lie in a human sense. It generates the next tokens based on learned probabilities, and a statistically plausible continuation is not necessarily a correct one. When the model lacks the relevant knowledge, it often still produces an answer instead of saying that it does not know.
For production systems this is a reliability problem, not only a quality issue. A hallucinated answer usually returns successfully, with normal latency and no error code, so classic monitoring does not notice it. Detecting it requires the methods of LLM observability and systematic evaluation.
Types of hallucinations
- Factual errors: wrong dates, names, figures or claims about the world.
- Unfaithful answers: the answer contradicts or goes beyond the documents supplied in the prompt, for example in a RAG application.
- Fabricated references: invented sources, URLs, paper titles, case numbers or quotes.
- Code hallucinations: calls to library functions, parameters or configuration keys that do not exist.
- Tool and agent hallucinations: calling a tool that is not available, inventing arguments or claiming that an action was performed when it was not.
Why do LLMs hallucinate?
Several factors contribute. Training data can be incomplete, outdated or contradictory. Models have a knowledge cutoff and do not know recent events unless they receive them as context. Training objectives reward plausible text and, depending on the setup, may reward guessing more than admitting uncertainty. On the application side, vague prompts, missing or irrelevant retrieved context, overly long contexts in which relevant facts get lost, and high sampling temperature all raise the risk.
In agent workflows the risk compounds. A hallucinated intermediate result, such as an invented order number, becomes the input of the next step, and later steps build on it. The final answer can then look consistent while resting on a false premise, which is why each step of an agent run should be traceable and, where possible, validated.
How to detect hallucinations in production
No single signal reliably detects every hallucination, so teams combine several approaches:
- Groundedness checks: compare each claim in the answer with the retrieved sources and flag statements without support.
- LLM-as-a-judge: a second model scores faithfulness or correctness against a rubric. This is scalable but itself imperfect and should be calibrated against human labels.
- Deterministic validation: verify citations, links, IDs, numbers and code against the source system, a schema or a compiler.
- Reference tests: run a curated set of questions with known answers as part of LLM evaluation before each release.
- User feedback: thumbs down, corrections and escalations, linked to the trace of the request.
Store evaluation results as attributes or events on the trace of the request. You can then filter hallucinated answers by model, prompt version or retrieval source and see what the model actually received as input.
Example: tracing a wrong answer
An illustrative case: an internal assistant tells employees that a policy allows 30 days of remote work abroad, but the current policy says 20. The trace of the request shows that the retrieval step returned an outdated version of the document with a high similarity score, while the current version was not indexed yet. The model answered faithfully to its context, so the root cause is stale data in the index, not the model. The fix is to re-index on document changes and to add the document date to the retrieved metadata.
How to reduce hallucinations
- Ground answers in retrieved, current and access-controlled sources and ask the model to cite them.
- Instruct the model to say when the context does not contain the answer, and accept that answer as valid.
- Use structured outputs and schemas for tool calls, and validate arguments before execution.
- Lower sampling temperature for factual tasks.
- Add AI guardrails that block or flag unsupported claims in sensitive domains.
- Keep a human in the loop for decisions with legal, financial or safety impact.
How Obseria helps
Obseria stores the full trace of a GenAI request, including retrieval and model spans, so a flagged answer can be traced back to its context, prompt version and model. Evaluation scores and user feedback sent as OpenTelemetry attributes or events can be charted and alerted on next to latency and token metrics. More in AI observability.
