LLM evaluation (evals) is the systematic measurement of how well a language model application performs a task. It combines test datasets, automated checks, model-based judges and human review to score properties such as correctness, faithfulness to sources, relevance, safety and format, both before a release and continuously on production traffic.
What is LLM evaluation?
LLM evaluation, usually shortened to evals, answers a question that classic tests cannot: is the output of a non-deterministic model good enough? A unit test checks that a function returns an exact value. An LLM can phrase a correct answer in many ways, and a fluent answer can still be wrong. Evals therefore score outputs against criteria, across many examples, and track those scores over time.
Evals matter whenever something changes: a new model version, a modified prompt, a different retrieval strategy or a new tool. Without them, teams judge changes by a handful of manual tries, and regressions surface only when users complain.
A good eval suite turns vague impressions such as "the new model feels better" into numbers per criterion and per category. That makes decisions explicit: a team can see that a cheaper model is equally good for simple questions but worse for complex ones, and route traffic accordingly. Evals also document what the application is supposed to do, which helps when several people change prompts and configuration.
Types of evals
- Offline evals run a fixed dataset of inputs, ideally with reference answers, through the application before release. They work like a regression test suite.
- Online evals score a sample of real production traffic continuously, often without a reference answer.
- Human evaluation by domain experts remains the reference for subtle quality questions and is used to calibrate automated methods.
- Red-team evals test robustness against adversarial inputs such as prompt injection.
Methods and metrics
| Method | Suitable for | Limitation |
|---|---|---|
| Deterministic checks (exact match, regex, JSON schema) | Classification, extraction, structured output | Cannot judge free text |
| Reference similarity (e.g. embedding similarity) | Answers with a known reference | Similar wording is not the same as correct content |
| LLM-as-a-judge | Relevance, faithfulness, tone, helpfulness | Judges have biases and need calibration against human labels |
| Task success | Agents: was the goal reached, with how many steps | Requires a clear definition of success |
For RAG systems, common criteria are context relevance (did retrieval find the right documents), faithfulness (is the answer supported by them) and answer relevance. Faithfulness scores are a practical way to quantify hallucinations.
Example: evaluating a prompt change
An illustrative workflow: a team maintains 300 curated questions for its documentation assistant, each with a reference answer and the expected source. Before switching to a new prompt version, it runs both versions over the dataset. A judge model scores faithfulness and relevance from 0 to 1, deterministic checks verify that every answer cites a source, and token usage and latency are recorded alongside. The new prompt scores higher on relevance but lower on faithfulness for pricing questions, so the team fixes it before rollout, then releases it to a small share of traffic as a canary with online evals active.
Best practices
- Build the dataset from real user questions, including hard and edge cases, and extend it with every production failure.
- Define criteria precisely and give judges a rubric with examples.
- Calibrate LLM judges against human ratings and re-check after changing the judge model.
- Evaluate quality together with cost and latency; a better answer that takes three times as long may not be an improvement.
- Compare against a baseline run, not against absolute numbers alone.
- Version datasets, prompts, models and judge configurations so results stay reproducible.
Common mistakes
Typical pitfalls are datasets that are too small or too easy, judges that reward long answers, evaluating only before release and never in production, and averages that hide failures in one important category. Report scores per category and look at the worst examples, not only the mean.
Another frequent issue is evaluating the model in isolation. In production, quality depends on the whole chain: retrieval, prompt assembly, tool calls and post-processing. Evals should run against the real application path, and their results should be linked to traces so that a low score can be traced back to the step that caused it. Finally, keep the judge model and its prompt stable while comparing versions; changing the judge and the application at the same time makes results impossible to interpret.
How Obseria helps
Obseria's AI observability includes built-in evaluators for relevance, faithfulness, toxicity and PII leaks on production traffic, and compares quality, cost and latency across model and prompt versions in the same traces.
