Latency is the time between the start of an operation and its completion, for example from sending a request to receiving the response. In observability, latency is measured as a distribution and reported with percentiles such as p50, p95 and p99, because averages hide the slow outliers that users notice most. It is one of the four golden signals.
What latency means
Latency is how long something takes. In networking, the term often refers to the delay for data to travel from one point to another, such as round-trip time. In application observability, latency usually means the duration of a request or operation as seen by the caller: how long from sending an HTTP request until the full response arrives, or how long a database query, a function call or a queue message takes to be processed.
Strictly speaking, some engineers distinguish latency (time spent waiting before work starts) from response time (waiting plus service time). In day-to-day monitoring, both terms are commonly used for end-to-end duration. Latency is one of the four golden signals, together with traffic, errors and saturation.
Why averages mislead
Latency distributions are rarely symmetric. Most requests are fast, while a long tail is slow because of cache misses, garbage collection, lock contention, retries or slow dependencies. An average blends these together. Consider an illustrative sample of 1,000 requests where 980 take 100 ms and 20 take 3 seconds: the mean is 158 ms, which looks fine, yet one in fifty users waits 3 seconds.
That is why latency is reported as percentiles. The p50 (median) describes the typical experience, the p95 and p99 describe the slow tail. Percentiles cannot be averaged across instances; they must be computed from the combined distribution, which is why histograms are the preferred metric type for latency.
Tail latency and fan-out
Tail latency matters even more in distributed systems. When one user request fans out to many backend calls, the slowest call determines the overall response time. Google's paper "The Tail at Scale" illustrates this: if a request touches 100 servers and each server is slow 1% of the time, about 63% of user requests will hit at least one slow server, because 1 − 0.99 to the power of 100 ≈ 0.63. Reducing the p99 of individual services is therefore one of the most effective ways to improve the median of the end-to-end experience.
Measuring latency
| Source | What it measures | Good for |
|---|---|---|
| Histogram metrics | Distribution per service, endpoint or route | Dashboards, SLOs, alerting on percentiles |
| Distributed traces | Per-request breakdown across services and spans | Finding where time is spent |
| Real user monitoring | Latency as experienced in browsers and apps | User-perceived performance, regions, devices |
| Synthetic checks | Latency of scripted requests from fixed locations | Baseline comparisons, availability outside traffic peaks |
Measure as close to the user as possible. Server-side latency excludes network time, TLS handshakes and client rendering. Load tests should also avoid coordinated omission, where a load generator waits for slow responses before sending the next request and therefore under-reports tail latency.
Example: reading a latency breakdown
In an illustrative case, the p99 of a search API rises from 600 ms to 1.4 seconds after a release, while p50 stays at 120 ms. A trace from the slow tail shows the server span at 1.35 seconds, of which 1.1 seconds are spent in a child span calling a recommendation service. That service's own p99 is only 200 ms, but the trace shows five sequential calls instead of one batched call. The fix is a code change in the search API, not more capacity for the recommendation service. Without the per-span breakdown, the team might have scaled the wrong component.
Common causes of high latency
- Saturation and queuing. As utilization approaches capacity, queues grow and latency rises sharply, long before CPU reaches 100%.
- Slow dependencies such as databases, third-party APIs or overloaded downstream services.
- Inefficient code paths, for example N+1 queries, serialization overhead or lock contention, visible in continuous profiles.
- Runtime pauses such as garbage collection or cold starts in serverless functions.
- Network effects: cross-region calls, DNS resolution, retransmits and connection setup.
Best practices
- Define latency SLOs on percentiles, for example "99% of checkout requests complete within 800 ms", rather than on averages.
- Record latency as histograms with buckets that cover your SLO thresholds, and use exemplars to jump to slow traces.
- Separate successful and failed requests. Fast errors can make latency look better than it is.
- Set timeouts and budgets along the call chain so one slow dependency cannot consume the entire request budget.
- Compare against a baseline to catch gradual regressions after releases.
How Obseria helps
Obseria records latency distributions for every service and endpoint from OpenTelemetry traces and metrics, and distributed tracing shows which span consumed the time. Real user monitoring adds the browser and mobile perspective, and SLOs on percentile latency come with burn-rate alerting. Learn more about defining targets in What is an SLO?.