A metric is a numeric measurement of a system captured at regular intervals and stored as a time series. Each data point combines a metric name, a value, a timestamp and a set of attributes (labels). Metrics are cheap to store and query, which makes them the backbone of dashboards, alerting and SLOs.
What is a metric?
A metric is a number that describes the state or behavior of a system at a point in time: requests per second, error count, heap usage, consumer lag. Because a metric is recorded repeatedly, it forms a time series, a sequence of timestamped values identified by a metric name plus a unique combination of attributes such as service.name, http.route or region.
Metrics are one of the core signals of observability, next to logs, traces and profiles. They are pre-aggregated, so they stay compact no matter how much traffic flows through a service. That property makes them ideal for long-term trends, capacity planning and alerting, while traces and logs provide the per-request detail you need when an alert fires.
Anatomy of a metric data point
- Name: a stable identifier such as
http.server.request.duration. OpenTelemetry semantic conventions define names for common technologies. - Value: an integer or floating-point number, or for histograms a set of bucket counts plus sum and count.
- Timestamp: when the value was observed; cumulative sums also carry a start time.
- Attributes (labels in Prometheus terms): key-value pairs that split a metric into individual series.
- Unit: for example
s,Byor{request}, so tools can convert and display values correctly.
Metric types: OpenTelemetry and Prometheus
OpenTelemetry separates the instrument you use in code from the data model that is exported. Synchronous instruments are called inline with your code; asynchronous (observable) instruments are read through a callback at collection time.
| OpenTelemetry instrument | Typical use | Closest Prometheus type |
|---|---|---|
| Counter / ObservableCounter | Monotonic totals: requests, bytes sent | counter |
| UpDownCounter / ObservableUpDownCounter | Values that rise and fall: active connections, queue size | gauge |
| Gauge / ObservableGauge | Sampled current values: temperature, CPU utilization | gauge |
| Histogram | Distributions: latency, payload size | histogram |
Prometheus knows four metric types: counter, gauge, histogram and summary. Summaries compute quantiles on the client and cannot be aggregated across instances, which is why histograms are usually the better choice. OpenTelemetry also supports exponential histograms, which adjust bucket boundaries automatically and give accurate percentiles without hand-tuned buckets.
Temporality
Sums and histograms are exported either as cumulative values (total since process start, the Prometheus model) or as delta values (change since the last export). Backends must know which one they receive; mixing them up produces rates that are wildly wrong.
Example: instrumenting a request handler
The following illustrative Python snippet records request count and latency with the OpenTelemetry API:
meter = metrics.get_meter("checkout")
requests = meter.create_counter("app.checkout.requests", unit="{request}")
duration = meter.create_histogram("app.checkout.duration", unit="s")
requests.add(1, {"payment.method": "card", "outcome": "ok"})
duration.record(0.182, {"payment.method": "card"})With two payment methods and two outcomes, the counter produces four time series per service instance. Adding a user.id attribute instead would create one series per user, a classic cardinality explosion.
Best practices
- Start with the golden signals. Latency, traffic, errors and saturation cover most service health questions; see golden signals.
- Use histograms for latency. Averages hide tail behavior; histograms let you compute p95 and p99 after the fact and aggregate across pods.
- Keep attributes bounded. Use route templates such as
/orders/{id}, never raw URLs, IDs or timestamps. - Follow semantic conventions. Standard names make dashboards and alerts portable across languages and vendors.
- Record units. A latency metric without a unit invites off-by-1000 errors between milliseconds and seconds.
Metrics, alerts and SLOs
Most alerting is built on metrics because they are evaluated continuously and cheaply. A good alert compares a rate or ratio over a window, for example the share of HTTP 5xx responses over the last five minutes, rather than a single raw value. Service level objectives go one step further: an SLI such as "proportion of requests faster than 300 ms" is computed from a latency histogram, and the error budget shows how much unreliability is left in the current window. Burn-rate alerts on that budget page people only when users are measurably affected, which keeps noise low.
Common mistakes
- Computing a rate from a gauge, or reading a counter's raw value instead of its rate.
- Averaging precomputed percentiles across hosts, which is mathematically meaningless.
- Letting every team invent its own names for the same measurement.
- Treating metrics as a replacement for traces: a metric tells you that p99 latency rose, a trace tells you where.
How Obseria helps
Obseria ingests metrics via OTLP, scrapes Prometheus endpoints and collects host and container metrics through the Obseria Agent. You can query them with OQL, PromQL or SQL, link them to traces through exemplars, and let Obseria Intelligence baseline them for anomaly detection. Obseria Pipeline can drop or aggregate high-cardinality attributes at ingest, so you only pay for the series you actually use.
