Downsampling

What downsampling is

Downsampling trades detail for reach. A metric scraped every 15 seconds produces 5,760 points per series per day. For the last few hours that resolution is essential: you want to see a 30-second spike during an incident. For a capacity review covering last year, nobody needs it, and querying millions of raw points per series makes dashboards slow and storage expensive. Downsampling replaces old raw points with rollups at coarser resolution, typically in tiers such as raw, 5 minutes and 1 hour.

It is a standard technique in time series databases. Graphite's Whisper format defines retention tiers like 10s:1d,1m:30d,1h:1y. Prometheus itself does not downsample, but Thanos does it in its compactor, creating 5-minute blocks from data older than 40 hours and 1-hour blocks from data older than 10 days.

Downsampling versus sampling

The names are similar, the techniques are not:

	Downsampling	Sampling
Applies to	Time series (metrics)	Individual events (traces, logs)
Method	Aggregates many points into one per interval	Keeps a subset of events and drops the rest
What survives	Statistical summary of every series	Complete detail for selected events
Typical timing	After data ages past a threshold	At collection or ingest

A downsampled series still accounts for every original value in its aggregates. Sampling, such as head- or tail-based trace sampling, keeps some events in full fidelity and loses others entirely. Both reduce volume, and most mature setups use both.

How rollups preserve meaning

A single average per interval is not enough. If you store only the mean, a 10-second CPU spike to 100% disappears into a 5-minute average of 30%. Good downsampling keeps several aggregates per interval so different questions remain answerable:

• min and max to preserve peaks and troughs.
• sum and count so averages can be recomputed correctly across intervals and series.
• Counter-aware values so rates stay correct across counter resets. Thanos stores a dedicated counter aggregate for this.
• Histogram buckets, summed per interval, so percentiles can still be estimated later.

What you cannot do is average pre-computed percentiles. The mean of twelve 5-minute p99 values is not the hourly p99. Keep bucket counts instead.

Example: retention tiers

An illustrative tiering for a platform team:

raw (15s)  -> keep 14 days    incident analysis, alert tuning
5m rollup  -> keep 90 days    weekly reviews, SLO reports
1h rollup  -> keep 13 months  capacity planning, year-over-year

In Thanos the equivalent is set with --retention.resolution-raw, --retention.resolution-5m and --retention.resolution-1h. Note that Thanos documents downsampling mainly as a way to speed up long-range queries: it adds blocks, so you only save storage if raw retention is shorter than the rollup retention.

Downsampling with recording rules

Where the storage layer does not downsample, teams often pre-aggregate with recording rules. A rule such as job:http_requests:rate5m = sum by (job) (rate(http_requests_total[5m])) computes a compact series that drops per-instance and per-path labels. It reduces both resolution and cardinality, and long-range dashboards query the small series instead of thousands of raw ones. The trade-off is that you must decide in advance which questions matter: dimensions removed by the rule cannot be recovered later. A common pattern is to keep raw series for a short period and recorded aggregates much longer, which mirrors classic tiered downsampling.

Best practices

• Align query step with resolution. A PromQL rate() over 1 minute on 1-hour data returns nothing useful; use windows of several resolution steps.
• Reduce cardinality first. Downsampling shrinks points per series, not the number of series. High cardinality stays expensive at every resolution.
• Keep raw data long enough to cover your incident and postmortem window.
• Document the tiers so dashboard users know why a year-long graph looks smoother than last week's.

Common mistakes

Frequent mistakes include keeping only averages, alerting on downsampled data and wondering why short spikes never fire, and confusing downsampling with display-level decimation. Chart libraries often thin points for rendering, for example with algorithms such as LTTB, which changes what you see but not what is stored.

How Obseria helps

Obseria retains metric data points for 13 months by default, so long-range trends stay available without running your own compaction layer, and it accepts Prometheus metrics alongside OTLP. Obseria Pipeline filters data at ingest, and filtered data costs only ingestion, which is often the bigger lever: our article on a pipeline-first approach to observability costs explains why.