RED Method

What is the RED Method?

The RED Method is a simple rule for monitoring services: for every service, measure the Rate of requests, the number of Errors and the Duration of those requests. It was popularised by Tom Wilkie, then at Weaveworks, as a service-oriented counterpart to Brendan Gregg's USE Method, which looks at resources rather than requests.

The idea is deliberately narrow. Instead of designing a bespoke dashboard for every microservice, teams apply the same three signals everywhere. An on-call engineer who opens any service sees the same layout and can answer the same questions: how much traffic is arriving, how much of it fails and how long it takes. That consistency is what makes RED valuable in architectures with dozens or hundreds of services.

The three RED signals

Signal	What it measures	Typical unit
Rate	Requests the service handles	Requests per second
Errors	Requests that fail	Errors per second or % of requests
Duration	Time to serve a request	Latency distribution, e.g. p50, p95, p99

Rate is closely related to throughput and shows load, traffic shifts and outages that look like a sudden drop to zero. Errors should count what the user experiences as a failure: HTTP 5xx responses, gRPC error codes, timeouts and, if relevant, responses that are technically successful but semantically wrong. Expressing errors as an error rate makes services with different traffic comparable. Duration must be measured as a distribution, not an average, because averages hide the slow tail that users actually notice. Track percentiles such as p95 and p99, ideally from histograms so they can be aggregated across instances.

RED vs. USE and the Golden Signals

RED, USE and Google's four Golden Signals overlap but have different focus. The Golden Signals are latency, traffic, errors and saturation. RED is essentially the first three, applied consistently per service. USE (utilization, saturation, errors) applies to resources such as CPUs, disks, queues and connection pools. In practice, teams combine them: RED tells you that a service is unhealthy from the caller's point of view, USE tells you which resource underneath is running out. RED alone has no saturation signal, so a service can look fine on RED until the moment a resource is exhausted.

Example: RED with PromQL

Assuming a service exposes an http_server_request_duration_seconds histogram (the OpenTelemetry semantic convention name, exported to Prometheus), the three signals can be queried like this:

# Rate: requests per second per service
sum by (service) (rate(http_server_request_duration_seconds_count[5m]))

# Errors: share of 5xx responses
sum by (service) (rate(http_server_request_duration_seconds_count{http_response_status_code=~"5.."}[5m]))
/ sum by (service) (rate(http_server_request_duration_seconds_count[5m]))

# Duration: p99 latency
histogram_quantile(0.99, sum by (service, le) (rate(http_server_request_duration_seconds_bucket[5m])))

A single histogram is enough for all three signals: its count gives the rate, a filtered count gives errors and the buckets give the duration distribution. Exact metric and label names depend on your instrumentation and exporter.

On a dashboard, place the three panels side by side for each service and use the same time range. Reading them together is more informative than any single signal: a drop in rate combined with rising errors often means callers fail fast or give up, while rising duration at a constant rate points to a slow dependency or resource contention. A sudden rate increase with stable errors and duration simply shows that the service absorbs more load. This first reading takes seconds and tells you where to look next, in traces, logs or resource metrics.

Best practices

• Measure at the service boundary. Instrument incoming requests on the server side and, where useful, outgoing calls from clients, so you can tell your own errors from those of dependencies.
• Use the same names and labels everywhere. Following OpenTelemetry semantic conventions lets one dashboard template serve every service.
• Separate errors from duration. Fast failures can make latency look better; track the duration of successful requests separately if needed.
• Alert on symptoms, not causes. RED signals map well to SLIs and SLO-based alerting, while resource metrics are better for diagnosis.
• Keep label cardinality under control. Group routes by template (/orders/{id}) instead of raw URLs.

Common mistakes

• Reporting average latency instead of percentiles from histograms.
• Counting 4xx client errors as service errors, which hides real failures in noise.
• Applying RED to batch jobs or queue consumers without adapting it; there, throughput, lag and job failures are the more meaningful signals.
• Stopping at RED and ignoring saturation, so capacity problems surface only when users already feel them.

How Obseria helps

Obseria derives rate, errors and duration for every instrumented service from OpenTelemetry traces and metrics, and Obseria TopoGraph places them on the automatically discovered service topology. RED metrics can be queried with PromQL, OQL or SQL, used as SLIs for SLOs with burn-rate alerts and linked to the underlying traces via distributed tracing when a signal turns red.