Exponential Backoff

What is exponential backoff?

Exponential backoff is the standard way to retry failed remote calls without making the failure worse. Instead of retrying immediately or at a fixed interval, a client waits a little after the first failure, twice as long after the second, four times as long after the third, and so on, until it either succeeds, reaches a maximum delay or gives up after a set number of attempts.

The idea is simple: if a dependency is overloaded, every immediate retry adds load at exactly the wrong moment. Growing delays reduce retry pressure quickly and give the dependency time to recover, while still masking short, transient errors such as a dropped connection or a brief leader election.

The exponential backoff formula

The basic formula for attempt n (starting at 0) is:

delay = min(cap, base * 2^n)

base is the initial delay, 2 is the multiplier (some libraries use 1.5 or make it configurable) and cap is the maximum delay. With an illustrative base of 100 ms and a cap of 10 s, the delays are 100 ms, 200 ms, 400 ms, 800 ms, 1.6 s, 3.2 s, 6.4 s and then 10 s for every further attempt. A maximum number of attempts, or an overall deadline, ends the sequence.

Why exponential backoff needs jitter

Pure exponential backoff has a flaw: if a thousand clients fail at the same moment, they all retry after exactly 100 ms, then after exactly 200 ms, and keep hitting the service in synchronized waves. Adding randomness, called jitter, breaks up those waves. Common variants, popularized by an AWS Architecture Blog article on backoff and jitter:

Variant	Formula	Behavior
No jitter	min(cap, base * 2^n)	Synchronized retry waves
Full jitter	random(0, min(cap, base * 2^n))	Spreads retries widely, low total load
Equal jitter	d/2 + random(0, d/2) with d = min(cap, base * 2^n)	Guarantees at least half the delay
Decorrelated jitter	min(cap, random(base, previous * 3))	Each delay depends on the previous one

Full jitter is a good default for most clients. The important point is to use some jitter at all.

Example: retries in Go

base, maxDelay := 100*time.Millisecond, 10*time.Second
for attempt := 0; attempt < 6; attempt++ {
    err := callPaymentAPI(ctx)
    if err == nil || !isRetryable(err) {
        return err
    }
    d := base << attempt          // base * 2^attempt
    if d > maxDelay { d = maxDelay }
    time.Sleep(time.Duration(rand.Int63n(int64(d)))) // full jitter
}

Only retryable errors are retried, for example timeouts, HTTP 429, 502, 503 and 504 or gRPC UNAVAILABLE. A 400 Bad Request will fail the same way every time. If the server sends a Retry-After header with a 429 or 503 response, honor it instead of the computed delay; this is how backoff cooperates with server-side rate limiting.

Best practices

• Retry only idempotent operations, or use idempotency keys, so a retry after a timeout does not charge a customer twice.
• Bound everything: a cap on the delay, a maximum number of attempts and an overall deadline that fits the caller's own timeout.
• Retry at one layer. If three layers each make three attempts, the bottom service can see up to 27 calls per user request. Retry budgets limit retries to a fraction of normal traffic.
• Combine with a circuit breaker that stops calls entirely when a dependency is clearly down.
• Use the built-in policies of your SDK or framework. AWS SDKs, gRPC retry policies (initialBackoff, maxBackoff, backoffMultiplier) and resilience libraries implement backoff with jitter correctly.
• Hand off to a queue for work that can wait: after the last attempt, move the message to a dead letter queue instead of retrying forever.

Making retries observable

Retries hide problems as well as solve them. A request that succeeds on the fourth attempt looks healthy in an error counter but adds seconds of latency. Record the attempt number as an attribute, emit a counter of retries per dependency, and make each attempt visible as its own client span. A sudden rise in the retry rate is often the earliest sign that a downstream dependency is degrading. Also compare the number of attempts with the number of original requests: if that ratio climbs sharply, your clients are amplifying load on a service that is already in trouble.

How Obseria helps

With OpenTelemetry instrumentation, each retry attempt appears as a separate span in distributed tracing, so Obseria shows where time went and which dependency caused the retries. Retry counters and latency can be queried together in OQL or PromQL and used for alerts before users notice.