Rate limiting is a technique that restricts how many requests a client, user, API key or tenant may send to a service within a time window. Requests above the limit are rejected, typically with HTTP 429 Too Many Requests, or delayed. It protects capacity, ensures fairness between tenants and keeps costs predictable.
What is rate limiting?
Rate limiting caps the number of operations a caller may perform per unit of time, for example 100 requests per second per API key or 5 login attempts per minute per account. When a caller exceeds the limit, the service rejects or delays further requests until the caller is back within budget.
Rate limits serve several goals at once: protecting a service from overload, preventing one tenant from becoming a noisy neighbor, slowing down brute-force and scraping attacks, and keeping usage-based costs of downstream APIs under control. They are a form of admission control: it is better to reject a fraction of requests quickly than to let all requests slow down.
Rate limiting algorithms
| Algorithm | How it works | Trade-off |
|---|---|---|
| Token bucket | A bucket holds up to b tokens and is refilled at rate r. Each request consumes a token; with no token left, the request is rejected. | Allows bursts up to b while enforcing the average rate r. The most common choice for APIs. |
| Leaky bucket | Requests enter a bucket that drains at a constant rate. When the bucket is full, new requests are dropped. | Produces a smooth output rate (traffic shaping), but adds queueing delay and absorbs bursts less flexibly. |
| Fixed window counter | Counts requests per fixed interval, such as per calendar minute. | Simple and cheap, but allows up to twice the limit around window boundaries. |
| Sliding window log | Stores a timestamp per request and counts those within the last interval. | Exact, but memory grows with traffic. |
| Sliding window counter | Weights the previous window's count with the current one. | Good approximation with constant memory per key. |
In distributed systems, counters usually live in a shared store such as Redis, or each instance enforces a local share of the global limit. Local limits are faster and fail independently; global limits are more precise.
HTTP 429 and Retry-After
For HTTP APIs, the standard response to a client that exceeds its limit is 429 Too Many Requests, defined in RFC 6585. The response can include a Retry-After header (RFC 9110), given either as a number of seconds or as an HTTP date, telling the client when to try again:
HTTP/1.1 429 Too Many Requests
Content-Type: application/json
Retry-After: 30
{"error": "rate_limited", "message": "Limit of 100 requests per minute exceeded"}Many APIs also return headers with the limit, remaining quota and reset time, often named X-RateLimit-Limit, X-RateLimit-Remaining and X-RateLimit-Reset. These names are a convention, not a standard; the IETF HTTPAPI working group is drafting standardized RateLimit header fields. A service that is overloaded as a whole, rather than limiting one client, typically responds with 503 Service Unavailable, which can also carry Retry-After.
Handling rate limits as a client
- Honor Retry-After. If present, wait at least that long before retrying.
- Back off exponentially with jitter. Randomized delays prevent all clients from retrying at the same instant and creating a new spike.
- Cap retries. Combine a retry budget with a circuit breaker so a struggling dependency is not hammered further.
- Throttle proactively. Use the remaining-quota headers to slow down before you hit the limit.
Monitoring rate limiting
Rate limiting is only useful if you can see it working. Track the number of throttled requests per client, route and limit, the share of 429 responses in your overall error rate, and how close top consumers are to their limits. Decide deliberately whether 429s count against your SLO: rejections caused by a client exceeding its contract usually do not, while rejections caused by limits set too low for normal traffic do. A sudden rise in 429s after a deploy often points to a misconfigured limit or a retry storm from a client.
Common mistakes
- Limits set without data: measure real per-client traffic and its peaks before choosing numbers.
- Keying limits on IP address alone, which punishes users behind shared NAT and is easy to evade with many IPs.
- Returning 500 or 503 instead of 429, so clients cannot tell throttling from failures.
- Rate limiting at the edge but not protecting expensive internal endpoints.
How Obseria helps
Obseria records HTTP status codes and attributes such as route and client on every span, so you can chart 429s per API key or tenant and see in distributed traces where throttling happened. Anomaly detection highlights unusual spikes in throttled requests, and gateway and proxy metrics, for example from NGINX, show limit hits at the edge.
