Introduction

There is no single best rate-limiting algorithm for every web service. The right choice depends on whether you care most about strict fairness, burst tolerance, simplicity, or distributed enforcement across many servers.

The Usual Candidates

Most production systems choose from four families of rate limiters:

• fixed window
• sliding log or sliding window
• leaky bucket
• token bucket

For web APIs, token bucket is often the most practical default because it allows short bursts while still enforcing a stable long-term rate.

Why Token Bucket Works Well

A token bucket stores tokens up to a maximum capacity. Each request consumes one token. New tokens are added over time at a configured refill rate.

This gives you two useful behaviors at once:

• sustained traffic is capped by the refill rate
• short bursts are allowed up to the bucket size

For example, a limit of 10 requests per second with a bucket capacity of 20 allows a client to make 20 immediate requests after being idle, but not to sustain that burst forever.

A Runnable Token Bucket Example

The following Python implementation stores the current token count and refill timestamp in memory. It is suitable for understanding the algorithm or for a single-process service.

1import time
2
3
4class TokenBucket:
5    def __init__(self, capacity, refill_rate):
6        self.capacity = capacity
7        self.refill_rate = refill_rate
8        self.tokens = capacity
9        self.last_refill = time.monotonic()
10
11    def allow(self, cost=1):
12        now = time.monotonic()
13        elapsed = now - self.last_refill
14        self.tokens = min(
15            self.capacity,
16            self.tokens + elapsed * self.refill_rate
17        )
18        self.last_refill = now
19
20        if self.tokens >= cost:
21            self.tokens -= cost
22            return True
23        return False
24
25
26bucket = TokenBucket(capacity=5, refill_rate=1)
27for i in range(7):
28    print(i, bucket.allow())
29
30time.sleep(2)
31print("after refill", bucket.allow())

The logic is simple: refill based on elapsed time, cap the bucket, then decide whether the request can spend a token.

Comparing The Alternatives

A fixed-window limiter counts requests inside a discrete interval such as one minute. It is easy to implement, but it has a boundary problem. A client can send many requests at the end of one window and many more at the start of the next.

A sliding log stores timestamps for recent requests and removes old ones. It is accurate, but it can use significant memory under heavy traffic.

A leaky bucket processes requests at a steady outflow rate. That is useful when you want smooth throughput, but it is less flexible when short bursts are acceptable.

Token bucket sits in a practical middle ground. It is simple, cheap, and aligned with how many public APIs behave.

Distributed Systems Change The Design

An in-memory limiter works only when one process handles all requests for a client. In a multi-instance deployment, requests may hit different servers, so the state must live in a shared store such as Redis.

The usual distributed pattern is:

1. choose a key such as user id, API key, or IP address
2. store limiter state in Redis
3. update and check the limit atomically
4. return 429 Too Many Requests when the request is rejected

Redis Lua scripts are popular because they combine refill and consume logic in one atomic operation. Without atomicity, race conditions appear under concurrency.

Choosing The Key And Scope

Algorithm choice is only half of rate limiting. You also need to decide what you are limiting:

• per IP address
• per authenticated user
• per API key
• per route
• global system protection

Most real services use more than one limiter. For example, a service may enforce a global emergency limit, a per-user limit, and a stricter limit on expensive endpoints.

What To Return To Clients

A rejected request should be explicit. The standard response is HTTP 429. It is also useful to return metadata headers so clients know when to retry.

Typical headers include:

• remaining quota
• reset time
• retry delay

That does not change the core algorithm, but it improves client behavior and reduces support noise.

Common Pitfalls

A common mistake is selecting fixed windows only because they are easy. They often create unfair bursts at window boundaries and surprise users with inconsistent enforcement.

Another issue is using local memory in a horizontally scaled system. If each instance tracks limits independently, clients can exceed the intended rate by spreading traffic across servers.

Be careful with key choice as well. Limiting only by IP address can punish many legitimate users behind one NAT or proxy. Limiting only by user id may fail to block anonymous abuse.

Finally, do not treat rate limiting as a complete abuse-prevention system. It reduces load and slows attacks, but it does not replace authentication, authorization, bot detection, or request validation.

Summary

• Token bucket is often the best default for web requests because it balances burst tolerance and long-term control.
• Fixed windows are simple but have boundary artifacts, while sliding approaches trade memory for fairness.
• In distributed systems, keep limiter state in a shared store and update it atomically.
• Choose rate-limit keys carefully based on your threat model and product behavior.
• Return 429 responses with useful metadata so well-behaved clients can adapt.