Loading...
X-RateLimit-LimitX-RateLimit-RemainingX-RateLimit-ResetRetry-AfterLet us assume that the traffic shall be about 1 million requests per second, since all requests shall pass through the rate limiter.
Storage estimation depends upon the rules of the rate limiter i.e. scope (per api/ per user/ per ip/ per region/ per org/ all together), the type of algorithm used for rate limiting(token bucket, sliding window, leaky bucket, fixed window, etc.) , etc. So in the worst case lets say 1000 rules per user, for a million users. Thus, the storage estimations will be
As we can see storage isn't the issue, the crux of the problem is throughput to handle heavy traffic.
Moreover, the latency should be at minimum. Let us assume an API sitting behind the rate limiter takes a total of 100ms per call. Now, if the rate limiter takes 30ms, then total time becomes 130ms. The rate limiter itself caused an increase of 30% time for the total round trip. So, let us estimate that an acceptable latency would be sub 10ms , so we need the rate limiter to make decisions under 10ms.
On rules modification side:
We shall let users i.e. people using the rate limiter to modify and add rules, apis to the limiter. These shall be much less frequent, let's estimate at around 10 modifications per day. This is easy to maintain and handle since it is just storing rules and at a very low frequency.
Request Path:
Client → API Gateway → RL-Sidecar → (allow/reject) → Backend
Rule Update Path:
Admin → RL-Modify → PostgreSQL → Kafka/PubSub → Sidecars
Instead of a single Redis:
Key format:
rl:{hash(scope_value)}:{api}
👉 Ensures:
We use Token Bucket Algorithm with local buffering
tokens=min(capacity,tokens+rate⋅Δt)tokens = \min(capacity, tokens + rate \cdot \Delta t)tokens=min(capacity,tokens+rate⋅Δt)
Instead of:
1 request → 1 Redis call
We do:
Sidecar fetches 1000 tokens → serves locally
👉 Benefits:
Rules Table:
rule_id (UUID PK)
scope_type (user/ip/org/api_key)
scope_value
api_endpoint
algorithm (token_bucket, leaky_bucket)
max_requests
window_seconds
burst_size
Constraint:
UNIQUE(scope_type, scope_value, api_endpoint)
Key:
rl:{scope}:{scope_value}:{api}:{shard}
Value:
Use Redis Lua Scripts:
Instead of:
rl:user123:/payments
Use:
rl:user123:/payments:shard1
rl:user123:/payments:shard2
...
👉 Distributes load across Redis nodes
Instead of direct push:
Flow:
RL-Modify → Kafka → Sidecars consume → update cache
👉 Benefits:
If Redis unavailable:
Adaptive:
👉 Optional:
Track:
Tools:
| Decision | Benefit | Tradeoff |
| Sidecar | Low latency | More instances |
| Redis Cluster | Scalable | Operational complexity |
| Local buffering | Low Redis load | Slight inconsistency |
| Event streaming | Scalable updates | Eventual consistency |