Detailed Component Design

The main components are:

API gateway: In charge of caching the rate limiter rules from PostgreSQL refreshing after a given cadence (for instance 5 minutes since rules should not change very often). For every request from a non-admin user, it would use those cached rules as input parameters for Redis that will respond with 429 and a Retry-After header specifying the seconds where the user should retry or allowing the request to the API servers.
Counter (Redis): Implements the token bucket algorithm on demand. This means that for every request received checks the number of tokens allowed for user/API that is the minimum between the capacity of the token bucket and number of remaining tokens plus the refill rate (if we have 100 req per minute, the rate is 100/60 per second, so we calculate the number of seconds that have passed since the last timestamp and now). If the number of tokens after this operation is greater than one we allow the operation so the gateway can redirect to the API servers. Otherwise it is 429 after saving in the counter the current number of tokens and now (will be used in the next iteration)
Rules DB: SQL db that will store the rules to be applied by the rate limit. It is modified by the Admin via API. This DB is consulted directly by the API gateway to keep the cache up-to-date so it can be used as input parameters for the counter.

High QPS and Fast Decision Path: The rate limit decision must happen in under 5ms to not add significant latency. The entire check runs as a Lua script on Redis, making it atomic and avoiding round-trips. The gateway does NOT query the Rules DB per request — rules are cached locally in memory and refreshed every 5 minutes. This means the fast path is: read rule from local memory, execute Lua script on Redis, return allow/reject. No database involved in the hot path.

Burst Traffic and Local Aggregation: Each gateway instance maintains a local in-memory counter per user. If a user has already been rejected in the last N seconds locally, the gateway rejects immediately without consulting Redis. This serves as a first layer that absorbs burst traffic and reduces Redis load by up to 90% for repeat offenders. Only requests that pass the local check reach Redis.

High Availability and Partial Outages: Redis runs in replica mode with Redis Sentinel for automatic failover — if primary fails, a replica promotes within seconds. If Redis is completely unavailable, gateways fail open (allow requests) and alert the operations team, since blocking all traffic is worse than temporarily exceeding rate limits. The Rules DB has read replicas. If the DB is down entirely, gateways continue using their local cache. Cache misses (new user with no cached rule) fall back to a default rate limit until the cache refreshes.

If Redis and Rules DB are completely unavailable, we have the cache of the API gateway with the responses for the requests.

Configuration Rollout Without Downtime: Admin updates write to the Rules DB. Gateways refresh cache every 5 minutes via polling. For urgent changes, the Admin API publishes an invalidation message via Redis Pub/Sub, forcing all gateways to refresh immediately. Rules propagate gradually — no restart or downtime required.

Handling burst traffic at the gateway: Use a local aggregation and handling traffic using the Load Balancer so we can add more instances at ease.

Handle very high QPS and fast decision paths: Adding more API gateway instances, adding more Redis replicas using Redis Sentinel. Fast decision paths will be related to the cache in the API Gateway that will have the rules for the last 5 minutes.

Submissions

Displaying Solutions with Scores Ranging from 1 to 10