Loading...
Define the APIs expected from the system. This is your chance to analyze and define the read and write paths so that you can come up with the high-level design...
Post/isAllowed(boolean allowed, retry_after time)
get/getLimit
Post/admin/add{ rate, windowburst}
Post/admin/update{id, rate, windowburst}
Post/admin/delete{id}
get/admin/view{id} { rate, windowBurst, }
Describe the overall system architecture. Identify the main components needed to solve the problem end-to-end. Use the diagramming tool to create a block diagram.
client, which can be either a mobile application or a web application. It talks to CDN to get the static content of it, and there is a small content offered limiting present in CDN as well.
api gateway- manager auth, routing to rate limiting
Ratelimiter- Stateless instance can work with redis to get the stored information about the config
Admin- will directly talk to RDBMS for crud api
Rediscache-- will have the cached details of service and limits, if there is a cache miss, we will focus on request coalescing, and work with a default low level rate limiting per server
For Highvolume data we can move to separate redis cachee
requests be validated -- once the service comes to API Gateway, we will check the authentication and authorization. If the service is authenticated and authorized, we will move it to rate limiting servers. If not, we will be sending it back
This API gateway, we can also have a load auto load balancer where if there is a heavy traffic on the servers, we can scale the rate limiters to multiple instances, so then it can actually use the per user, per API rate limiting properly. And at the same point of time, we can actually shard the database as well as the Redis cache with the per user thing on the hotkeys, so that we can have proper workaround for scalabilities.
avoid hotspots for very active keys-The hotkeys will be separated into a different Redis cache so that we can have a separate cluster for them for any rate limit, so that the rate limiter is properly defined for them as well. And this can be done by sharding for the hotkeys.
config changes are rolled out without downtime- we will do a public green deployment where we will try to deploy a config and then slowly move the traffic to the new deployment and then kind of change it. Or what we can do is we can actually, once the config is deployed, we can have this RDBMS data store as read-write and then do it like that.
What policy will you choose if the RL service is downIf my rate limiter service is down, I would choose the per traffic per IP bandwidth, like where suppose 100 requests per second on a slower pace, and so that is, at that point, we will still be our service will still be able to work.
Deep dive into 2-3 key components. Explain how they work, how they scale, discuss tradeoffs, capacity, and any relevant algorithms or data structures.
So we will have for the algorithm, we will have multiple algorithms like token bucket algorithm, leaky bucket algorithm, sliding window algorithm, and sliding window counter algorithm. So we can choose depending on our own preferences, like if we use a payment platform, we don't want any leakage, so we will have a token bucket algorithm. And in this way, we will have tokens and we will process those tokens and request can wait. If in other things, we can actually use the sliding window counter algorithm where we can actually, on every slide, we can actually check what is the total number of things and we can support that.
For sharding the database, I think it will actually depend on the per API, per user sharding, because if there are hotkeys, that means if certain API is doing, then we can have a per API sharding. But I think mostly it should be from, yeah, per API sharding can be the best one. And in that case, we will be able to handle the traffic. If there is a lot, lot of things are coming up, we can actually shard the database and store it in a different Redis cluster cache.
concurrent requests - we will use an atomic key for the Redis cache and RDBMS, and we will use the idempotency key so that there is no mix-match in the concurrency. And we will make sure that we can have a retry mechanism, and then if anything goes bad, then we will have a conflict pops up to check, like, how it is coming off.
I have explicitly told you that if my external cache is unavailable, I would try to do the request coalescing model, and I will try to repopulate the cache. At the same point of time, I will make sure that the different service can repopulate the cache in different times, so we will have a jitter in place as well as, like, we'll also have a standard, you can say, API rate limits, so that that can be addressed by the time we are doing the repopulation.
We use Redis Lua scripts to ensure atomic read-modify-write for token consumption, preventing race conditions under high concurrency
All rate limiting decisions use Redis server time to avoid client clock skew. We also add a small buffer (~100–200ms) to avoid boundary inconsistencies.
connection pool saturation-- backpressure queue
batching- Batching is only applied for soft limits and aggregation-friendly algorithms like sliding window counters. For strict guarantees (e.g., payments), every request is evaluated individually.