Loading...
List functional requirements for the system (Ask the chat bot for hints if stuck.)...
rate limiter will be server side
"hard" vs "soft" rate limiter
will be a standalone service
user should be notified when they have been rate limited
what are we rate limiting on?
make it configurable to adjust rate limiting settings
List non-functional requirements for the system...
should not add significant latency
highly available/reliable
1000 rps
Rate limiter configurations are not updated frequently
"eventually" consistent
Estimate the scale of the system you are going to design...
Define what APIs are expected from the system...
HTTP(method, url, version)
headers
body
returns either:
Defining the system data model early on will clarify how data will flow among different components of the system. Also you could draw an ER diagram using the diagramming tool to enhance your design...
configuration store: This can be a standard relational database to store the user-defined rate limiter configuration
rate limiter state: a queue of requests
in-memory DB, with a KV store (eg: Redis)
each ip_address is a key, mapping to a count for the number of requests recieved with that IP and a timestamp indicating when that value was last updated.
You should identify enough components that are needed to solve the actual problem from end to end. Also remember to draw a block diagram using the diagramming tool to augment your design. If you are unfamiliar with the tool, you can simply describe your design to the chat bot and ask it to generate a starter diagram for you to modify...
HTTP client
Rate limiter service
Configuration storage
Destination service
Explain how the request flows from end to end in your high level design. Also you could draw a sequence diagram using the diagramming tool to enhance your explanation...
client sends an HTTP request
load balancer distributes requests across our api gateways (which contain the rate limiter)
rate limiter received the request
based on the rate limiters configuration, and rate limiting algorithm
Dig deeper into 2-3 components and explain in detail how they work. For example, how well does each component scale? Any relevant algorithm or data structure you like to use for a component? Also you could draw a diagram using the diagramming tool to enhance your design...
rate limiter:
rate limiter store:
Explain any trade offs you have made and why you made certain tech choices...
"hard" vs "soft" rate limiter:
"hard" requires a performance hit, but ensure that the rate limit is never exceeded.
standalone service:
Pros: Gives us more control over the rate limiter, and is decoupled from our applications.
Cons: Higher latency due to extra network hop. Might require extra time to build a standalone rate limiter service.
Try to discuss as many failure scenarios/bottlenecks as possible.
Rate limiter down: a replica comes up and used in place of the failed rate limiter. Will cause a blip in availability.
Rate limiter store down: Will cause a blip in availability while we failover to a replica.
Config service down: won't affect customers, until we need to update the config. Should be auto-scalable.
What are some future improvements you would make? How would you mitigate the failure scenario(s) you described above?
Better analytics, logging, metrics, etc to view the performance of the rate limiter. Alerting as well.