System requirements

Functional:

1. Limits user based on a specific header passed in, "IP"
2. Guaranteed consistency on counts
3. We rate limit across all instances of a service

Non-Functional:

1. Low latency
2. Fails open
3. Metrics and alerts

Capacity estimation

1. 1000 req/s total
2. 5 instances of our service
   1. So each instance gets 200 requests
3. If a request takes 10ms and rate limiting takes 2ms, it's an increase by 20% so we'll need to add one more instance

API design

1. On all APIs, we require an IP header
2. We'll use a fixed window strategy
3. On each API call, the service will upsert and increment the record associated with the IP and timestamp. It will also get the record. If the number is above the threshold for that time, return 429

Database design

Use a Redis DB. Each entry is a row with primary key of "{IP address}#{minute}".

High-level design

The related service will call the redis DB directly and return 429 or continue processing the request

Request flows

Explain how the request flows from end to end in your high level design. Also you could draw a sequence diagram using the diagramming tool to enhance your explanation...

Detailed component design

Dig deeper into 2-3 components and explain in detail how they work. For example, how well does each component scale? Any relevant algorithm or data structure you like to use for a component? Also you could draw a diagram using the diagramming tool to enhance your design...

Trade offs/Tech choices

Explain any trade offs you have made and why you made certain tech choices...

Failure scenarios/bottlenecks

Try to discuss as many failure scenarios/bottlenecks as possible.

Future improvements

What are some future improvements you would make? How would you mitigate the failure scenario(s) you described above?