System requirements

Functional:

List functional requirements for the system (Ask the chat bot for hints if stuck.)...

1. Admin can configure rate limit - configuration service.
2. The system will identify users based on API keys, user accounts or IP.
3. the requests will be blocked if the limit is exceeded
4. Monitoring and logging
5. The request count will be reset after the time window
6. User will be notified when they have reached the limit

Non-Functional:

List non-functional requirements for the system...

1. availability 99.9%, at most 4 hours per year downtiem
2. low latency, 50ms for real-time apps (instant messaging, gaming), 300ms for web apis. -> on average 200ms.
3. Scalability: 20% growth per year

Capacity estimation

Estimate the scale of the system you are going to design...

1. total = 1,000 users
2. growth rate: 10%, 100 more users every year
3. 30% daily active users = 300 per day
4. peak traffic: 600 per day
5. 200 requests per user = 300 * 200 = 60000 requests per day
6. request size: 2KB for GET, 5KB for POST -> 3KB on avg.
7. Total request size per day: 60,000 * 3 = 180mb
8. Avg user meta data (API keys, user ID, request_count) = 2kb
9. Total user metadata/day = 60,000 * 2kb.= 120mb
10. Total storage per year= (120 + 180) * 365 = 110 gb

API design

Define what APIs are expected from the system...

POST V1/register

POST v1/authenticate

POST v1/setRateLimit

GET v1/getRateLimit

GET v1/getToken

GET v1/getRequestCount

Database design

Defining the system data model early on will clarify how data will flow among different components of the system. Also you could draw an ER diagram using the diagramming tool to enhance your design...

High-level design

You should identify enough components that are needed to solve the actual problem from end to end. Also remember to draw a block diagram using the diagramming tool to augment your design. If you are unfamiliar with the tool, you can simply describe your design to the chat bot and ask it to generate a starter diagram for you to modify...

Request flows

Explain how the request flows from end to end in your high level design. Also you could draw a sequence diagram using the diagramming tool to enhance your explanation...

Detailed component design

Dig deeper into 2-3 components and explain in detail how they work. For example, how well does each component scale? Any relevant algorithm or data structure you like to use for a component? Also you could draw a diagram using the diagramming tool to enhance your design...

CDN: using a lambda fucntion to push data periodically from redis to CDN.

Use a redis cluster to handle

Trade offs/Tech choices

Explain any trade offs you have made and why you made certain tech choices...

Token bukect algo:

Pros: easy to implement,

Smooth traffic

Cons:

If the tokens are run out, user has to wait and it might be idea for real-time app.

Sliding window:

Pros: stricter rate limit size

Cons: higher complexity and latency since each window has to compute the total amount of requests per window

Failure scenarios/bottlenecks

Try to discuss as many failure scenarios/bottlenecks as possible.

When the Redis fails, use an in-memory counter to temporarily handle the rate limits. Once the redis is recovered, sync data to redis.

Future improvements

What are some future improvements you would make? How would you mitigate the failure scenario(s) you described above?