System requirements

Functional:

1. we should limit the access to actual service if the user access it by a very high rate
2. User should be unlocked after a certain amount of time

Non-Functional:

1. System should be super available >99.99%
2. System should have a very low latency < 100ms
3. System should be consistent

Capacity estimation

Estimate the scale of the system you are going to design...

100M DAU -> 100k QPS

API design

1. GET /checkRateLimit/{ip_address}&{userid}&{endpoint} userId will be in jwt(nullable)

Database design

• user id
• ip address
• device id
• ...
• count

Redis come with the TTL

High-level design

client should connect to api gateway(load balancer) the before each service call it should call rate limiter; rate limiteer connect to redis

Redis should use a single leader replication

Request flows

client should connect to api gateway(load balancer) the before each service call it should call rate limiter; rate limiteer connect to redis

Detailed component design

We; will use a sliding window algorithm, for the rate limiter, the service will maintain a doulbly linked list(thread safe) each add will pop out the old entries. Rate limiter service is stateless, so we can partition it by user id/ip address

Trade offs/Tech choices

We must use a thread safe deque.

Failure scenarios/bottlenecks

in the event fail: because we use a single leader redis, if the leader node failed, another node will automatically became the leader node.

In th worst case what we lost is just a data for a few minutes.

Future improvements

We can have a snapshot for redis