Loading...
Server-side rate limiter.
Rate limit a client to an API globally even if they are hitting different servers.
We don't need perfectly strict precision.
The rate limiter should add minimal latency to the request.
1M RPS. 1M DAU. 100k peak concurrent users.
Since it is a backend concern, gRPC is an appropriate way to call the service internally.
/checkLimit
{
"userId": int
"API": string
}
returns:
{
"ok": boolean
}
We can have a Redis for the rate limiter to use for recording the calls that a user has left in their budget for the time window.
We will have a replicated rate limiter service that will be called at the start of every API call. The service will check Redis to see how many tokens are in the bucket for a user and API.
Client sends a request to server for a given API. At the start of the API call, server calls rate limiter service to check if the rate limit has been exceeded. The rate limiter service records the request using Redis and returns the status. If the rate limit has been exceeded, the server returns an error to the client.
The rate limiter service is stateless and can serve any client. When there is a request it checks Redis to see how many tokens the user has left in their bucket for the API. If 0, return error. Tokens are refreshed in buckets by a periodic job that resets each bucket to a fixed number.
The rate limit may not be super strict due to the race condition of tokens being able to be used multiple times since the read and write of the bucket is not atomic.
If a rate limiter goes down it can just be replaced by a replica. Same with the Redis.
We could make the rate limiter configurable.