Loading...
The rate limiter can be placed along with api gateway or load balancer or above of both.
Our primary api will call redis with a lua script and decide whether request is allowed or not.
We can use a token bucket algorithm to handle the api requests to handle the burst.
For database, redis is best option to get atomic completion of operations.
We will have a bucket with a capacity of 50 tokens and 5 token/sec.
Whenever an api request came , one token will be consumed.
Since we are using redis, consistency will be achieved. For scalability we have to follow sharding and separate keys in different shards based on hash.
We can use Redis to handle the bucket capacity and incrementing and decrementing. Since we are using token bucket algorithm, we need the database operations to be performed as atomic and single threaded. So Redis is the best option for that.
We can define bucket as below
capacity (bucket capacity): number
key (Redis key):
refill-token-limiter: number
lastRefillTimeInMs:
Redis Execution:
We can use Lua Script to perform the required actions to define whether the request is allowed or denied
Sample Lua Script:
GET Key
IF not exists then add key with ttl and tokens number
If exists
check tokens present in bucket , if available then decrement 1
return allow/deny
Based on allow/deny , we can either pass the request to backend service or deny it or kept in queue.