Requirements

Functional Requirements:

• per-user / per-API / per ip address rate limits.
• Admins can add/update/delete/view rate-limit rules via an API or console.

Non-Functional Requirements:

• Server should not crash because of an api over load.
• fast decision design to allow/deny a request.
• scalable to all services, and their api's

API Design

1.) decision check

POST : v1/rate-limit/api/{api_id}

request {user_id , ip_address}

2.) create or update

POST : v1/rate-limit/api/{api_id}

request: {total , per_user , }

3.) DELETE : v1/rate-limit/api/{api_id}

request : {}

4.) audit purpose

GET : v1/rate-limit/api/{api_id}

respose: {api_id, total_request, r_p_u, ....}

High-Level Design

flowchart TD

 B["client"];

 C{"server"};

 D["Database"];

 cdn["cdn"];

 ld["load balancer"];

 rd[("redis cache")];

 rls["rate limit service"];

 B --> cdn;

 cdn --> ld;

 ld --> rls

 rls --> rd

 rls --> C

 C --> D;

• we have a rate limiting service which is like a wrapper to all the other backend services for the rate limiting logic
• The gateway should first check incoming requests against the rate limiter before forwarding them to backend services. This can be achieved through a token bucket or leaky bucket algorithm, which allows for bursts of traffic while maintaining an overall limit. The rate limiter can be implemented as a centralized service or as a sidecar pattern, depending on your architecture.
• In scenarios where the rate limiter becomes slow or unavailable, consider implementing a fail-open policy that allows requests to pass through with a warning, or a fail-closed policy that blocks all requests until the rate limiter is restored. This ensures that your system can handle failures gracefully.
• To achieve statelessness, you can leverage shared data stores, such as Redis or a distributed cache, to maintain the current state of rate limits.

Detailed Component Design

• Using the leaky bucket algorithm
• use a centralized service with distributed caching to manage state efficiently.
• Distributed Locks
• Leverage Local Caching
• Atomic Operations