Requirements

Functional Requirements:

List the key functional requirements for the system (Ask the AI for hints if stuck)...

1. The system should block the request from the user if the threshold is reached, i.e., per user per endpoint breach.
2. The system should unblock the request from the user after the request comes back below threshold.
3. Limits must be configurable by admins.

Non-Functional Requirements:

List the key non-functional requirements (performance, scalability, reliability, etc.)...

1. The system should be reliable enough to handle the hight traffic of request and tolerant enough to handle system crashes.
2. The system should be very fast to limit or pass through the api calls.

Capacity Estimation

Estimate the scale of the system. Consider daily active users, read/write ratio, storage requirements, bandwidth, and any relevant QPS calculations...

The rate limiter should handle millions request per second.

The peak traffic is 2 times average traffic.

1 M users * 1000 limit rules ~ 32 GB data across cluster.

rule changes are rare <10 writes per sec.

API Design

Define the APIs expected from the system. This is your chance to analyze and define the read and write paths so that you can come up with the high-level design...

POST /check

request body - userID, API Key, timestamp.

This returns 429 with retry-after header for user breaching threshold

POST /rate_limits - create a rule

PUT /rate_limits/{id} - update a rule

GET /rate_limits/{id} - get rule

High-Level Design

Describe the overall system architecture. Identify the main components needed to solve the problem end-to-end. Use the diagramming tool to create a block diagram.

API Gateway - For authentication/authorization with OAuth/JWT. This also works as circuit breaker and stop the failure early.

Rate Limiter Service- The flow comes from API Gateway to this centralized service. This service uses Redis, which stores IP address and timestamp in key value format. Uses sliding window rate limiter approach. This will be scaled up based on the traffic, we will have multiple stateless nodes of this service.

In case of outage, fail-open with a conservative local token-bucket window.

There will be separate redis cache which will include the config rules as well. This will be stored in the Rate Limiter Service and every time there is config changes, the cache will be invalidated.

The redis cache will also include sharding.

Database - which stores user and ip address tables and config rules as well.

Database Design

Define the data model. Identify the main entities, their attributes, and relationships. Consider the choice of database type (SQL vs NoSQL) and justify your decision based on access patterns...

Detailed Component Design

Deep dive into 2-3 key components. Explain how they work, how they scale, discuss tradeoffs, capacity, and any relevant algorithms or data structures.

This design will be using sliding window algorithm and Redis data structures uses fixed counter with data type Strings.

When Redis encounters a network partition or a crash, high-availability rate limiters automatically switch to a local fallback window.

Synchronization is straightforward. Instances invoke the INCRBY command against the target window key using their accumulated fallback counts. This merges distributed metrics additively.