Requirements

Functional Requirements:

• No two IDs collide across nodes, time, or datacenters.
• Generate at least 10k IDs/sec and scale beyond via more nodes.
• Later IDs are generally larger than earlier IDs; supports chronological sorting.
• Optional API to request multiple IDs per call to amortize latency.

Non-Functional Requirements:

• List the key non-functional requirements (eg low latency, scalability, reliability, etc.)...
• low latency <10ms result
• high availability 99.99%
• horizontal scalability
• high reliability
• no single point of failure
• needs to be 64 bit.

Capacity Estimation

Read throughput: 50K reads per second.

Write throughput: 100K writes per second.

Read-to-write ratio: 1:2. This is important. A 1:2 ratio is much more write-heavy than typical web applications (which are often 100:1). This tells us the storage engine must be optimized for writes.

Average value size: 64 bit.

Raw write bandwidth: 100K writes/sec x 64 bit = 6.4 MB/s to a single replica.

With replication factor 3: Every write goes to 3 replicas, so cluster-wide write throughput is 300,000 writes/sec, consuming approximately 2.4 MB/s of aggregate disk I/O. Modern NVMe SSDs handle 500+ MB/s sequential writes, so this is comfortably achievable.

Storage growth

API Design

Define the APIs expected from the system. This is your chance to analyze and define the read and write paths so that you can come up with the high-level design...

POST /api/v1/generateID

POST /api/v1/generateBatchID?count={}

GET /api/v1/getID

DELETE /api/v1/deleteID

GET /api/v1/workerID - uses consistent hashing to get the workerid

High-Level Design

Describe the overall system architecture. Identify the main components needed to solve the problem end-to-end. Use the diagramming tool to create a block diagram.

READ PATH: Client -> CDN -> API Gateway -> Load Balancer -> Retrieval Service -> Rabbit MQ -> Redis Cluster -> DB Layer

WRITE PATH: Client -> CDN -> API Gateway -> Load Balancer -> Generate Service -> Rabbit MQ -> Redis Cluster -> DB Layer

We will be using consistent hashing to assign the cluster which will be generating the unique ID. At the same time a gossip protocol is sent to other clusters saying so and so cluster is now generating the ids.

Time synchronization is a dependency of the ID generation system. All nodes synchronize their clocks using NTP against approved time sources, and clock drift is continuously monitored. The gossip protocol is used for node membership, health monitoring, and dissemination of node identifiers, but it is not relied upon for clock synchronization.

To maintain ID uniqueness and monotonic ordering, the ID generator detects clock regressions. If a node observes its local clock moving backwards, it temporarily stops issuing IDs until the clock catches up to the last observed timestamp. During this period, requests can either be retried or redirected to healthy nodes. This prevents duplicate or out-of-order IDs caused by time drift.

Operationally, NTP offset and synchronization status are monitored, with alerts generated if clock skew exceeds defined thresholds. The system is designed to tolerate normal NTP adjustments while preserving globally unique ID generation across all nodes.

Generate Service handles the generation of the ids. It can generate a single id or it can generate ids in bulk as well by taking in the parameter from the user.

Retrieval service handles retrieving the generated ids. This is mainly a read api.

Delete service handles the deletion of the ids.

Cleanup service handles the cleanup of the cache in redis cluster and cdn to keep them updated.

Database Design

Define the data model. Identify the main entities, their attributes, and relationships. Consider the choice of database type (SQL vs NoSQL) and justify your decision based on access patterns...

We will be using DynamoDB to store the generated Unique IDs.

Table:

Id - int64

UniqueID - varchar(64)

Postgres will be used to store the metadata of the unique ID

ID - int64

CreatedAt - timestamp

Detailed Component Design

Deep dive into 2-3 key components. Explain how they work, how they scale, discuss tradeoffs, capacity, and any relevant algorithms or data structures.

Generate service generates the ids.

Retrieval service retrieves the ids that are generated.

Delete service deletes the ids.

We will be using idempotency keys to make sure duplicate writes are avoided.

Dynamo DB is used because it is a key value store and can retrieve in double digit milliseconds which is what we are aiming for.

We are using postgres to store the metadata of the ids that are generated.

We will be using Rabbit MQ to decouple the components so that there is not single point of failure. Also we will be using atomic operations and mutex to make sure thread safety is present.

We will be using consistent hashing to assign unique worker ids.