High-Level Design

Client - the external clients that are requesting IDs
Load Balancer - routes the requests between the nodes to split the load and allocates node IDs to them. It uses simple round-robin since generators are stateless.
ID Generator instances - stateless services which generate the IDs locally

As seen on the diagram, there is no database.

Client calls POST /ids
Load balancer routes to any generator
The generator
1. reads the current time
2. increments the local sequence
3. assembles ID
IDs are returned to the client

There are no database calls.

Generator instance crash
1. the Load Balancer re-routes the traffic
2. uniqueness is preserved as no two live instances share the node id
Clock goes backwards on a node
1. generator does not generate IDs until time catches up
2. alert logged/emitted
3. throughput impacted locally
Generator sequence exhaustion
1. if the generator exhausts its per ms sequence then it blocks until the next ms
2. additionally batch size is capped to sequence size and for batch requests the generator will wait until the next ms so that it fits

The IDs are not stored.

The Load Balancer stores in memory mappings for (instance, node_id).

Per node: thousands of IDs/sec easily.

We can scale horizontally by adding generators.

The IDs are not strictly globally ordered but only time-ordered.

This trades perfect ordering for minimal synchronisation overhead.

We could: