Design a Distributed Unique Id Generator

Every tweet, every database row, every distributed transaction needs a unique identifier. UUIDs solve this at 128 bits, but they waste space in database indexes and cannot be sorted by time. The challenge: generate globally unique, time-ordered IDs that fit in 64 bits, across thousands of machines, millions of times per second, with no central coordinator.

Functional

Global uniqueness across all nodes: No two IDs ever collide, even when generated on different machines at the same millisecond. A single collision means corrupted data in every system using the ID as a primary key.

The bit budget is the real capacity constraint (not storage, not bandwidth). Every bit allocated to one field is stolen from another. 41 timestamp bits at millisecond resolution cover 2 to the power of 41 milliseconds, which is roughly 69.7 years from a custom epoch. 10 worker bits allow 1,024 nodes. 12 sequence bits allow 4,096 IDs per millisecond per node. That is 4,096 times 1,024 times 1,000 which equals roughly 4.19 billion IDs per second system-wide. At 10K IDs per second baseline, we are using 0.00024% of capacity.

Traffic

Baseline: 10,000 IDs per second. Peak: 100,000 IDs per second. Per node: 4,096 IDs per millisecond equals 4.096 million IDs per second theoretical max. With 10 nodes: 40.96 million IDs per second, 400x headroom over peak.

The API is deceptively simple, one endpoint to generate an ID, one for batch. But the worker registration endpoint at startup is what makes the whole system safe. Without it, two nodes could claim the same worker ID and produce colliding IDs that look valid until data corruption surfaces weeks later.

ID Generation

Generate single ID: POST /ids

This system barely uses a database. The hot path (generating an ID) is pure in-memory computation with no I/O. The only persistent state is the worker ID registry in a coordination store, accessed once at startup, not on every request. This is what makes Snowflake fundamentally different from a ticket server: zero database writes per ID.

Worker ID Registry (ZooKeeper/etcd)

The coordination store holds worker-to-node mappings. Each entry is an ephemeral node with a lease:

The architecture has two distinct operational phases, and understanding this separation is the key insight.

Startup phase (coordinator-dependent): A new node boots, contacts the coordination store (ZooKeeper or etcd), acquires a unique worker ID with a lease, and loads the epoch and bit layout configuration. This is the only moment the node depends on an external system.

ID Generation Flow

This is the hot path; it touches no external systems.

The Snowflake 64-Bit Layout

The entire design rests on one insight: if you divide a 64-bit integer into non-overlapping fields, and each node controls a unique subset of one field, then all nodes can generate IDs independently without any runtime coordination.

Snowflake vs Ticket Server

Ticket server: A MySQL table with AUTO_INCREMENT. Simple, strictly sequential, easy to implement. But every ID is a database write. At 100K IDs per second, you need a database capable of 100K writes per second, or you shard across multiple ticket databases with different auto-increment steps (database 1 generates odd IDs, database 2 generates even). Sharding adds complexity: you need a routing layer, and IDs are no longer globally sequential (only sequential within each shard). If a shard fails, half the ID space is unavailable.

Every component in a distributed system can fail. The ID generator's key advantage is that failures are isolated, a single node crash reduces capacity but never halts the system or loses data.

Generator Node Failure

A generator node crashes. Other nodes continue without interruption. The load balancer removes the dead node within seconds. The worker ID is freed when the ZooKeeper session expires (10-30 seconds). No data loss. IDs are stateless. A replacement node acquires a new worker ID and starts immediately.

The Snowflake design handles today's requirements, but several enhancements can push it further in scale, reliability, and flexibility.

UUIDv7 / 128-Bit Option

RFC 9562 defines UUIDv7, a 128-bit, time-sortable UUID. Use UUIDv7 for external-facing IDs while keeping 64-bit Snowflake IDs internally for database efficiency. A mapping layer converts between formats at the API boundary.