System requirements

Functional:

1. ID fits 64 bits
2. ID is numerical
3. IDs are sorted by time
4. IDs are unique
5. Generate up to 10,000 IDs per second

Non-Functional:

High available: as the ID generator is critical for the system, it must be high available.

Low latency:

Capacity estimation

Support generating up to 10,000 IDs per second

API design

getID(): returns an unique numeric ID

Database design

Defining the system data model early on will clarify how data will flow among different components of the system. Also you could draw an ER diagram using the diagramming tool to enhance your design...

High-level design

Comparing a few options:

1. multi-master replication:
2. there are K masters databases. Each one generate its own IDs by increment their base ID by k instead of 1.
3. cons:
4. hard to scale with multiple data centers
5. not go up with time
6. do not scale well when a server is added / removed
7. UUID:
8. pros:
9. easy implementation
10. scale well
11. cons:
12. not fit 64 bits
13. not numerical
14. not go up with time
15. ticket server:
16. one id generator server generating ID for multiple servers
17. cons:
18. single point of failure
19. Twitter Snowflake Approach ID generator:
20. instead of generating ID directly, we split the whole 64 bits ID into a few parts:
21. sign bit: 1 bit. It is always 0 and reserved for future use.
22. timestamp: 41 bits for milisec timestamp. 2^41 = about 69 years.
23. datacenter ID: 5 bits
24. machine ID: 5 bits
25. sequence ID: 12 bits. Every ID generated by a machine / process is increment by 1 and it is reset every second.

Request flows

Explain how the request flows from end to end in your high level design. Also you could draw a sequence diagram using the diagramming tool to enhance your explanation...

Detailed component design

Dig deeper into 2-3 components and explain in detail how they work. For example, how well does each component scale? Any relevant algorithm or data structure you like to use for a component? Also you could draw a diagram using the diagramming tool to enhance your design...

1. timestamp:
2. 41 bits.
3. timestamp is converted to decimal and convert to UTC time
4. 2^41 is about 69 years.
5. Sequence number:
6. 12 bits.
7. 2^12 = 4096. The ID generator can support up to 4096 ID generation on a machine every second.

Trade offs/Tech choices

Explain any trade offs you have made and why you made certain tech choices...

why not use primary key with auto increment attribute?

Because primary key with auto increment attribute can only work in a single server but single server can not support large data set and generating these ID across multiple servers with minimal delay is challenging.

Failure scenarios/bottlenecks

Try to discuss as many failure scenarios/bottlenecks as possible.

Future improvements

What are some future improvements you would make? How would you mitigate the failure scenario(s) you described above?

1. clock synchronization
2. high availability