System requirements

Functional:

List functional requirements for the system (Ask the chat bot for hints if stuck.)...

Non-Functional:

Capacity estimation

1. Event Rate:
   • Given: The system must handle up to 1 million events per second.
   • Calculate the daily event ingest rate: 1 million events/second * 60 seconds/minute * 60 minutes/hour * 24 hours/day = 86.4 billion events per day.
2. Event Size:
   • Assume each event includes:
     • event_id: 16 bytes (UUID)
     • event_type: 4 bytes (integer or short string)
     • description: Assume average 128 bytes (could vary)
     • start_time, end_time, and arrival_time: Assume 8 bytes each (typically stored as 64-bit timestamps)
   • Total approximate size per event: 16 + 4 + 128 + 3 * 8 = 172 bytes
3. Daily Data Storage Requirement:
   • Total daily data: 86.4 billion events/day * 172 bytes/event = 14.85 TB/day
4. Retention and Historical Queries:
   • Determine how long data needs to be retained. If, for example, we need to keep data for one year:
   • Annual storage requirement: 14.85 TB/day * 365 days = 5.42 PB/year

API design

Define what APIs are expected from the system...

Database design

Store event in Nosql DB, like Cassandra.

Event table has {

event_id

description

event_type

timestamp

status

start_time

end_time

}

High-level design

• Event Producer: Publishes events to the message queue.
• Message Queue: Buffers the events, providing fault tolerance, absorbing spikes, and enabling asynchronous processing.
• Event Processor: Consumes and processes events from the queue, updating the Events Database.

Request flows

Explain how the request flows from end to end in your high level design. Also you could draw a sequence diagram using the diagramming tool to enhance your explanation...

Detailed component design

Dig deeper into 2-3 components and explain in detail how they work. For example, how well does each component scale? Any relevant algorithm or data structure you like to use for a component? Also you could draw a diagram using the diagramming tool to enhance your design...

Trade offs/Tech choices

Explain any trade offs you have made and why you made certain tech choices...

Failure scenarios/bottlenecks

Try to discuss as many failure scenarios/bottlenecks as possible.

Future improvements

What are some future improvements you would make? How would you mitigate the failure scenario(s) you described above?