Loading...
Estimate the scale of the system you are going to design...
search(query) - returns list of logs that matches the query, for instance it could be period of time
getServiceMetrics(service_id) - returns list of metrics for particular service.
Defining the system data model early on will clarify how data will flow among different components of the system. Also you could draw an ER diagram using the diagramming tool to enhance your design...
Log producer - pushes logs.
Message queue - handles increasing numbers of logs requests. Kafka is the best option for logs.
Log collections service - pulls logs from Kafka , process them and store in DB.
Statistic service - pulls logs from Kafka and collects different metrics and store them in DB.
Cassandra DB - nonSql DB that stores logs and metrics.
Log search service - pulls logs from Kafka and store and handle search requests from user.
Log view service - fetches user requested data from DB service that visualize logs/statistics/notifications.
Explain how the request flows from end to end in your high level design. Also you could draw a sequence diagram using the diagramming tool to enhance your explanation...
Dig deeper into 2-3 components and explain in detail how they work. For example, how well does each component scale? Any relevant algorithm or data structure you like to use for a component? Also you could draw a diagram using the diagramming tool to enhance your design...
Explain any trade offs you have made and why you made certain tech choices...
Try to discuss as many failure scenarios/bottlenecks as possible.
What are some future improvements you would make? How would you mitigate the failure scenario(s) you described above?