Loading...
List functional requirements for the system (Ask the chat bot for hints if stuck.)...
1) User should be able to monitor metrics of various system
2) User should be able to use different visualization to visualize the system
3) User will be able to recieve notification
List non-functional requirements for the system...
System should be highly available
System should be highly scalable
System should be able to handle high write throughput
Estimate the scale of the system you are going to design...
As discussed with the interviewer we will be getting node metrics in every 1 second
Application Metrics 1-5 seconds
Network Metrics 1-10
Assume 10,000 nodes then total number of events =10,000*86400=0.8 million
Define what APIs are expected from the system...
/metric
request Body
{
metricName:"",
metric quantity:"",
timestamp:time,
}
response body
{
message:"sucessfully created",
code :201
}
on failure
response body{
message:"error"
code :400
}
Defining the system data model early on will clarify how data will flow among different components of the system. Also you could draw an ER diagram using the diagramming tool to enhance your design...
We will be using cassandra to store the metric details we will have the metric table which will store the metric name metric value ,timestamp and partitioning key of metric name and clustering key of timestamp
You should identify enough components that are needed to solve the actual problem from end to end. Also remember to draw a block diagram using the diagramming tool to augment your design. If you are unfamiliar with the tool, you can simply describe your design to the chat bot and ask it to generate a starter diagram for you to modify...
We have two ways of pulling the events from the monitoring system using pull or push method in pull method our data gathering service will the events from the metric system each minute but for these we need to have the ip for these servers defined we can use a centreal configuration service like apache zookeeper to manage that but the downside with this approach is our system needs to pull even if there is no change in the metric
Second way is for the server to push the metric to our monitoring service in this approach we need to define an endpoint where we will get the data from each metric as soon as there are any changes in that server but the problem with this approch is debugging is very difficult. For now we will be going by puch based approach we will be defining unique topic for each metrics where we will be publishing the events. Kafka gurantess ordering and strong durability we can have the storage define for 10 days in kafka from kafka we will be perfroming the aggregate operation on the events in a tumbing window of 10 minutes finally we will storing the flnk output in cassandra and from there we will be having visualiztion service which will be using casandra to draw the charts for specific time range
Explain how the request flows from end to end in your high level design. Also you could draw a sequence diagram using the diagramming tool to enhance your explanation...
We have two ways of pulling the events from the monitoring system using pull or push method in pull method our data gathering service will the events from the metric system each minute but for these we need to have the ip for these servers defined we can use a centreal configuration service like apache zookeeper to manage that but the downside with this approach is our system needs to pull even if there is no change in the metric
Second way is for the server to push the metric to our monitoring service in this approach we need to define an endpoint where we will get the data from each metric as soon as there are any changes in that server but the problem with this approch is debugging is very difficult. For now we will be going by puch based approach we will be defining unique topic for each metrics where we will be publishing the events. Kafka gurantess ordering and strong durability we can have the storage define for 10 days in kafka from kafka we will be perfroming the aggregate operation on the events in a tumbing window of 10 minutes finally we will storing the flnk output in cassandra and from there we will be having visualiztion service which will be using casandra to draw the charts for specific time range
Dig deeper into 2-3 components and explain in detail how they work. For example, how well does each component scale? Any relevant algorithm or data structure you like to use for a component? Also you could draw a diagram using the diagramming tool to enhance your design...
We have two ways of pulling the events from the monitoring system using pull or push method in pull method our data gathering service will the events from the metric system each minute but for these we need to have the ip for these servers defined we can use a centreal configuration service like apache zookeeper to manage that but the downside with this approach is our system needs to pull even if there is no change in the metric
Second way is for the server to push the metric to our monitoring service in this approach we need to define an endpoint where we will get the data from each metric as soon as there are any changes in that server but the problem with this approch is debugging is very difficult. For now we will be going by puch based approach we will be defining unique topic for each metrics where we will be publishing the events. Kafka gurantess ordering and strong durability we can have the storage define for 10 days in kafka from kafka we will be perfroming the aggregate operation on the events in a tumbing window of 10 minutes finally we will storing the flnk output in cassandra and from there we will be having visualiztion service which will be using casandra to draw the charts for specific time range
Explain any trade offs you have made and why you made certain tech choices...
Instead of using flink we could have used a time series database and then perfrom operations at the database end
Try to discuss as many failure scenarios/bottlenecks as possible.
1) Kafka gurantees availabity by having multiple brokers to write to
2) Using flike we can use checkpont to start from the point we have ended
3) Cassandra gurantees high availablity by replecating
What are some future improvements you would make? How would you mitigate the failure scenario(s) you described above?