System requirements
Functional:
List functional requirements for the system (Ask the chat bot for hints if stuck.)...
1) System should be able to schedule tasks based on priorities.
2) Higher priorities tasks should be schedule first than lower priorities
Non-Functional:
List non-functional requirements for the system...
1) System should be highly available
2) System should be scalable
3) System should be fault tolorent
Capacity estimation
Estimate the scale of the system you are going to design...
As discussed with the interviewer qps is 1000 to 10,000
API design
Define what APIs are expected from the system...
We will have only one api to submit the tasks the api will also recieve a request body that will contain the priority of the tasks,tasksId, task StartDate,Tasks End Date
Database design
Defining the system data model early on will clarify how data will flow among different components of the system. Also you could draw an ER diagram using the diagramming tool to enhance your design...
We will be using two databases for the design of tasks scheduler that is the relational and graph database
1) Relational database is used to store the metadata of each tasks that is the task id,priority,task Id,
task StartTime, task EndTime
2) Graph DataBase - We will use graaph database to store all the relations among the various tasks.
High-level design
You should identify enough components that are needed to solve the actual problem from end to end. Also remember to draw a block diagram using the diagramming tool to augment your design. If you are unfamiliar with the tool, you can simply describe your design to the chat bot and ask it to generate a starter diagram for you to modify...
User makes a request that goes to the rate limiter from the rate limiter the request is forwarded to sequence generator where each task is assigned a unique ID then the tasks is stored into relational databases if a tasks has a dependency with a particular tasks then the tasks will also be stored in graph database after adding entry in the database tasks are published to the queue the queue manager selects a task and assign a resource manager for that tasks. The resource manager assigns resources and the task is executed.
Request flows
Explain how the request flows from end to end in your high level design. Also you could draw a sequence diagram using the diagramming tool to enhance your explanation...
User makes a request that goes to the rate limiter from the rate limiter the request is forwarded to sequence generator where each task is assigned a unique ID then the tasks is stored into relational databases if a tasks has a dependency with a particular tasks then the tasks will also be stored in graph database after adding entry in the database tasks are published to the queue the queue manager selects a task and assign a resource manager for that tasks. The resource manager assigns resources and the task is executed.
Detailed component design
Dig deeper into 2-3 components and explain in detail how they work. For example, how well does each component scale? Any relevant algorithm or data structure you like to use for a component? Also you could draw a diagram using the diagramming tool to enhance your design...
User makes a request that goes to the rate limiter from the rate limiter the request is forwarded to sequence generator where each task is assigned a unique ID then the tasks is stored into relational databases if a tasks has a dependency with a particular tasks then the tasks will also be stored in graph database after adding entry in the database tasks are published to the queue the queue manager selects a task and assign a resource manager for that tasks. The resource manager assigns resources and the task is executed.
Trade offs/Tech choices
Explain any trade offs you have made and why you made certain tech choices...
1) Using graph database will help us to check the dependency for all the tasks
Failure scenarios/bottlenecks
Try to discuss as many failure scenarios/bottlenecks as possible.
1) If the tasks execution queue fails then the tasks will not be able to execute to fix this issue we can maintain a write ahead log that will write all the takss details .
Future improvements
What are some future improvements you would make? How would you mitigate the failure scenario(s) you described above?