System requirements

Functional:

1. System should act as container orchestrator. Which also includes Manging lifecycle of container like
2. Scheduling contianer to specific nodes
3. Creation of the container
4. Pre-emption of container.
5. Assigning container to a network
6. Attaching/detaching volumes to container.
7. Load balancing between container
8. Self healing

1. System provide API's to deploy, scale and manager other life cycle of container accross hosts part of cluster.
2. Workload deployment should follow microservice architecture
3. Monitoring capablities
4. Container resource metrics collection

1. Integration with CI/CD

Non-Functional:

List non-functional requirements for the system...

1. HA
2. Resilient
3. Scalable
4. secure
5. performant

Capacity estimation

Estimate the scale of the system you are going to design...

1. How many host orchestrator should handle.
2. How many containers expected to run in cluster.
3. How many private subnet needed and their size
4. How much/many volumes expected to be attached.
5. How many container CRUD call/sec
6. Latency and throughput req of API's

API design

Define what APIs are expected from the system...

POST v1/container/create

Payload:{

container_name:"", image:"", command:"",env:{}, volumes:{},network:""}

GET v1/container/:container_name

Response:[{

container_name:"", image:"", command:"",env:{}, volumes:{},network:""}]

DELETE v1/container/container_name

PATCH v1/container/update

Payload:{

container_name:"", image:"", command:"",env:{}, volumes:{},network:""}

GET v1/container/list

Response:{

container_name:"", image:"", command:"",env:{}, volumes:{},network:""}

Database design

Defining the system data model early on will clarify how data will flow among different components of the system. Also you could draw an ER diagram using the diagramming tool to enhance your design...

I need database for configuration storage

1. Highly available - To avoid down time of orchestrator
2. Replicated
3. Horizontally scalable - To scale based on load increase
4. Sharded
5. Multiple version copy: since it store configuration i need to keep older few version intact.
6. NOSQL type of DB.
7. Reads should be faster i expect reads to be more than writes in this case.
8. Key value store kind of DB (key is the uuid of the object i am storing)

High-level design

You should identify enough components that are needed to solve the actual problem from end to end. Also remember to draw a block diagram using the diagramming tool to augment your design...

Request flows

Explain how the request flows from end to end in your high level design. Also you could draw a sequence diagram using the diagramming tool to enhance your explanation...

Detailed component design

Dig deeper into 2-3 components and explain in detail how they work. For example, how well does each component scale? Any relevant algorithm or data structure you like to use for a component? Also you could draw a diagram using the diagramming tool to enhance your design...

API server

1. API Server : Is a bunch of API's behind authentication and authorisation boundary
2. API server exposes bunch of API's for CRUD operation on to create container.
3. API server request handler receive the request and persist in the DB.

Scheduler

1. Scheduler expose some API to take input from controller
2. Scheduler acts as a client to Agent running in host machine.
3. Scheduler constantly connect to host agent and get the status of the running container in that.
4. Scheduler maintain the location/host where agent is scheduled.

Agent

1. Agent expect Container runtime to be available on host
2. Agent contact CRI to create a container.
3. Agent will constantly monitor the container and expose API to fetch the status of it.

Trade offs/Tech choices

Explain any trade offs you have made and why you made certain tech choices...

1. Sharded, replicated NOSQL key value store chosen but we need to handle concurrent request to avoid race condition. Single instance of scheduler is used but some os process created to handle it if scheduler fail.

Failure scenarios/bottlenecks

Try to discuss as many failure scenarios/bottlenecks as possible.

1. If scheduler , controller manager fail entier system is down

Future improvements

What are some future improvements you would make? How would you mitigate the failure scenario(s) you described above?

1. Data store in DB should show transactional nature so that scheduler can scale horizontally.