System requirements

Functional:

• Users can deploy their container applications into orchestration system
• System can automatically detect the available node to deploy the container
• System can monitor the state of containers and ensure they are matched with predefined state
• System support multiple type of workloads, including stateless application, stateful application, once off jobs and recurring jobs
• Containers are deployed with replication, our system handles the routing and traffic balancing among them

Non-Functional:

• High availability with automatic failover control
• System can scale horizontally by adding or removing nodes
• Containers can scale horizontally by adjusting replication number on the fly
• Strong security and zero data loss

Capacity estimation

• Assume the average incoming request is 10,000 TPS, the peak traffic is double the average which is 20,000 TPS.
• Assume each container consumes 0.5 core and 500 MB memory, and maximum there are 1 million containers running at the same time, then in total resources consumed are 500,000 cores and 500,000 GB memory.
• Assume each VM node has 4 cores and 32 GB, so in total we need 125,000 VMs in the cluster
• To achieve high availability and fault tolerance, we need to reserve additional resources. Therefore in total we need 250,000 VMs

API design

• POST /v1/clusters/create
• request parameters:
• cluster name
• node details
• response
• clusterId
• GET /v1/clusters/{:clusterId}
• response
• cluster details
• DELET /v1/clusters/{:clusterId}
• response
• success or fail
• POST (PUT) /v1/clusters/{:clusterId}/deployment
• request parameters
• container image location
• desired number
• application type (stateless, stateful)
• deployment type (eg. rolling update)
• response
• deploymentId
• status
• GET /v1/clusters/{:clusterId}/deployment/{:deploymentId}
• response:
• deployment details
• state
• DELET /v1/clusters/{:clusterId}/deployment/{:deploymentId}
• response:
• success or fail

Database design

For application deployment, we can have a table called Deployment, with the properties like the following:

• (PK) deploymentId
• clusterId
• deployment details
• status
• last modified

Since the deployments volume can be huge, we choose a Non-SQL database, like DynamoDB or Cassandra, for the better horizontal scalability

For cluster info, we need to in favour for consistency over availability in the situation of network partition. Therefore, we can choose a strong consistency metadata database, like Zookeeper or etcd.

• Cluster table
• (PK) clusterId
• master nodes
• worker nodes
• status
• created at
• Node table
• (PK) nodeId
• node type
• containers
• status

High-level design

Overall the systems conform Master-Slave architecture

Control plane (Master node)

• Master node takes the request from external users and coordinate with worker nodes to distribute the container deployments
• It contains the following components
• API-Server
• Scheduler
• Controller Manager
• To achieve high availability, we configure standby secondary master node to achieve automatic failover switch in case of system failure

Data plane (Worker node)

• Worker node accept commands given by master and deploy corresponding containers as requested
• It contains the following components
• container runtime
• deployment agent
• proxy agent

Request flows

Explain how the request flows from end to end in your high level design. Also you could draw a sequence diagram using the diagramming tool to enhance your explanation...

Detailed component design

Dig deeper into 2-3 components and explain in detail how they work. For example, how well does each component scale? Any relevant algorithm or data structure you like to use for a component? Also you could draw a diagram using the diagramming tool to enhance your design...

Trade offs/Tech choices

Explain any trade offs you have made and why you made certain tech choices...

Failure scenarios/bottlenecks

Try to discuss as many failure scenarios/bottlenecks as possible.

Future improvements

What are some future improvements you would make? How would you mitigate the failure scenario(s) you described above?