Detailed component design

Control Plane components

Now let's discuss about the control plane components one by one.

API Server:

Functionality:
API Server is the central component for communication and interaction with the system.
It exposes RESTful APIs for managing clusters, containers, deployments, and other resources.
Role:
API Server handles incoming requests from users and other components.
It validates and authorizes requests based on configured authentication and authorization policies.
Various API's which are required to manage the cluster are exposed by the container, it communicates the actions to the POD Manager and waits for the response.
Scaling:
API Server can be scaled horizontally by deploying multiple instances behind a load balancer.
Load balancers distribute incoming requests among API server instances.

Scheduler

Functionality:
Scheduler is responsible for container scheduling based on resource requirements, constraints, and load balancing policies.
It evaluates resource availability on nodes and selects appropriate nodes for container placement.
Role:
Scheduler determines where to run pods (groups of one or more containers).
It optimizes resource utilization across the cluster.
Scaling:
Typically scales vertically by increasing the resources allocated to the scheduler component.
Can also be scaled horizontally by deploying multiple scheduler instances.
Algorithms and Metrics:
The scheduler employs various algorithms and metrics to determine which containers to run on which nodes. These decisions are crucial for optimizing resource utilization, maintaining high availability, and meeting performance requirements. Here are some commonly used algorithms and metrics:
Resource Requirements: The scheduler considers the resource requests and limits specified by containers when making scheduling decisions. It tries to place containers on nodes with sufficient available CPU and memory resources to meet their requirements.
Node Affinity and Anti-affinity: Provides more sophisticated rules for pod placement based on node labels, allowing users to express preferences or constraints for pod placement.
Quality of Service (QoS): Assigns priority levels to pods based on their QoS requirements (Guaranteed, Burstable, BestEffort). Pods with higher priority may be scheduled ahead of lower-priority pods.
Utilization Metrics: The scheduler may consider node resource utilization metrics (CPU, memory, disk, etc.) to identify nodes that are underutilized or overloaded and make scheduling decisions accordingly.
Container Constraints: The scheduler can consider constraints for the container, these constraints are defined by the user and can be used to influence the scheduling decision. These contraints are usually based on business requirements to improve fault tolerance, resilience and reliability.

Controller Manager

Functionality:
Controller Manager consists of various controllers responsible for managing different aspects of the system.
It monitors the state of clusters, nodes, and containers, and takes corrective actions as needed.
The correction actions can include scaling, self-healing, and replication of resources. Controller Managers sends instructions to the API server to make the necessary changes.
Role:
Ensures desired state of the system is maintained.
Manages replication, scaling, and self-healing mechanisms.
Scaling:
Similar to the API server, it can be scaled horizontally by deploying multiple instances behind a load balancer.
Each controller can also be scaled independently based on workload.

etcd

Functionality:
This components is Utilized as a distributed key-value store for storing cluster state and configuration data.
Ensures reliability and consistency of data across the cluster.
The responsibility of maintaining data in etcd primarily falls on the API Server component. The API Server is responsible for interacting with etcd to store and retrieve cluster state information. When users make requests to the Kubernetes API (e.g., creating, updating, or deleting resources), the API Server translates these requests into etcd operations and ensures the consistency of the cluster state stored in etcd.
Role:
etcd stores configuration information such as cluster settings, node status, and pod definitions.
Acts as the source of truth for the entire system.
Scaling and Replication:
etcd can be scaled horizontally by deploying multiple instances in a cluster.
Clusters can be configured with an appropriate number of etcd nodes to handle expected load and provide redundancy.
It employs a distributed consensus algorithm called Raft. Raft ensures that etcd maintains consistency and availability across multiple nodes in a cluster.
To maintain consistency, etcd requires a quorum of nodes to agree on updates before committing them. By default, etcd uses a majority quorum, ensuring that updates are replicated to a majority of nodes before they are considered committed.

Worker Node and its components

Before we start with worker nodes, lets understand about the PODs.

A POD in Kubernetes is the smallest deployable unit that represents a group of one or more containers that share the same network namespace and storage volumes and can be scheduled and managed together.
A POD encapsulates one or more containers, storage resources, and network configurations. Containers within a Pod share the same IP address and port space and can communicate with each other using localhost.
Containers within the same Pod typically co-locate tightly coupled application components that need to share resources and communicate over localhost. For example, a web server container and a sidecar container that handles logging may be colocated within the same Pod.
Containers within a Pod share the same lifecycle, and they are scheduled, started, stopped, and deleted together. They also share access to the same set of resources, such as volumes and environment variables.
PODs share the resources of the node they are scheduled on.

Container Runtime:

Functionality:
Container Runtime supports various container runtimes such as Docker, containerd, and others.
It is responsible for executing and managing containers on cluster nodes.
Role:
Container Runtime provides an environment for running containerized applications.
It manages container lifecycle including creation, execution, and termination.

Proxy Component:

Functionality:
This component runs inside each pod and helps manage communication from and to the POD.
It handles port forwarding and routing traffic to the appropriate containers.
Role:
Proxy component facilitates communication between containers within the same pod and with external services.
Enables access to services running inside the pod from outside the cluster.
Scaling:
Scales automatically with the deployment of pods.

Handling Sudden Spikes in Traffic:

Auto-scaling: we will implement auto-scaling mechanisms that dynamically adjust the number of replicas for services based on metrics such as CPU utilization, memory usage, or incoming request rates. This ensures that the system can handle sudden increases in traffic by automatically provisioning additional resources. The user can provide inputs for autoscaling based on metrics and then the Scheduling component can use that configuration to scale-up and scale down the systems.
Horizontal Pod Autoscaler (HPA): Configure HPAs to automatically scale the number of Pod replicas based on observed CPU utilization or custom metrics. This allows the system to scale out during spikes in traffic and scale in during periods of low demand.

Maintaining Consistency During Updates:

Whenever the customer wants to update the containers, then they can start the CI/CD for the container images, but to update the images in all the worker nodes takes time, below are the points considered while updating the container images.

Rolling Updates: Perform rolling updates for deployments, ensuring that updates are applied gradually to avoid downtime and maintain consistency. This involves replacing Pods one by one with new versions while ensuring that a sufficient number of healthy Pods are available at all times.
Readiness and Liveness Probes: Define readiness and liveness probes for Pods to ensure that they are ready to serve traffic and healthy before receiving traffic. This prevents traffic from being routed to Pods that are not yet ready or are experiencing issues after an update.
Rollback Mechanisms: Implement rollback mechanisms that allow for quick and automated rollback to a previous version in case of issues or failures during updates. This ensures that the system can revert to a stable state in case of unexpected behavior.

Ensuring Fault Tolerance in Geographically Distributed Clusters

To make sure our system is fault tolerant, we need to have geographically distributed clusters, along with this we would also need smart load balancers to re-route traffic to nearest cluster. We will also need to have replication system which makes sure data is replicated to all regions.

Multi-Region Deployment: Deploy clusters across multiple geographic regions to distribute workloads and improve fault tolerance. Utilize cloud providers' multi-region capabilities or deploy Kubernetes clusters across different data centers.
Replication and Data Synchronization: Replicate data and resources across geographically distributed clusters to ensure data consistency and availability. Leverage distributed databases, object storage systems, or data synchronization tools to replicate data across clusters in different regions.
Global Load Balancing: Implement global load balancing solutions that can intelligently route traffic to the nearest available cluster based on proximity and health. This ensures that users are directed to the closest and most responsive cluster while providing fault tolerance and high availability.

Self-Healing with Orchestration System

Scenario: A cluster is running a Pod consisting of multiple containers, including a web server and a database. Due to a sudden increase in traffic, the web server container experiences high CPU utilization, causing it to become unresponsive.

Sequence of Events:

Container Monitoring:

The monitoring component continuously monitors the health and resource utilization of containers within the Pod. In this scenario, the monitoring component detects that the CPU utilization of the web server container has exceeded a predefined threshold, indicating a potential issue. It determines that the web server container has failed its health checks, signaling that it is unresponsive or experiencing issues.

Event Reporting:

The monitoring component reports the detected issue to the control plane, specifically to the apiserver. This event includes information about the unhealthy container, its Pod, and the node it is running on.

Scheduler Evaluation:

The apiserver forwards the event to the scheduler component. The scheduler evaluates the current state of the cluster, considering factors such as node health, resource availability, and Pod affinity/anti-affinity rules. The scheduler schedules a new instance of the web server container as part of the same Pod on a different node in the cluster. It ensures that the Pod's affinity/anti-affinity rules are maintained and that the new node has sufficient resources to accommodate the Pod.

Pod Rescheduling:

Based on its evaluation, the scheduler determines that the unhealthy Pod needs to be rescheduled to a different node in the cluster. It selects a suitable node that meets the Pod's resource requirements and scheduling constraints.

Container Termination:

The POD manager running on the node where the unhealthy Pod is hosted receives instructions to terminate the unresponsive web server container. It gracefully shuts down the container, ensuring that any active connections are properly closed and resources are released. After terminating the unhealthy container, it notifies the apiserver that the Pod has been evicted from the node. This frees up resources on the node for other workloads.

Container Initialization:

The POD Manager on the new node pulls the required container image for the web server and initializes the new container instance. Once initialized, the container's readiness probe signals that it is ready to serve traffic. POD manager sends the information such as DNS, IP address, and other metadata to the apiserver, toroute incoming traffic to the new instance of the web server container. With the new instance of the web server container ready to handle traffic, Orchestration system directs incoming requests to the Pod's service endpoint, ensuring that the application remains available to users.

Submissions

Displaying Solutions with Scores Ranging from 1 to 10