Kubernetes
Pods
Replication
Self-Healing
Container Orchestration

Kubernetes pod gets recreated when deleted

System Design practice on Codemia

Work through 120+ system design problems with detailed solutions, from rate limiters to multi-region storage.

Practice system design

As the demand for scalable and resilient applications has grown, Kubernetes has emerged as a dominant platform for container orchestration. One of the defining features of Kubernetes is its ability to manage the lifecycle of pods—collections of one or more containers—and ensure that specified states are maintained. One interesting aspect of Kubernetes is how it handles the deletion of pods. When a pod is manually deleted, Kubernetes automatically recreates it to ensure that the desired state of the application is maintained. This article dives deep into this behavior, explaining why it happens and the mechanics behind it.

Understanding Kubernetes Desired State

In Kubernetes, the "desired state" is a core concept. It refers to the specification that an operator provides to Kubernetes regarding how an application should run. This specification includes the number of pod replicas, configuration data, and other operational specifics. The desired state is specified in Kubernetes via objects like Deployments, ReplicaSets, and StatefulSets.

Example: Kubernetes Deployment

A Kubernetes Deployment is a high-level abstraction that manages a ReplicaSet to maintain a specified number of identical pods. Consider this simple Deployment configuration:

yaml
1apiVersion: apps/v1
2kind: Deployment
3metadata:
4  name: example-deployment
5spec:
6  replicas: 3
7  selector:
8    matchLabels:
9      app: example
10  template:
11    metadata:
12      labels:
13        app: example
14    spec:
15      containers:
16      - name: example-container
17        image: nginx

In this example, the example-deployment ensures that three pods running the nginx container are up and running at all times.

What Happens When a Pod is Deleted?

When a pod is manually deleted using a command such as:

bash
kubectl delete pod <pod-name>

Kubernetes' control loop detects that the current state (fewer pods) does not match the desired state (three pods, according to the Deployment). The Kubernetes controller that manages the Deployment takes corrective action by creating a new pod to align the actual state with the desired state.

The Control Loop Mechanism

Kubernetes operates using a control loop, which continuously monitors and adjusts the state of objects to match desired specifications:

  1. Monitoring: Control loops in Kubernetes constantly monitor the state of the cluster's resources.
  2. Detecting Divergences: As soon as a divergence from the desired state is detected—such as a pod being deleted—the control loop initiates actions.
  3. Correction: Kubernetes takes necessary corrective actions, such as recreating the missing pod.

This results in the pods getting automatically recreated, ensuring application reliability and availability.

Detailed Breakdown: Pods and Replicasets

The Role of ReplicaSets

A ReplicaSet in Kubernetes is responsible for maintaining the specified number of pod replicas. It is the component that directly manages the pod lifecycle:

  • Selector: Defines how to identify the pods managed by the ReplicaSet.
  • Replicas: Indicates how many pod replicas should be running.
  • Template: Specifies the pod configuration.

Pod Deletion Handling

When a pod deletion command is executed, the following occurs behind the scenes:

  • Event Triggered: The Kubernetes API server registers the deletion event.
  • State Evaluation: The ReplicaSet controller detects a deviation from the desired number of pods.
  • Pod Recreation: A new pod is scheduled on the cluster to replace the deleted one, based on the template specified in the ReplicaSet.

Key Points Table

Here is a summary of the key points discussed:

TopicDescription
Desired StateSpecification provided by operator with the required number of replicas.
Control LoopMechanism by which Kubernetes reconciles actual and desired cluster states.
DeploymentHigh-level abstraction managing pods through ReplicaSets.
ReplicaSet RoleMaintains the desired number of pod replicas using a template.
Pod Recreation TriggerOccurs when a pod is deleted, prompting the ReplicaSet to replace it.

Conclusion

Kubernetes' architecture is designed for resilience and automation, with the control loop playing a crucial role in maintaining system integrity. The automatic recreation of pods when they are deleted highlights Kubernetes’ commitment to ensuring applications remain highly available and perform consistently according to specifications. Understanding this mechanism is essential for operators and developers to effectively develop, deploy, and maintain applications on Kubernetes.

Whether you're a developer pushing updates or an operator ensuring system reliability, recognizing the underlying principles of pod recreation in Kubernetes enables a better grasp of how this powerful platform maintains continuity and stability in dynamic environments.


Related reading
Course
Beginner
27 lessons
10 hours
System Design Fundamentals

Build a strong foundation in designing scalable, reliable distributed systems.

View the course
Track what you have practised

A free account saves your progress, solutions and study plan across every problem on Codemia.

System Design practice on Codemia

Work through 120+ system design problems with detailed solutions, from rate limiters to multi-region storage.

Practice system design

All Rights Reserved.