Kubernetes
kubectl
pods
troubleshooting
DevOps

Why do pods with completed status still show up in kubctl get pods?

Master System Design with Codemia

Enhance your system design skills with over 120 practice problems, detailed solutions, and hands-on exercises.

Kubernetes, a powerful container orchestration platform, provides a variety of tools and commands to manage workloads. One commonly used command-line tool is `kubectl`, primarily used to communicate with Kubernetes clusters. A common question among Kubernetes users is: "Why do pods with a 'Completed' status still appear when running `kubectl get pods`?" This article delves into the technical reasons behind this phenomenon, supplemented by examples and additional insights.

Understanding Pod Lifecycle in Kubernetes

Before diving into why 'Completed' pods appear in the output of `kubectl get pods`, it is crucial to understand the lifecycle of a pod:

  • Pending: Pod is accepted by the cluster but one or more container images have not been created.
  • Running: Pod has been bound to a node and all containers have been created. At least one container is still running.
  • Succeeded: All containers in the pod have successfully completed.
  • Failed: All containers terminated, and at least one container has exited with a failure code.
  • Unknown: Pod state is unknown due to unavailability of pod information.

Pods reach a 'Completed' state when they execute their tasks successfully and are terminated. This usually happens for batch jobs or cron jobs that perform tasks and do not need to run indefinitely.

Why Completed Pods Still Appear

  1. Historical Records: By default, Kubernetes maintains pods with 'Completed' status for visibility and for access to their logs, exit codes, and other diagnostic information. This is especially useful for debugging or auditing past jobs.
  2. Jobs and CronJobs: If pods are associated with `Job` or `CronJob` resources, they will remain in the 'Completed' state until the resource managing them is deleted or until they are manually cleaned up. This allows users to check the results of job executions.
  3. Default TTL (Time To Live): Kubernetes does not automatically delete completed or failed pods unless a TTL is set. You can configure a TTL controller to automatically clean up these resources post-completion via the `TTLAfterFinished` feature.
  4. Manual Cleanup: In absence of an automatic cleanup mechanism, completed pods will persist until they are manually deleted. Users should regularly clean up these pods to free up resources.

Managing Completed Pods

Configuring TTL for Pods

To automatically clean up completed pods, consider enabling the `TTLAfterFinished` feature:

  • name: example

Course illustration
Course illustration

All Rights Reserved.