Kubernetes
Pods
Log Files
Troubleshooting
Crash Debugging

View log files of crashed pods in kubernetes

System Design practice on Codemia

Work through 120+ system design problems with detailed solutions, from rate limiters to multi-region storage.

Practice system design

Introduction

In Kubernetes, containers may encounter various issues leading them to crash. A crashed pod can disrupt workloads, especially in production environments. This article explores how to view log files of crashed pods in Kubernetes, allowing administrators to troubleshoot and resolve issues promptly. Managing Kubernetes effectively involves understanding how to debug and extract information from these logs to maintain optimal operational states.

Understanding Pods and Containers

Kubernetes runs containers within entities called pods. A pod is the smallest deployable unit in Kubernetes, typically containing one or more containers. When a pod crashes, it's often due to issues within the containers it encapsulates, such as application failures or misconfigurations.

Viewing Logs of Crashed Pods

Viewing the log files of crashed pods involves several steps and commands. Primarily, logs are obtained from the individual containers within a pod. Here is a step-by-step guide on how to access these logs, even when the pod has exited due to crashes:

1. List Pods and Identify Crashed Ones

Start by listing the pods in your namespace to identify which ones have crashed:

bash
kubectl get pods

Check the STATUS column; pods with a CrashLoopBackOff status indicate repeated failures.

2. Examine Pod Descriptions

Before delving into logs, it can be helpful to get a detailed description of the pod:

bash
kubectl describe pod <pod-name>

This command provides insights into events, configurations, and error messages, offering clues about the cause of the crash.

3. Access Logs of Crashed Pods

For pods that have statuses like CrashLoopBackOff or Error, you can still access their logs:

bash
kubectl logs <pod-name> --previous

The --previous flag is crucial here. It retrieves logs from the previous instance of the container, which is helpful when the container has crashed and restarted.

4. Check StatefulSets, DaemonSets, and ReplicaSets

If your application is running using controllers like StatefulSets, DaemonSets, or ReplicaSets, ensure you check logs for potential volume or replica-related issues:

bash
kubectl get statefulsets
kubectl logs statefulset/<statefulset-name> -c <container-name>

Replace <statefulset-name> and <container-name> with the appropriate names you obtain by using kubectl get statefulsets.

Advanced Debugging Techniques

Apart from manual log retrieval, Kubernetes offers several advanced tools and strategies to enhance your debugging process:

  1. Elasticsearch, Fluentd, and Kibana (EFK) Stack: For a more comprehensive and centralized logging solution, integrate the EFK stack to aggregate logs from all pods and nodes.
  2. Liveness and Readiness Probes: Ensure these probes are appropriately configured to automatically restart unhealthy pods and provide insight into application health.

Table: Kubernetes Pod Logging Commands

CommandDescription
kubectl get podsLists all pods in the current namespace.
kubectl describe pod <pod-name>Displays detailed information about the pod, including configuration and events.
kubectl logs <pod-name>Fetches logs from a running pod.
kubectl logs <pod-name> --previousRetrieves logs from the last terminated instance of the container within a pod.
kubectl logs -f <pod-name>Follows log output in real-time, useful for live debugging.
kubectl get statefulsetsLists StatefulSets in the current namespace.
kubectl logs statefulset/<statefulset-name> -c <container-name>Fetches logs from a specific container in a StatefulSet.

Conclusion

Viewing and analyzing Kubernetes pod logs, particularly from crashed pods, is essential for maintaining a stable, reliable Kubernetes environment. By following the procedures outlined in this article and leveraging advanced logging strategies, administrators can efficiently troubleshoot and mitigate issues, ensuring the seamless operation of containerized applications.


Related reading
Course
Beginner
27 lessons
10 hours
System Design Fundamentals

Build a strong foundation in designing scalable, reliable distributed systems.

View the course
Track what you have practised

A free account saves your progress, solutions and study plan across every problem on Codemia.

System Design practice on Codemia

Work through 120+ system design problems with detailed solutions, from rate limiters to multi-region storage.

Practice system design

All Rights Reserved.