View log files of crashed pods in kubernetes
System Design practice on Codemia
Work through 120+ system design problems with detailed solutions, from rate limiters to multi-region storage.
Introduction
In Kubernetes, containers may encounter various issues leading them to crash. A crashed pod can disrupt workloads, especially in production environments. This article explores how to view log files of crashed pods in Kubernetes, allowing administrators to troubleshoot and resolve issues promptly. Managing Kubernetes effectively involves understanding how to debug and extract information from these logs to maintain optimal operational states.
Understanding Pods and Containers
Kubernetes runs containers within entities called pods. A pod is the smallest deployable unit in Kubernetes, typically containing one or more containers. When a pod crashes, it's often due to issues within the containers it encapsulates, such as application failures or misconfigurations.
Viewing Logs of Crashed Pods
Viewing the log files of crashed pods involves several steps and commands. Primarily, logs are obtained from the individual containers within a pod. Here is a step-by-step guide on how to access these logs, even when the pod has exited due to crashes:
1. List Pods and Identify Crashed Ones
Start by listing the pods in your namespace to identify which ones have crashed:
Check the STATUS column; pods with a CrashLoopBackOff status indicate repeated failures.
2. Examine Pod Descriptions
Before delving into logs, it can be helpful to get a detailed description of the pod:
This command provides insights into events, configurations, and error messages, offering clues about the cause of the crash.
3. Access Logs of Crashed Pods
For pods that have statuses like CrashLoopBackOff or Error, you can still access their logs:
The --previous flag is crucial here. It retrieves logs from the previous instance of the container, which is helpful when the container has crashed and restarted.
4. Check StatefulSets, DaemonSets, and ReplicaSets
If your application is running using controllers like StatefulSets, DaemonSets, or ReplicaSets, ensure you check logs for potential volume or replica-related issues:
Replace <statefulset-name> and <container-name> with the appropriate names you obtain by using kubectl get statefulsets.
Advanced Debugging Techniques
Apart from manual log retrieval, Kubernetes offers several advanced tools and strategies to enhance your debugging process:
- Elasticsearch, Fluentd, and Kibana (EFK) Stack: For a more comprehensive and centralized logging solution, integrate the EFK stack to aggregate logs from all pods and nodes.
- Liveness and Readiness Probes: Ensure these probes are appropriately configured to automatically restart unhealthy pods and provide insight into application health.
Table: Kubernetes Pod Logging Commands
| Command | Description |
kubectl get pods | Lists all pods in the current namespace. |
kubectl describe pod <pod-name> | Displays detailed information about the pod, including configuration and events. |
kubectl logs <pod-name> | Fetches logs from a running pod. |
kubectl logs <pod-name> --previous | Retrieves logs from the last terminated instance of the container within a pod. |
kubectl logs -f <pod-name> | Follows log output in real-time, useful for live debugging. |
kubectl get statefulsets | Lists StatefulSets in the current namespace. |
kubectl logs statefulset/<statefulset-name> -c <container-name> | Fetches logs from a specific container in a StatefulSet. |
Conclusion
Viewing and analyzing Kubernetes pod logs, particularly from crashed pods, is essential for maintaining a stable, reliable Kubernetes environment. By following the procedures outlined in this article and leveraging advanced logging strategies, administrators can efficiently troubleshoot and mitigate issues, ensuring the seamless operation of containerized applications.
Related reading
- Volume is already exclusively attached to one node and can't be attached to another
- VPN access for applications running inside a shared Kubernetes cluster
- Wait for kubernetes job to complete on either failure/success using command line
- WaitForFirstConsumer PersistentVolumeClaim waiting for first consumer to be created before binding
- View not attached to window manager crash
- ''View TensorBoard'' goes to 502 Bad Gateway
- Websockets Spring boot Kubernetes
- What are all the The resourceVersion for the provided watch is too old warnings from event-exporter container?

System Design Fundamentals
Build a strong foundation in designing scalable, reliable distributed systems.
View the courseTrack what you have practised
A free account saves your progress, solutions and study plan across every problem on Codemia.
System Design practice on Codemia
Work through 120+ system design problems with detailed solutions, from rate limiters to multi-region storage.