Scala
Heap Dump
Kubernetes
Azure
Troubleshooting

How to do scala heap dump in Kubernetes in Azure

System Design practice on Codemia

Work through 120+ system design problems with detailed solutions, from rate limiters to multi-region storage.

Practice system design

Introduction

A Scala service running on Azure Kubernetes Service is still a JVM process, so heap-dump collection works the same way as for any Java application. The Azure-specific part is mostly about getting cluster access and making sure the dump has somewhere safe to be written, because heap dumps are large and can easily overwhelm container storage.

Get Access to the AKS Cluster and Pod

Start by connecting kubectl to the AKS cluster:

bash
az aks get-credentials --resource-group my-rg --name my-aks
kubectl get pods -n payments

Then identify the target pod and container. If the pod has more than one container, include -c <container-name> in later commands so the dump runs in the correct JVM container.

Use jcmd or jmap Inside the Container

If the image includes JDK tools, the cleanest option is jcmd:

bash
kubectl exec -n payments api-7cf6d4d5d9-abcde -- \
  jcmd 1 GC.heap_dump /tmp/api.hprof

On many containerized JVM apps, the main Java process is PID 1. If you are unsure, inspect the process list first:

bash
kubectl exec -n payments api-7cf6d4d5d9-abcde -- ps -ef

If jcmd is unavailable but jmap exists, this also works:

bash
kubectl exec -n payments api-7cf6d4d5d9-abcde -- \
  jmap -dump:live,format=b,file=/tmp/api.hprof 1

The live option reduces dump size by keeping only live objects, which can make analysis easier during production incidents.

Copy the Dump Out of the Pod

After the dump is written, copy it to your workstation:

bash
kubectl cp payments/api-7cf6d4d5d9-abcde:/tmp/api.hprof ./api.hprof

You can then open the file in Eclipse MAT, VisualVM, or another heap-analysis tool.

This is often enough for one-off debugging, but it depends on the container having enough writable space for a large .hprof file.

Use a Persistent Volume for Safer Dumps

Heap dumps can be hundreds of megabytes or larger. Writing them to /tmp inside the container is risky if the writable layer is small. A better pattern is to mount a volume dedicated to diagnostics.

Example deployment fragment:

yaml
1volumeMounts:
2  - name: heap-dumps
3    mountPath: /var/dumps
4
5volumes:
6  - name: heap-dumps
7    persistentVolumeClaim:
8      claimName: heap-dumps-pvc

Then dump to that path instead:

bash
kubectl exec -n payments api-7cf6d4d5d9-abcde -- \
  jcmd 1 GC.heap_dump /var/dumps/api.hprof

In Azure, that PVC might be backed by Azure Disk or Azure Files depending on your storage class and operational needs.

Configure Automatic Dumps on OOM

If the service is crashing with OutOfMemoryError, manual collection may be too late. In that case, configure the JVM to write a dump automatically.

yaml
1env:
2  - name: JAVA_TOOL_OPTIONS
3    value: >-
4      -XX:+HeapDumpOnOutOfMemoryError
5      -XX:HeapDumpPath=/var/dumps

This is especially useful for intermittent production failures because the dump is captured at the time of the crash rather than after a restart erased the evidence.

Common Pitfalls

  • Using a JRE-only image leaves you without jcmd or jmap. If that happens, switch to a JDK-based image for debugging or attach a debug container with the needed tools.
  • Writing the dump to container-local storage can fail when the file is larger than the writable layer.
  • Forgetting the namespace or container name leads to confusing kubectl exec errors in AKS environments with many workloads.
  • Taking a heap dump from a busy production JVM can pause the process and consume significant CPU and I/O. Schedule carefully.

Summary

  • A Scala service on AKS is still a JVM process, so heap dumps are collected with standard Java tools.
  • Use kubectl exec with jcmd or jmap, then copy the .hprof file out with kubectl cp.
  • Prefer a mounted diagnostic volume over /tmp for large dumps.
  • Enable automatic OOM heap dumps when the failure is hard to reproduce manually.

Related reading
Course
Beginner
27 lessons
10 hours
System Design Fundamentals

Build a strong foundation in designing scalable, reliable distributed systems.

View the course
Track what you have practised

A free account saves your progress, solutions and study plan across every problem on Codemia.

System Design practice on Codemia

Work through 120+ system design problems with detailed solutions, from rate limiters to multi-region storage.

Practice system design

All Rights Reserved.