Kubernetes
Helm
Container
Deployment
Troubleshooting

Helm upgrade doesn't pull new container

System Design practice on Codemia

Work through 120+ system design problems with detailed solutions, from rate limiters to multi-region storage.

Practice system design

Introduction

When a helm upgrade appears successful but pods keep running the old image, the problem is usually not Helm itself. Helm updates Kubernetes objects, but Kubernetes only creates a new pod when the rendered deployment changes in a way that affects the pod template. Image tags, pull policy, and rollout triggers all matter.

Understand What Helm Actually Updates

Helm renders templates into Kubernetes manifests and submits them to the cluster. If the rendered Deployment pod template is effectively unchanged, Kubernetes may not start new pods.

Typical example:

yaml
1image:
2  repository: my-registry/my-app
3  tag: latest
4  pullPolicy: IfNotPresent

If you push a new image with the same tag and run helm upgrade without changing the manifest meaningfully, nodes may continue using the cached image.

Prefer Immutable Image Tags

The best fix is to stop reusing mutable tags such as latest for deployable releases.

Use versioned or commit-based tags:

yaml
1image:
2  repository: my-registry/my-app
3  tag: "2026.03.07-abc123"
4  pullPolicy: IfNotPresent

Then deploy with:

bash
helm upgrade my-app ./chart --set image.tag=2026.03.07-abc123

When the tag changes, the pod template changes, and Kubernetes performs a rollout.

Know When imagePullPolicy Matters

imagePullPolicy affects whether the kubelet tries to fetch the image when a pod starts.

Common values:

  • 'IfNotPresent pulls only if image is not already cached'
  • 'Always asks the runtime to pull on each pod start'

If you insist on mutable tags, Always reduces the chance of stale images:

yaml
1image:
2  repository: my-registry/my-app
3  tag: latest
4  pullPolicy: Always

But this is still weaker than immutable tagging because rollout correctness now depends on node pull behavior and registry state.

Force a Deployment Rollout

If the image tag stayed the same and you need pods recreated, you must change the pod template. One common pattern is updating an annotation with a timestamp or release value.

Chart snippet:

yaml
metadata:
  annotations:
    rollme: "{{ .Release.Revision }}"

Because the pod template annotation changes on each upgrade, Kubernetes creates new pods even if the image tag is unchanged.

This is useful for config changes too, but it should not replace good image versioning.

Verify the Rendered Deployment

Before blaming the cluster, inspect what Helm actually rendered.

bash
helm upgrade my-app ./chart --dry-run --debug

Then check the live deployment:

bash
kubectl get deployment my-app -o yaml

Compare:

  • image repository and tag
  • image pull policy
  • pod template annotations

If the rendered pod template did not change, no rollout should be expected.

Watch the Actual Image on Running Pods

After upgrade, confirm what image the pods are using.

bash
kubectl get pods
kubectl describe pod <pod-name>

Look at the exact image reference and, if possible, the pulled image digest. This tells you whether the node actually refreshed the image or simply reused cache.

CI and Release Pipeline Strategy

The reliable operational pattern is:

  1. build image once
  2. tag it immutably
  3. push to registry
  4. pass that exact tag into Helm

This removes ambiguity from deployments and makes rollback straightforward. If your pipeline uses mutable tags, you are pushing complexity into runtime behavior instead of release engineering.

Common Pitfalls

  • Reusing latest and expecting Kubernetes to notice new registry content automatically.
  • Assuming helm upgrade alone forces new pod creation.
  • Using IfNotPresent with mutable image tags.
  • Not checking whether the rendered pod template actually changed.
  • Debugging Helm when the real issue is image versioning policy.

Summary

  • Helm updates manifests; Kubernetes only rolls pods when the pod template changes.
  • Immutable image tags are the safest fix for stale image problems.
  • 'imagePullPolicy: Always helps with mutable tags but is not the best long-term strategy.'
  • A rollout annotation can force pod recreation when needed.
  • Verify rendered manifests and running pod image references before troubleshooting further.

Related reading
Course
Beginner
27 lessons
10 hours
System Design Fundamentals

Build a strong foundation in designing scalable, reliable distributed systems.

View the course
Track what you have practised

A free account saves your progress, solutions and study plan across every problem on Codemia.

System Design practice on Codemia

Work through 120+ system design problems with detailed solutions, from rate limiters to multi-region storage.

Practice system design

All Rights Reserved.