Introduction

Kubernetes does not evaluate variables inside YAML manifests by itself. If you want a Service annotation value to change dynamically, you either generate the manifest before applying it, patch the Service after creation, or let a controller update it. The right answer depends on when the value becomes known: deploy time or runtime.

Dynamic at Deploy Time Means Templating

If the annotation value comes from CI, Helm values, or environment-specific configuration, render it into the manifest before calling kubectl apply.

A Helm example:

1apiVersion: v1
2kind: Service
3metadata:
4  name: my-service
5  annotations:
6    external-dns.alpha.kubernetes.io/hostname: {{ .Values.hostname | quote }}
7spec:
8  selector:
9    app: my-app
10  ports:
11    - port: 80
12      targetPort: 8080

Then provide the value at deploy time:

helm upgrade --install my-app ./chart --set hostname=api.example.com

This is the cleanest answer when the value is known during deployment.

Patch the Service After Creation

If the value is discovered later, patch the Service with kubectl annotate or a JSON merge patch.

kubectl annotate service my-service example.com/build-id=build-742 --overwrite

Or with a patch:

kubectl patch service my-service \
  --type merge \
  -p '{"metadata":{"annotations":{"example.com/build-id":"build-742"}}}'

This is useful in scripts, controllers, or post-deploy automation.

Kubernetes Itself Does Not Substitute Shell Variables in YAML

A manifest like this will not magically read your shell environment:

metadata:
  annotations:
    example.com/build-id: ${BUILD_ID}

Kubernetes stores that literal string unless a tool such as Helm, envsubst, Kustomize replacements, or your CI system renders the value first.

A simple shell-based deployment example is:

export BUILD_ID=build-742
envsubst < service.yaml | kubectl apply -f -

That is dynamic from the deployment pipeline's point of view, but it is still template rendering before the object reaches the API server.

Runtime Updates Usually Belong to a Controller

If the annotation should change automatically in response to cluster state, a controller or operator is usually the right tool. For example, a custom controller can watch Services and update annotations based on cloud information, DNS state, or external metadata.

That is genuinely dynamic runtime behavior. It is different from templating.

Be Careful with GitOps and Helm Ownership

If a Service is managed by Helm, Argo CD, or Flux, an imperative kubectl annotate command may be overwritten on the next reconciliation. In those environments, the durable fix is usually to change the source template or the controller logic rather than patching the live object by hand.

This is one of the most common reasons “dynamic” annotation updates seem to disappear.

Choose Annotations Only for Metadata

Annotations are the right place for controller hints, external integration settings, and descriptive metadata. They are not a substitute for fields that already exist in the Service spec. If the value changes core Service behavior and Kubernetes already has a proper field for it, prefer the real field over an annotation.

That keeps manifests easier to understand and reduces controller-specific coupling.

Common Pitfalls

• Expecting raw Kubernetes YAML to evaluate shell variables on its own.
• Hardcoding dynamic values that should come from deployment inputs.
• Patching live Services manually when a template or controller would be more repeatable.
• Letting an imperative patch fight with Helm or GitOps ownership of the same field.
• Calling a value “dynamic” when it is really just a deploy-time template parameter.

Summary

• Kubernetes does not perform variable substitution inside Service annotations by itself.
• Use Helm, Kustomize, envsubst, or CI templating when the value is known at deploy time.
• Use kubectl annotate or kubectl patch for post-creation changes.
• Use a controller when annotations must change automatically at runtime.
• Pick one ownership model so your annotation values do not drift between tools.