Knative
scaling
custom metrics
cloud-native
Kubernetes

How to scale knative service based on custom metrics?

Master System Design with Codemia

Enhance your system design skills with over 120 practice problems, detailed solutions, and hands-on exercises.

Introduction

Knative does not use arbitrary custom metrics through its default request-based autoscaler. If you want a Knative Service to scale from a custom metric, the practical path is to use the HPA autoscaler class and expose that metric through the Kubernetes metrics pipeline.

Know the Two Autoscaler Modes

Knative Serving has two main autoscaler classes:

  • 'kpa.autoscaling.knative.dev, which is the default and focuses on concurrency or rps'
  • 'hpa.autoscaling.knative.dev, which delegates scaling decisions to Kubernetes HPA logic'

According to the current Knative Serving docs, custom metrics are supported on revisions that use the HPA class. That means the first configuration step is switching the service away from the default KPA mode.

Configure the Knative Service for HPA-Based Scaling

At the Knative Service level, set the autoscaler class, the metric name, and the target value:

yaml
1apiVersion: serving.knative.dev/v1
2kind: Service
3metadata:
4  name: queue-worker
5  namespace: default
6spec:
7  template:
8    metadata:
9      annotations:
10        autoscaling.knative.dev/class: "hpa.autoscaling.knative.dev"
11        autoscaling.knative.dev/metric: "queue_depth"
12        autoscaling.knative.dev/target: "30"
13        autoscaling.knative.dev/min-scale: "1"
14        autoscaling.knative.dev/max-scale: "20"
15    spec:
16      containers:
17        - image: ghcr.io/example/queue-worker:latest
18          ports:
19            - containerPort: 8080

This tells Knative that the revision should be scaled through the HPA path and that the target metric is named queue_depth.

Make the Custom Metric Visible to Kubernetes

The autoscaler cannot use a metric that Kubernetes does not know how to read. In most clusters, that means:

  1. your application exposes a per-pod metric, often through Prometheus
  2. Prometheus scrapes it
  3. a metrics adapter such as Prometheus Adapter publishes it through the custom or external metrics API

A typical Prometheus Adapter rule looks like this:

yaml
1rules:
2  - seriesQuery: 'queue_depth{namespace!="",pod!=""}'
3    resources:
4      overrides:
5        namespace:
6          resource: "namespace"
7        pod:
8          resource: "pod"
9    name:
10      as: "queue_depth"
11    metricsQuery: 'avg(queue_depth{<<.LabelMatchers>>}) by (<<.GroupBy>>)'

The exact adapter configuration depends on your metric names and labels, but the important part is that the resulting metric name matches the value you put in autoscaling.knative.dev/metric.

You can verify that Kubernetes sees the metric before debugging Knative:

bash
kubectl get --raw /apis/custom.metrics.k8s.io/v1beta2 | jq

If the metric is missing there, Knative will not be able to scale from it.

Design the Metric for Autoscaling

Custom autoscaling works best when the metric represents pressure that should be averaged across pods. Queue depth, in-flight jobs, or outstanding messages per worker are common examples. Metrics that jump wildly or are not clearly tied to per-pod load usually make scaling unstable.

Another important constraint is that HPA-based Knative revisions do not behave like request-driven KPA services. In particular, scale-to-zero behavior is not the same, so you should decide whether custom metrics are worth that tradeoff for the service you are deploying.

Debugging the Setup

Once the service is deployed, inspect the Knative Service and the generated HPA objects:

bash
kubectl get ksvc queue-worker -o yaml
kubectl get hpa
kubectl describe hpa

If scaling is not happening, check these in order:

  • the service annotations
  • metric visibility through the Kubernetes metrics API
  • Prometheus scrape labels
  • HPA events and target values

That order is more efficient than starting with application logs.

Common Pitfalls

  • Leaving the service on the default KPA class and expecting arbitrary custom metrics to work.
  • Choosing a metric name in Knative that does not match the name exported by the metrics adapter.
  • Using a cluster-level total instead of a per-pod metric, which makes scaling math misleading.
  • Forgetting that HPA-based scaling changes the scale-to-zero story compared with pure request-driven Knative.
  • Debugging Knative first when the real problem is that the custom metric never reached the Kubernetes metrics API.

Summary

  • Custom metric scaling in Knative requires the HPA autoscaler class, not the default KPA path.
  • Set autoscaling.knative.dev/class, metric, and target on the revision template.
  • Expose the metric through Prometheus and a Kubernetes metrics adapter.
  • Verify the metric in the Kubernetes metrics API before expecting Knative to react to it.
  • Choose stable, per-pod load metrics so scaling decisions stay predictable.

Course illustration
Course illustration

All Rights Reserved.