Introduction

In GKE, you do not autoscale directly on raw log lines. The workable pattern is to convert logs into a logs-based metric, let Cloud Monitoring store that metric, and have the Horizontal Pod Autoscaler read it as an external signal. This can be useful, but it is slower and less direct than scaling on CPU or request rate.

The Pipeline You Actually Build

A logging-driven autoscaling setup has three parts:

1. your workload emits logs
2. Cloud Logging turns matching entries into a metric
3. HPA reads that metric through the Cloud Monitoring integration

That architecture matters because most failures happen between those stages, not in the HPA object itself.

Also note the metric type: a logs-based metric is usually treated as an external metric for HPA purposes. It is not a pod resource metric like CPU, and it is not a native Kubernetes object.

Create a Logs-Based Metric

Suppose your service emits error logs when queue backlog becomes unhealthy. A logs-based counter metric can count those events.

gcloud logging metrics create api_error_count \
  --description="Count error logs from the api container" \
  --log-filter='resource.type="k8s_container" AND resource.labels.container_name="api" AND severity>=ERROR'

After creating the metric, do not jump straight to autoscaling. First verify that matching log entries actually increase the metric in Cloud Monitoring. If the metric is flat, the HPA cannot make a meaningful decision no matter how clean the YAML looks.

Wire the Metric into HPA

A typical HPA for this pattern uses autoscaling/v2 and an external metric reference:

1apiVersion: autoscaling/v2
2kind: HorizontalPodAutoscaler
3metadata:
4  name: api-hpa
5spec:
6  scaleTargetRef:
7    apiVersion: apps/v1
8    kind: Deployment
9    name: api
10  minReplicas: 2
11  maxReplicas: 10
12  metrics:
13    - type: External
14      external:
15        metric:
16          name: logging.googleapis.com/user/api_error_count
17        target:
18          type: AverageValue
19          averageValue: "5"

The key point is that the HPA is not parsing logs. It is consuming a numeric metric that was already derived from logs.

Choose the Signal Carefully

This pattern works best when log volume is a strong operational proxy for stress. Good candidates include:

• queue overflow warnings
• overload markers emitted by the app
• repeated downstream throttling events

Poor candidates include:

• rare exception logs
• noisy application messages
• logs that arrive well after the real load spike

Because logs must be ingested, transformed, and then queried as metrics, the signal has delay. That makes logging-based autoscaling a poor choice for workloads that need near-instant reaction.

Debug the Chain in Order

When scaling does not happen, inspect the path one stage at a time:

1. confirm the workload is emitting the expected logs
2. confirm the logs-based metric increments
3. confirm Cloud Monitoring shows recent points
4. confirm the HPA references the correct metric name
5. confirm the target value is realistic

A practical command during debugging is:

kubectl describe hpa api-hpa

If the HPA shows no current value, the problem is usually metric wiring. If it shows a current value that never crosses the target, the metric may be valid but not useful for autoscaling.

When to Prefer Another Metric

If your real scaling signal is already available as request rate, queue depth, CPU, memory, or application latency, use that first. Logging-derived metrics are valuable when the only trustworthy signal is encoded in logs, but they are rarely the simplest option.

In other words, logs-based autoscaling is a specialization, not the default recommendation.

Common Pitfalls

• Expecting HPA to react to logs directly instead of through a metric pipeline.
• Creating the logs-based metric and never verifying that it actually receives points.
• Using a noisy or delayed log signal that causes unstable or late scaling.
• Pointing the HPA at the wrong metric name or wrong metric type.
• Treating a logs-based metric as the first choice when a cleaner operational metric already exists.

Summary

• In GKE, autoscaling on logs really means autoscaling on a metric derived from logs.
• The HPA usually consumes that signal as an external metric.
• Validate the logs-based metric in Cloud Monitoring before debugging the HPA.
• Choose log-derived metrics only when they are a strong proxy for load or stress.
• Expect some delay compared with direct resource or request metrics.