kubectl
memory usage
Kubernetes
performance monitoring
node metrics

Understanding memory usage for kubectl top node

Master System Design with Codemia

Enhance your system design skills with over 120 practice problems, detailed solutions, and hands-on exercises.

Introduction

kubectl top node is a fast way to inspect memory pressure in a Kubernetes cluster, but the output is often misread. The command shows current resource usage from the Kubernetes metrics pipeline, not a full operating-system memory report. To use it well, you need to understand what it measures, what it omits, and how to combine it with other node signals.

What the Command Shows

kubectl top node reads metrics from Metrics Server, which collects node and pod resource usage from the kubelet. A typical result looks like this:

bash
kubectl top node
text
NAME         CPU(cores)   CPU%   MEMORY(bytes)   MEMORY%
worker-01    850m         21%    6120Mi          78%
worker-02    420m         10%    4880Mi          62%

For memory, the command gives a current usage figure and a percentage relative to node capacity or allocatable context exposed through Kubernetes metrics. It is intended for operational triage, not for complete host accounting.

Why It Does Not Match free or top

One of the biggest sources of confusion is comparing kubectl top node with Linux tools such as free, top, or cloud-provider dashboards. Those tools may include different categories such as:

  • Page cache.
  • Buffers.
  • Kernel memory details.
  • Reclaimable memory.

Kubernetes surfaces a working-set style metric that is better suited for cluster scheduling and workload monitoring. So when the numbers differ, that does not automatically mean one source is wrong.

What MEMORY% Is Good For

MEMORY% is best used as a pressure indicator. It helps answer questions such as:

  • Which node is currently hottest.
  • Whether memory usage is balanced across the cluster.
  • Whether a node is approaching risky levels over time.

It does not, by itself, prove a node will be OOM-killed or evict pods immediately. Real risk appears when high memory usage combines with eviction events, OOM kills, or sustained imbalance between nodes.

Compare Node and Pod Views Together

Node-level memory becomes much more useful when paired with pod-level metrics.

bash
kubectl top pod -A --sort-by=memory

This helps determine whether memory pressure comes from:

  • One oversized workload.
  • Many medium workloads.
  • System components such as DaemonSets.
  • Host overhead outside the obvious application pods.

If node memory looks high but pod usage does not explain it, inspect system pods and node conditions next.

A Practical Investigation Workflow

When a node looks memory-heavy, use a layered check:

  1. Inspect node usage with kubectl top node.
  2. Inspect top memory-consuming pods.
  3. Describe the node for pressure conditions.
  4. Check cluster events for OOM or eviction messages.

Useful commands:

bash
1kubectl top node
2kubectl top pod -A --sort-by=memory
3kubectl describe node worker-01
4kubectl get events -A

This sequence is much more effective than reacting to one percentage in isolation.

Requests, Limits, and Actual Usage Are Different

kubectl top node shows measured usage, not resource requests or limits. These are different concepts:

  • Requests affect scheduling.
  • Limits affect cgroup enforcement.
  • Top shows observed current consumption.

That means a node can look fine in top but still be oversubscribed by requests, or it can look busy even when requests are low. For real capacity analysis, compare usage with requested and limited memory across workloads.

Metrics Server Is for Triage, Not History

kubectl top is intentionally lightweight. It is excellent for a quick operational snapshot, but weak for:

  • Historical trend analysis.
  • Alert tuning.
  • Capacity forecasting.
  • Regression analysis after deployments.

For those cases, use a monitoring stack such as Prometheus and Grafana. Historical memory curves are essential when you want to know whether pressure is transient or structural.

When to Treat the Number as Actionable

Investigate more deeply when you see any of these:

  • Repeatedly high memory percentage on a node.
  • Pod evictions due to memory pressure.
  • 'OOMKilled container restarts.'
  • One node consistently much hotter than peers.

Those combinations usually point to a workload-placement problem, a memory leak, or node-sizing mismatch.

Common Pitfalls

  • Treating kubectl top node as the same thing as Linux host memory accounting.
  • Reacting to high memory percentage without checking pod-level consumers.
  • Ignoring requests and limits while discussing memory pressure.
  • Using kubectl top alone for capacity planning.
  • Assuming a single snapshot is enough to diagnose a memory problem.

Summary

  • 'kubectl top node is a quick pressure signal, not a full memory forensics tool.'
  • Its memory number comes from the Kubernetes metrics pipeline and may differ from OS tools.
  • Use node metrics together with pod metrics, node describe output, and cluster events.
  • Compare measured usage with requests and limits before making capacity decisions.
  • For long-term analysis, use historical monitoring instead of one-time top output.

Course illustration
Course illustration

All Rights Reserved.