Introduction

Monitoring persistent-volume usage in Kubernetes is harder than watching CPU or memory because the volume exists below the application and outside normal pod resource metrics. The practical answer is to combine storage metrics from the cluster with in-pod checks so you can see both what Kubernetes thinks the volume can hold and what the filesystem is actually consuming.

Start with PVC and kubelet volume metrics

If your storage driver supports volume stats, kubelet exposes metrics such as:

• 'kubelet_volume_stats_used_bytes'
• 'kubelet_volume_stats_available_bytes'
• 'kubelet_volume_stats_capacity_bytes'

Prometheus can scrape these and Grafana can visualize them per PVC. A useful PromQL query is:

1100 *
2kubelet_volume_stats_used_bytes
3/
4kubelet_volume_stats_capacity_bytes

That gives percentage usage. From there, alerts are straightforward, such as warning at 80 percent and paging at 90 percent.

Spot-check usage from inside a pod

Metrics pipelines are great for dashboards, but df is still the fastest way to verify what the mounted filesystem looks like from the workload's point of view.

kubectl exec -it my-pod -- df -h /data
kubectl exec -it my-pod -- du -sh /data

df shows filesystem capacity and free space. du shows how much data a directory tree is actually using. Both matter because sparse files, deleted-but-open files, and filesystem overhead can make them differ.

Example dashboard strategy

A practical monitoring setup usually tracks:

• percentage used for each PVC
• raw used bytes
• recent growth rate
• alerts for low available space

Growth rate is especially useful because a volume at 60 percent usage may still be urgent if it has doubled in size over the last day.

It also helps to correlate PVC usage with the application that owns it. A full volume is rarely just a storage problem. It is often a symptom of runaway logs, retention mistakes, stuck compaction, or a batch job that suddenly started writing more data than expected.

Watch the storage class and driver behavior

Not every storage backend reports usage the same way. Some CSI drivers expose accurate stats. Others do not. Thin-provisioned backends can also make the numbers confusing because Kubernetes capacity, backend allocation, and filesystem usage are not always identical.

That is why you should validate your monitoring path on the actual storage class you use rather than assuming every PVC metric is trustworthy.

Example alert rule

A simple Prometheus alert might look like this:

1groups:
2  - name: pvc-usage
3    rules:
4      - alert: PersistentVolumeAlmostFull
5        expr: |
6          kubelet_volume_stats_used_bytes
7          /
8          kubelet_volume_stats_capacity_bytes > 0.85
9        for: 10m
10        labels:
11          severity: warning
12        annotations:
13          summary: PVC is above 85 percent usage

This gives you time to expand the volume, clean data, or rotate logs before the application fails.

Common Pitfalls

• Assuming kubectl top shows PVC usage. It does not.
• Trusting PV capacity alone without checking actual used bytes.
• Ignoring whether the CSI driver exposes kubelet volume stats at all.
• Monitoring only total usage and not growth rate, which hides approaching exhaustion.
• Forgetting to check in-pod filesystem usage when the metrics look suspicious.

Summary

• Use kubelet volume metrics such as kubelet_volume_stats_used_bytes when your driver supports them.
• Build Prometheus and Grafana views around percentage used, free space, and growth rate.
• Use df and du inside a pod for fast reality checks.
• Validate the behavior of your actual storage backend instead of assuming all PVC metrics are equal.
• Alert before volumes are full so remediation happens before the application starts failing.