prometheus
disk space monitoring
total disk space
free disk space
system metrics

Get total and free disk space using Prometheus

System Design practice on Codemia

Work through 120+ system design problems with detailed solutions, from rate limiters to multi-region storage.

Practice system design

Introduction

Prometheus monitors disk space using metrics from the Node Exporter, which exposes filesystem metrics like node_filesystem_size_bytes, node_filesystem_free_bytes, and node_filesystem_avail_bytes. You query these metrics using PromQL to calculate total space, free space, used space, and percentage utilization. Setting up alerting rules for low disk space is critical for preventing outages caused by full disks.

Prerequisites

Install and run the Node Exporter on each server:

bash
1# Download and run Node Exporter
2wget https://github.com/prometheus/node_exporter/releases/download/v1.7.0/node_exporter-1.7.0.linux-amd64.tar.gz
3tar xzf node_exporter-1.7.0.linux-amd64.tar.gz
4./node_exporter-1.7.0.linux-amd64/node_exporter &
5
6# Verify metrics are exposed
7curl http://localhost:9100/metrics | grep node_filesystem

Configure Prometheus to scrape the Node Exporter:

yaml
1# prometheus.yml
2scrape_configs:
3  - job_name: 'node'
4    static_configs:
5      - targets: ['server1:9100', 'server2:9100']

Key Filesystem Metrics

MetricDescription
node_filesystem_size_bytesTotal size of the filesystem
node_filesystem_free_bytesFree space (including reserved for root)
node_filesystem_avail_bytesSpace available for non-root users
node_filesystem_filesTotal number of inodes
node_filesystem_files_freeFree inodes

The difference between free and avail is that Linux reserves ~5% of disk space for root. avail_bytes reflects what regular users can actually use.

Basic PromQL Queries

Total Disk Space

promql
1# Total size of each filesystem in GB
2node_filesystem_size_bytes / (1024^3)
3
4# Total size for a specific mount point
5node_filesystem_size_bytes{mountpoint="/"} / (1024^3)
6
7# Total size per instance
8node_filesystem_size_bytes{mountpoint="/", fstype!="tmpfs"} / (1024^3)

Free Disk Space

promql
1# Free space available for non-root users (in GB)
2node_filesystem_avail_bytes{mountpoint="/"} / (1024^3)
3
4# Free space including root-reserved space
5node_filesystem_free_bytes{mountpoint="/"} / (1024^3)

Used Disk Space

promql
# Used space = total - free
(node_filesystem_size_bytes{mountpoint="/"} - node_filesystem_free_bytes{mountpoint="/"}) / (1024^3)

Disk Usage Percentage

promql
1# Percentage used (most useful for monitoring)
2100 - (node_filesystem_avail_bytes{mountpoint="/"} / node_filesystem_size_bytes{mountpoint="/"} * 100)
3
4# Or equivalently
5(1 - node_filesystem_avail_bytes{mountpoint="/"} / node_filesystem_size_bytes{mountpoint="/"}) * 100

Filtering Relevant Filesystems

By default, Node Exporter reports all mounted filesystems, including virtual ones. Filter them out:

promql
1# Exclude tmpfs, devtmpfs, and other virtual filesystems
2node_filesystem_avail_bytes{
3    fstype!~"tmpfs|devtmpfs|squashfs|overlay",
4    mountpoint!~"/run.*|/sys.*|/proc.*|/dev.*|/snap.*"
5}

Common fstype Filters

promql
1# Only real disk filesystems
2node_filesystem_size_bytes{fstype=~"ext4|xfs|btrfs|zfs"}
3
4# Exclude all virtual filesystems
5node_filesystem_size_bytes{fstype!~"tmpfs|devtmpfs|squashfs|overlay|nsfs|fuse.*"}

Alerting Rules

yaml
1# prometheus_rules.yml
2groups:
3  - name: disk_alerts
4    rules:
5      # Alert when disk is more than 80% full
6      - alert: DiskSpaceWarning
7        expr: |
8          (1 - node_filesystem_avail_bytes{fstype!~"tmpfs|devtmpfs"} /
9               node_filesystem_size_bytes{fstype!~"tmpfs|devtmpfs"}) * 100 > 80
10        for: 5m
11        labels:
12          severity: warning
13        annotations:
14          summary: "Disk space warning on {{ $labels.instance }}"
15          description: "{{ $labels.mountpoint }} is {{ $value | printf \"%.1f\" }}% full"
16
17      # Alert when disk is more than 95% full
18      - alert: DiskSpaceCritical
19        expr: |
20          (1 - node_filesystem_avail_bytes{fstype!~"tmpfs|devtmpfs"} /
21               node_filesystem_size_bytes{fstype!~"tmpfs|devtmpfs"}) * 100 > 95
22        for: 1m
23        labels:
24          severity: critical
25        annotations:
26          summary: "Critical disk space on {{ $labels.instance }}"
27          description: "{{ $labels.mountpoint }} is {{ $value | printf \"%.1f\" }}% full"
28
29      # Alert when disk will be full within 24 hours (predictive)
30      - alert: DiskWillFillIn24Hours
31        expr: |
32          predict_linear(
33            node_filesystem_avail_bytes{fstype!~"tmpfs|devtmpfs"}[6h], 24*3600
34          ) < 0
35        for: 30m
36        labels:
37          severity: warning
38        annotations:
39          summary: "Disk will fill within 24h on {{ $labels.instance }}"
40
41      # Low inode alert
42      - alert: InodeSpaceLow
43        expr: |
44          node_filesystem_files_free{fstype!~"tmpfs|devtmpfs"} /
45          node_filesystem_files{fstype!~"tmpfs|devtmpfs"} * 100 < 10
46        for: 5m
47        labels:
48          severity: warning
49        annotations:
50          summary: "Low inode space on {{ $labels.instance }}:{{ $labels.mountpoint }}"

Grafana Dashboard Queries

promql
1# Single stat: Current usage percentage for /
2(1 - node_filesystem_avail_bytes{instance="$instance", mountpoint="/"} /
3     node_filesystem_size_bytes{instance="$instance", mountpoint="/"}) * 100
4
5# Table: All filesystems with usage
6node_filesystem_size_bytes{instance="$instance", fstype!~"tmpfs|devtmpfs"} / (1024^3)
7
8# Time series: Free space over time
9node_filesystem_avail_bytes{instance="$instance", mountpoint="/"} / (1024^3)
10
11# Rate of disk consumption (GB per day)
12deriv(node_filesystem_avail_bytes{instance="$instance", mountpoint="/"}[1h]) * 3600 * 24 / (1024^3)

Common Pitfalls

  • Not filtering virtual filesystems: Without filtering tmpfs, devtmpfs, and overlay, your alerts fire on virtual filesystems that do not represent real disk space. Always filter by fstype.
  • Using free_bytes instead of avail_bytes: free_bytes includes space reserved for root (typically 5%). avail_bytes is what regular processes can actually use. Alerts should be based on avail_bytes.
  • Missing mount point labels: Different servers may have different mount points (/, /data, /var). Use label matchers to target specific mount points, or apply alerts to all non-virtual filesystems.
  • Not monitoring inodes: A filesystem can have plenty of free bytes but run out of inodes (file entries), especially with many small files. Monitor node_filesystem_files_free alongside disk space.
  • Ignoring predictive alerts: Threshold alerts fire only after the disk is already nearly full. Use predict_linear() to alert hours or days before the disk fills, giving time to respond.

Summary

  • Node Exporter exposes node_filesystem_size_bytes, node_filesystem_avail_bytes, and node_filesystem_free_bytes
  • Calculate usage percentage with (1 - avail/size) * 100 using avail_bytes (not free_bytes)
  • Filter virtual filesystems with fstype!~"tmpfs|devtmpfs|squashfs|overlay"
  • Set up threshold alerts at 80% (warning) and 95% (critical), plus a predictive predict_linear alert
  • Monitor inodes (node_filesystem_files_free) alongside disk space to catch inode exhaustion

Related reading
Course
Beginner
27 lessons
10 hours
System Design Fundamentals

Build a strong foundation in designing scalable, reliable distributed systems.

View the course
Track what you have practised

A free account saves your progress, solutions and study plan across every problem on Codemia.

System Design practice on Codemia

Work through 120+ system design problems with detailed solutions, from rate limiters to multi-region storage.

Practice system design

All Rights Reserved.