Introduction

Kubernetes can use Ceph RBD as dynamic block storage, and on CoreOS-style nodes the main concerns are no longer the old in-tree volume plugin but the CSI driver, node kernel support, and correct Ceph credentials. In modern clusters, the practical answer is to use Ceph CSI for RBD-backed PersistentVolumeClaim provisioning rather than the removed legacy rbd volume type.

The modern architecture: Kubernetes plus Ceph CSI

Ceph RBD provides network block devices backed by a Ceph pool. Kubernetes consumes them through the Ceph CSI RBD driver, which provisions and attaches RBD images for pods.

At a high level, the stack is:

1. Ceph cluster provides the RBD pool.
2. Ceph CSI runs in Kubernetes.
3. A StorageClass defines how to provision RBD volumes.
4. A PersistentVolumeClaim requests storage.
5. A pod mounts the resulting volume.

Older articles often reference the in-tree rbd plugin, but that path is obsolete for current Kubernetes versions. The CSI driver is the supported approach.

CoreOS-specific considerations

Whether you mean the older Container Linux by CoreOS or a modern CoreOS-style node OS such as Fedora CoreOS, the operational theme is similar: the node OS is minimal, so storage features must be present deliberately rather than assumed.

For Ceph RBD on the worker nodes, pay attention to:

• kernel RBD support on the node
• CSI node plugin deployment
• access to Ceph monitors from the node network
• secret distribution for Ceph authentication

Because the OS is minimal, troubleshooting often starts at the node layer, not in the application pod.

Example StorageClass for Ceph RBD

A typical CSI-based StorageClass looks like this:

1apiVersion: storage.k8s.io/v1
2kind: StorageClass
3metadata:
4  name: csi-rbd-sc
5provisioner: rbd.csi.ceph.com
6parameters:
7  clusterID: your-ceph-fsid
8  pool: kubernetes
9  imageFeatures: layering
10  csi.storage.k8s.io/provisioner-secret-name: csi-rbd-secret
11  csi.storage.k8s.io/provisioner-secret-namespace: default
12  csi.storage.k8s.io/node-stage-secret-name: csi-rbd-secret
13  csi.storage.k8s.io/node-stage-secret-namespace: default
14reclaimPolicy: Delete
15allowVolumeExpansion: true

Then request storage with a PersistentVolumeClaim:

1apiVersion: v1
2kind: PersistentVolumeClaim
3metadata:
4  name: app-data
5spec:
6  accessModes:
7    - ReadWriteOnce
8  storageClassName: csi-rbd-sc
9  resources:
10    requests:
11      storage: 10Gi

And mount it in a pod:

1apiVersion: v1
2kind: Pod
3metadata:
4  name: demo
5spec:
6  containers:
7    - name: app
8      image: busybox
9      command: ["sh", "-c", "sleep 3600"]
10      volumeMounts:
11        - name: data
12          mountPath: /data
13  volumes:
14    - name: data
15      persistentVolumeClaim:
16        claimName: app-data

That is the normal modern workflow for RBD-backed block storage in Kubernetes.

Ceph-side preparation still matters

Kubernetes is only one half of the setup. Ceph needs:

• an RBD pool initialized for use
• a Ceph user with appropriate mon and osd caps
• monitor addresses reachable from Kubernetes nodes

If the CSI driver is installed but the secret is wrong or the monitors are unreachable, PVC provisioning will stall even though the YAML looks correct.

Common Pitfalls

The biggest mistake is following old examples that use the in-tree rbd volume plugin. Modern Kubernetes clusters should use Ceph CSI instead.

Another common issue is assuming the application container needs Ceph tools. In CSI-based setups, the storage attach and mount logic belongs to the CSI components and node environment, not the application image.

On CoreOS-style nodes, missing kernel features or node plugin problems can also block volume staging. Minimal operating systems are great for consistency, but they make hidden assumptions about storage tooling more visible.

Finally, remember that RBD is block storage. The typical access mode is ReadWriteOnce, so do not expect one volume to be mounted read-write by many pods at the same time unless the storage and workload semantics explicitly support it.

Summary

• Kubernetes should use Ceph RBD through the Ceph CSI driver, not the removed legacy in-tree plugin.
• CoreOS-style nodes need the right node-level support for CSI staging and RBD mapping.
• The core objects are StorageClass, PersistentVolumeClaim, and the consuming pod.
• Ceph-side setup still matters: pool, authentication, and monitor reachability must all be correct.
• When troubleshooting, check both Kubernetes events and the CSI or node-side logs, not just the application pod.