Introduction

The Docker CLI flag --gpus is a host-level runtime option, so it does not appear directly in Kubernetes manifests. In Kubernetes, GPU access is expressed as a resource request such as nvidia.com/gpu, and the cluster handles device injection through the container runtime and the NVIDIA device plugin. The practical answer is that you do not translate --gpus into YAML one for one; you prepare GPU nodes correctly and then request GPU resources in the pod spec.

Understand the Kubernetes GPU Model

With plain Docker, a command such as the following is enough on one machine:

docker run --rm --gpus all nvidia/cuda:12.2.0-base-ubuntu22.04 nvidia-smi

Kubernetes works differently. The scheduler needs to know which nodes have GPUs, kubelet needs to see allocatable GPU resources, and the runtime must be able to expose the device files inside the container. That usually means:

• NVIDIA driver installed on the node
• NVIDIA container toolkit or equivalent runtime integration configured
• NVIDIA device plugin running in the cluster
• pod manifest requesting nvidia.com/gpu

That stack is the Kubernetes-native equivalent of local docker run --gpus.

Prepare GPU Nodes First

Do not start with the pod manifest. Start with the host. If the host cannot see the GPU, the pod never will.

Run this on the GPU node:

nvidia-smi

If that command fails, fix the driver installation first. After that, verify your container runtime setup. On newer clusters this is often containerd, not Docker, so older nvidia-docker2 guides may be outdated for your environment. The real requirement is runtime support for NVIDIA devices, not the package name from an older tutorial.

It is also common to label GPU nodes so workloads can target them cleanly:

kubectl label nodes gpu-node-1 accelerator=nvidia

Install the NVIDIA Device Plugin

The device plugin advertises GPU capacity to Kubernetes. Without it, the scheduler will not know that a node has GPUs available.

kubectl apply -f https://raw.githubusercontent.com/NVIDIA/k8s-device-plugin/main/nvidia-device-plugin.yml

Then verify allocatable GPU resources:

kubectl get nodes -o custom-columns=NAME:.metadata.name,GPUS:.status.allocatable.nvidia\\.com/gpu

You want to see a non-zero value on GPU nodes. If the field is empty, the node is not ready for GPU scheduling yet.

Request a GPU in the Pod Spec

Once the nodes and plugin are in place, the pod manifest is straightforward. Request the GPU with a resource limit:

1apiVersion: v1
2kind: Pod
3metadata:
4  name: gpu-test
5spec:
6  restartPolicy: Never
7  nodeSelector:
8    accelerator: nvidia
9  containers:
10    - name: cuda
11      image: nvidia/cuda:12.2.0-base-ubuntu22.04
12      command: ["nvidia-smi"]
13      resources:
14        limits:
15          nvidia.com/gpu: 1

Apply it and inspect the logs:

kubectl apply -f gpu-test.yaml
kubectl logs gpu-test

If the container prints GPU information, the chain from scheduler to runtime is working.

Modern Runtime Notes Without nvidia-docker2

A lot of older posts frame the problem as "install nvidia-docker2". That was a common path in older Docker-based setups, but it is not the Kubernetes abstraction you should optimize around. The stable rule is:

• local Docker uses --gpus
• Kubernetes uses device-plugin-advertised GPU resources

If your cluster runs containerd, you may need a runtime configuration based on NVIDIA container toolkit rather than a Docker-specific package. The exact file paths depend on your distribution, but the Kubernetes contract stays the same.

Some environments also use RuntimeClass for alternate runtimes. If your cluster requires it, add it explicitly:

1apiVersion: node.k8s.io/v1
2kind: RuntimeClass
3metadata:
4  name: nvidia
5handler: nvidia

Then reference it from the pod:

spec:
  runtimeClassName: nvidia

Only do this if your cluster runtime is actually configured that way.

Troubleshoot by Layer

When GPU pods do not work, debug from the bottom up:

1. host driver with nvidia-smi
2. runtime integration on the node
3. device plugin health
4. node allocatable GPU resource
5. pod scheduling events
6. container image compatibility with the host driver

This avoids wasting time on YAML changes when the real problem is host setup.

Common Pitfalls

• Trying to put Docker CLI --gpus flags directly into a Kubernetes manifest.
• Assuming nvidia-docker2 is required in every modern Kubernetes setup.
• Deploying the device plugin before verifying that the host driver works.
• Forgetting to request nvidia.com/gpu in the container resource limits.
• Scheduling GPU workloads onto general nodes because selectors, taints, or affinity rules were not defined.

Summary

• Kubernetes does not take Docker --gpus flags directly.
• GPU support comes from correct node runtime setup plus the NVIDIA device plugin.
• Pods request GPUs with nvidia.com/gpu resource limits.
• Older nvidia-docker2 advice is often runtime-specific and not the real Kubernetes model.
• Debug GPU issues from host driver upward instead of starting with the pod YAML.