Introduction

Checking GPU availability in TensorFlow is essential before model training, especially on shared workstations, cloud VMs, and containerized environments. Developers often assume "TensorFlow with CUDA installed" guarantees GPU execution, but runtime visibility can fail due to driver mismatch, missing CUDA libraries, memory configuration, or device masking.

A reliable GPU check should answer three questions: does TensorFlow detect physical GPU devices, can it place operations on those devices, and is memory behavior configured safely for your workload. This guide uses TensorFlow 2.x APIs and includes a compatibility note for older code patterns.

Core Sections

1. List visible GPUs programmatically

Use tf.config.list_physical_devices("GPU") as the primary runtime check.

1import tensorflow as tf
2
3print("TensorFlow version:", tf.__version__)
4
5gpus = tf.config.list_physical_devices("GPU")
6print("Visible GPUs:", gpus)
7
8if not gpus:
9    print("No GPU visible to TensorFlow")
10else:
11    print(f"Detected {len(gpus)} GPU(s)")

If this returns an empty list, investigate environment variables (CUDA_VISIBLE_DEVICES), driver installation, and TensorFlow/CUDA compatibility.

2. Configure memory growth and verify placement

By default, TensorFlow may try to allocate most GPU memory. On shared systems, enable memory growth.

1import tensorflow as tf
2
3gpus = tf.config.list_physical_devices("GPU")
4for gpu in gpus:
5    tf.config.experimental.set_memory_growth(gpu, True)
6
7with tf.device("/GPU:0"):
8    a = tf.random.uniform((2048, 2048))
9    b = tf.random.uniform((2048, 2048))
10    c = tf.matmul(a, b)
11
12print("Result shape:", c.shape)

To debug placement decisions, enable device logging:

tf.debugging.set_log_device_placement(True)

This confirms whether operations run on GPU or silently fall back to CPU.

3. Handling TensorFlow 1.x-style checks and modern migration

Older examples often use APIs such as tf.test.is_gpu_available() or session-based inspection. In modern TensorFlow, prefer tf.config.

1# Legacy compatibility helper for mixed codebases
2import tensorflow as tf
3
4def gpu_status():
5    gpus = tf.config.list_physical_devices("GPU")
6    return {
7        "tf_version": tf.__version__,
8        "gpu_count": len(gpus),
9        "gpu_names": [d.name for d in gpus],
10    }
11
12print(gpu_status())

In containerized deployments, also check host-level tools:

nvidia-smi

If nvidia-smi fails, TensorFlow will not access GPU regardless of Python code.

Common Pitfalls

• Relying only on package installation status without checking runtime device visibility via TensorFlow APIs.
• Ignoring CUDA and cuDNN version compatibility with the installed TensorFlow build.
• Forgetting memory growth on multi-tenant machines, causing out-of-memory errors for other processes.
• Assuming all ops run on GPU; some operations may still execute on CPU unless explicitly supported.
• Confusing "GPU present on host" with "GPU visible in container" when Docker runtime flags are missing.

Summary

The dependable way to check available GPUs in TensorFlow is to list physical GPU devices, verify actual op placement, and configure memory behavior explicitly. This workflow catches most environment issues early and avoids silent CPU fallback. Once detection and placement checks pass, you can trust that training workloads are using accelerator hardware as intended.

In shared clusters, add GPU checks to application startup logs so failures are visible immediately. Record TensorFlow version, detected GPU count, selected visible devices, and whether memory growth was enabled. These diagnostics make support much faster because infrastructure and model teams can compare runtime conditions between healthy and failing jobs without reproducing locally.

For reproducibility, pin your TensorFlow container image and CUDA stack instead of relying on floating tags. Small image updates can change device drivers or shared libraries and silently alter GPU visibility. If jobs are scheduled on multiple node types, include node labels and nvidia-smi output in run metadata. This operational context often explains intermittent "GPU not found" behavior that appears random when viewed only from Python code.

Finally, benchmark a tiny matmul or convolution on startup in non-production mode to verify not just visibility but functional execution. Device listing confirms discovery, but a quick compute check confirms end-to-end usability.

If your workload uses multiple GPUs, also verify logical device mapping and per-process visibility rules before training starts. A quick startup check that prints visible device names and counts per worker process can prevent distributed training misconfiguration that would otherwise surface only after long-running jobs begin.