Introduction

When TensorFlow or Keras refuses to use a GPU, the root cause is usually environment compatibility rather than model code. Developers often see one of three symptoms: GPU is not listed, runtime falls back silently to CPU, or training crashes with CUDA/cuDNN errors. All three come from mismatches between TensorFlow version, NVIDIA driver, CUDA runtime, and cuDNN libraries.

A reliable debug process starts with hardware detection, then verifies software versions, then checks runtime placement. Skipping this order causes wasted time on model-level tuning before basic device setup is correct.

Core Sections

1. Confirm TensorFlow sees the GPU

Start with a minimal detection script:

1import tensorflow as tf
2
3print("TensorFlow:", tf.__version__)
4print("Built with CUDA:", tf.test.is_built_with_cuda())
5print("Visible GPUs:", tf.config.list_physical_devices('GPU'))

If the list is empty, TensorFlow cannot use GPU regardless of model configuration.

2. Validate NVIDIA driver and CUDA stack

On Linux or WSL, inspect driver state:

nvidia-smi

If nvidia-smi fails, TensorFlow cannot access the GPU. Next, verify installed libraries match TensorFlow requirements for your version. Use official compatibility documentation and pin versions in your environment config.

3. Test actual device placement

A detected GPU does not guarantee operations run on GPU. Enable placement logs:

1import tensorflow as tf
2
3tf.debugging.set_log_device_placement(True)
4a = tf.random.normal([2048, 2048])
5b = tf.random.normal([2048, 2048])
6c = tf.matmul(a, b)
7print(c.shape)

Logs should show matmul placed on /device:GPU:0. If operations stay on CPU, check unsupported ops, explicit device scopes, or memory growth configuration.

4. Configure GPU memory behavior

TensorFlow may pre-allocate most GPU memory, causing conflicts with other processes.

gpus = tf.config.list_physical_devices('GPU')
for gpu in gpus:
    tf.config.experimental.set_memory_growth(gpu, True)

Set this early before GPU initialization.

5. Rebuild a clean virtual environment

Mixed package leftovers are a common issue. Create a clean environment and install only required packages.

1python -m venv .venv
2source .venv/bin/activate
3pip install --upgrade pip
4pip install tensorflow==2.16.1

Then rerun the minimal GPU test before adding other ML dependencies.

6. Containerized setups and cloud VMs

In Docker, ensure GPU runtime is enabled:

docker run --gpus all --rm nvidia/cuda:12.3.2-base-ubuntu22.04 nvidia-smi

For cloud VMs, verify GPU drivers are installed inside the instance image, not just attached at the infrastructure layer.

Common Pitfalls

• Installing TensorFlow successfully but ignoring CUDA/cuDNN and driver compatibility matrix.
• Assuming GPU detection implies ops are actually scheduled on GPU.
• Running in a polluted environment with conflicting deep learning package versions.
• Forgetting memory growth settings and misdiagnosing allocation conflicts as GPU failure.
• Debugging model architecture before confirming basic device availability and placement.

Summary

TensorFlow/Keras GPU failures are usually compatibility and runtime configuration issues. Validate the stack in order: hardware visibility, driver status, TensorFlow build compatibility, op placement, and clean environment reproducibility. Add a minimal GPU smoke test to your setup scripts so regressions are caught immediately. Once the platform contract is stable, model training behavior becomes predictable and performance tuning can proceed productively.

To make this guidance robust in day-to-day engineering work, treat it as an executable checklist instead of one-time reading material. Capture the expected environment, dependency versions, runtime flags, and validation commands in your repository so every contributor can reproduce the same behavior from a clean setup. This is especially important when onboarding new developers, rotating on-call ownership, or debugging incidents under time pressure. Documentation that includes concrete commands, expected outputs, and failure interpretation prevents repeat confusion and shortens recovery time.

It is also worth adding at least one automated guardrail in CI that validates the highest-risk assumption described in the article. Depending on the topic, that guardrail may be a smoke test, policy check, schema validation, benchmark threshold, import check, or integration assertion against a minimal fixture. The goal is to fail fast when environment drift or configuration changes reintroduce old errors. Teams that convert troubleshooting knowledge into small, repeatable checks reduce operational noise and keep this class of issue from returning every sprint.