Introduction

When TensorFlow will not use the GPU, the problem is usually one of four things: the runtime cannot see the GPU, the installed TensorFlow build is incompatible with the local CUDA stack, the notebook or process is still using an old environment, or the code is silently falling back to CPU. The fastest path is to verify each layer in order instead of changing packages blindly.

Confirm that the machine can see the GPU

Start outside TensorFlow. If the OS cannot see the device, TensorFlow will not fix it.

nvidia-smi

If this command fails or shows no GPU, the issue is at the driver or machine level. Fix that before touching Python packages.

Check whether TensorFlow detects a GPU

Once the driver looks healthy, inspect TensorFlow directly.

1import tensorflow as tf
2
3print(tf.__version__)
4print(tf.config.list_physical_devices("GPU"))

An empty list means TensorFlow is not seeing any usable GPU device. That could be an installation mismatch, a CPU-only environment, or a stale runtime session.

Make sure the environment is internally consistent

A common failure mode is mixing package versions. In notebook environments, users often install or upgrade TensorFlow but keep running the old kernel state. After changing packages, restart the runtime or Python process.

import sys
print(sys.executable)

This helps confirm which interpreter the notebook is actually using. If you installed TensorFlow in one environment but the notebook is attached to another, GPU support will appear broken even though the package installation itself succeeded.

Verify that the code really runs on GPU

Even when a GPU is visible, some workloads still execute on CPU. Turn on device placement logging and run a simple tensor operation.

1import tensorflow as tf
2
3tf.debugging.set_log_device_placement(True)
4
5with tf.device("/GPU:0"):
6    a = tf.random.normal((2000, 2000))
7    b = tf.random.normal((2000, 2000))
8    c = tf.matmul(a, b)
9
10print(c.shape)

If the logs show CPU placement, either the requested GPU is unavailable or the environment is falling back.

Watch for notebook-specific issues

Colab and Jupyter add a few extra traps:

• The runtime may be connected to CPU instead of GPU hardware.
• After pip install, the kernel may need a restart.
• A local file named tensorflow.py can shadow the real package.

Check for shadowing with:

import tensorflow as tf
print(tf.__file__)

If that path points to your project file instead of the installed library, rename the local file.

Small ops may not show a meaningful speedup

Users sometimes conclude that TensorFlow is not using the GPU because simple code runs at similar speed on CPU. Small tensors, input pipelines, and control-heavy code can be bottlenecked elsewhere. Test with a large matrix multiply or a real model training step before judging.

That matters because the right question is not only device visibility, but whether the actual workload benefits from acceleration.

Memory growth can prevent confusing startup failures

On some systems, TensorFlow tries to reserve too much GPU memory up front. Enabling memory growth can make the runtime behave more predictably.

1import tensorflow as tf
2
3gpus = tf.config.list_physical_devices("GPU")
4for gpu in gpus:
5    tf.config.experimental.set_memory_growth(gpu, True)

Use this near startup before running GPU-heavy code.

Common Pitfalls

• Troubleshooting TensorFlow before checking that nvidia-smi works at the machine level.
• Upgrading packages in a notebook and forgetting to restart the runtime.
• Assuming GPU acceleration is broken because tiny tensor operations are not noticeably faster.
• Ignoring tf.__file__ and accidentally importing a local tensorflow.py file.
• Forgetting to verify actual device placement after TensorFlow reports that a GPU exists.

Summary

• Verify the GPU and driver first with nvidia-smi.
• Check TensorFlow's device list with tf.config.list_physical_devices("GPU").
• Restart notebooks after package changes so the runtime picks up the new environment.
• Confirm real GPU placement with logging or a large test operation.
• Fix environment shadowing and memory behavior before assuming TensorFlow itself is broken.