Introduction

TensorFlow does not have a special attachment to the physical first GPU. What usually happens is simpler: TensorFlow sees a list of visible GPUs and, unless you tell it otherwise, places work on the first visible device in that list. That device is then reported as GPU:0, which makes it look as if TensorFlow always insists on the same card.

Why GPU:0 Is the Default

When TensorFlow starts, it asks the CUDA runtime which devices are available. The runtime exposes an ordered list of visible GPUs. TensorFlow names them logically as GPU:0, GPU:1, and so on.

The important detail is that these are logical device indexes, not necessarily your machine's permanent physical numbering. If only one GPU is visible, TensorFlow calls it GPU:0 even if it is physically the second or third card in the machine.

That is why this command:

CUDA_VISIBLE_DEVICES=1 python train.py

can still lead TensorFlow to log usage of GPU:0. In that process, the single visible GPU has been renumbered to logical device zero.

Restrict Which GPU TensorFlow Can See

The most common way to choose a GPU is to limit visibility before Python starts:

CUDA_VISIBLE_DEVICES=1 python train.py

If your machine has GPUs 0 and 1, this hides the first one from the process. TensorFlow now sees only one GPU and labels it as GPU:0 inside the application.

This behavior confuses many people because they expect TensorFlow logs to preserve the original CUDA numbering. They do not. Visibility happens first, then logical numbering happens inside the reduced set.

Select Devices From TensorFlow Code

You can also manage visible devices in code:

1import tensorflow as tf
2
3gpus = tf.config.list_physical_devices("GPU")
4if gpus:
5    tf.config.set_visible_devices(gpus[1], "GPU")
6    tf.config.experimental.set_memory_growth(gpus[1], True)
7
8logical_gpus = tf.config.list_logical_devices("GPU")
9print(logical_gpus)

After this runs, TensorFlow exposes only one logical GPU to the program. That logical device will typically appear as GPU:0 because it is the first visible GPU in the process.

Why It Looks Like TensorFlow Uses Only One GPU

Another common misunderstanding is expecting automatic multi-GPU training. TensorFlow does not spread a model across all available GPUs just because several cards exist. If you want multi-device execution, you usually need an explicit distribution strategy or model-parallel design.

For example:

1import tensorflow as tf
2
3strategy = tf.distribute.MirroredStrategy()
4
5with strategy.scope():
6    model = tf.keras.Sequential([
7        tf.keras.layers.Dense(64, activation="relu"),
8        tf.keras.layers.Dense(10)
9    ])
10    model.compile(optimizer="adam", loss="sparse_categorical_crossentropy")

Without code like this, the program often runs on a single default device, which again is usually the first visible GPU.

Inspect What TensorFlow Actually Sees

Before debugging placement logs, print the device lists:

1import tensorflow as tf
2
3print("Physical:", tf.config.list_physical_devices("GPU"))
4print("Logical:", tf.config.list_logical_devices("GPU"))

This tells you whether the issue is:

• only one GPU is visible
• TensorFlow renumbered visible GPUs
• the program is using one device by design

That is much more reliable than guessing from one log line.

Common Pitfalls

• Assuming GPU:0 always means the physically first GPU in the machine.
• Forgetting that CUDA_VISIBLE_DEVICES renumbers the visible set.
• Expecting TensorFlow to use multiple GPUs automatically without a distribution strategy.
• Changing device settings after TensorFlow has already initialized the runtime.
• Interpreting logical device names as hardware inventory instead of process-local names.

Summary

• TensorFlow usually uses the first visible GPU, not necessarily the physically first GPU.
• After visibility filtering, TensorFlow renumbers devices logically as GPU:0, GPU:1, and so on.
• 'CUDA_VISIBLE_DEVICES is the simplest way to choose which GPU a process can use.'
• Code-level device configuration can further control visibility and memory behavior.
• If you want multi-GPU execution, configure it explicitly rather than expecting it by default.