Introduction

TensorFlow can use two GPUs at the same time, but it does not happen just because two devices are installed. You need a distribution strategy or explicit device placement that tells TensorFlow how to split work across those GPUs. For most single-machine training jobs, the standard answer is tf.distribute.MirroredStrategy.

Verify That TensorFlow Sees Both GPUs

Before changing model code, confirm that TensorFlow detects the devices.

import tensorflow as tf

print(tf.config.list_physical_devices('GPU'))

If you do not see two GPUs here, multi-GPU training will not work no matter what strategy code you write. Device visibility must come first.

The Standard Solution: MirroredStrategy

MirroredStrategy replicates the model on each GPU and keeps the variables synchronized. During training:

• each GPU gets a shard of the batch
• each replica computes gradients locally
• TensorFlow reduces the gradients across devices
• all replicas update in sync

A minimal example:

1import tensorflow as tf
2import numpy as np
3
4strategy = tf.distribute.MirroredStrategy()
5
6with strategy.scope():
7    model = tf.keras.Sequential([
8        tf.keras.layers.Dense(64, activation='relu', input_shape=(20,)),
9        tf.keras.layers.Dense(1)
10    ])
11    model.compile(optimizer='adam', loss='mse')
12
13x = np.random.randn(1024, 20).astype('float32')
14y = np.random.randn(1024, 1).astype('float32')
15
16model.fit(x, y, epochs=3, batch_size=64)

This is the usual answer for using two GPUs on one machine with Keras-style training.

Batch Size Matters

With mirrored training, the effective global batch is split across the GPUs. If you set batch_size=64 and use two GPUs, each replica usually processes a local batch of 32.

That matters because:

• very small per-GPU batches can underutilize hardware
• larger global batches may require learning-rate tuning
• input pipelines must supply data fast enough for both devices

So "two GPUs" does not automatically mean "twice as fast". The model, batch size, and input pipeline all affect the result.

Keep the Input Pipeline Fast Enough

A slow data pipeline can make multi-GPU training disappointing. If CPU preprocessing or disk reading is the bottleneck, adding a second GPU does not help much.

TensorFlow datasets should usually be prefetched:

1import tensorflow as tf
2import numpy as np
3
4x = np.random.randn(1024, 20).astype('float32')
5y = np.random.randn(1024, 1).astype('float32')
6
7dataset = tf.data.Dataset.from_tensor_slices((x, y))
8dataset = dataset.shuffle(1024).batch(64).prefetch(tf.data.AUTOTUNE)

Then pass dataset to model.fit(...).

When Manual Placement Is Useful

If you are not doing mirrored data-parallel training, you can place specific operations on different GPUs manually.

1with tf.device('/GPU:0'):
2    a = tf.random.normal((1000, 1000))
3
4with tf.device('/GPU:1'):
5    b = tf.random.normal((1000, 1000))

That is useful for certain custom workloads, but for normal model training it is usually more complex and less maintainable than MirroredStrategy.

Watch Memory Growth and OOMs

By default, TensorFlow may reserve a lot of GPU memory. On development machines, enabling memory growth can make experimentation easier.

gpus = tf.config.list_physical_devices('GPU')
for gpu in gpus:
    tf.config.experimental.set_memory_growth(gpu, True)

This does not create multi-GPU training by itself, but it helps avoid some startup and resource-allocation surprises.

Common Pitfalls

A common mistake is expecting TensorFlow to automatically split training across both GPUs just because both are visible. You still need a strategy or explicit placement.

Another mistake is using a global batch that becomes too small per GPU, leaving both devices underutilized.

People also often ignore the input pipeline. Two GPUs waiting on slow data loading do not train efficiently.

Finally, do not measure success only by GPU visibility. Measure actual throughput and step time after enabling multi-GPU training.

Summary

• TensorFlow can use two GPUs on one machine, but the usual mechanism is tf.distribute.MirroredStrategy
• First confirm that TensorFlow detects both GPUs
• Mirrored training replicates the model and synchronizes gradients across devices
• Batch size and input pipeline performance strongly affect whether two GPUs help in practice
• Manual device placement exists, but it is usually not the easiest path for standard Keras training
• Multi-GPU setup is successful only when both device visibility and workload distribution are configured correctly