Introduction

Seeing only about 60 percent GPU utilization on a GTX 1070 does not automatically mean TensorFlow is broken. GPU utilization is the result of the whole training pipeline: input loading, preprocessing, batch size, model structure, and synchronization between CPU and GPU. The right question is not “why isn’t it 100 percent,” but “what is keeping the device from receiving more work.”

Low Utilization Usually Means the GPU Is Waiting

A GPU reaches high utilization only when it always has kernels ready to execute. If utilization is stuck around 60 percent, the device is often spending the rest of its time waiting for one of these:

• the CPU to prepare the next batch
• disk or network I/O to deliver data
• small kernels that do not saturate the device well
• synchronization points between training steps
• a batch size too small to keep the GPU busy

That is why a simple utilization number does not identify the root cause by itself.

Fix the Input Pipeline First

On many training jobs, the bottleneck is not the model. It is data loading. TensorFlow’s tf.data pipeline should do parsing, mapping, batching, and prefetching efficiently so the GPU is not starved.

1import tensorflow as tf
2
3dataset = tf.data.Dataset.range(100000)
4dataset = dataset.map(lambda x: x * 2, num_parallel_calls=tf.data.AUTOTUNE)
5dataset = dataset.batch(256)
6dataset = dataset.prefetch(tf.data.AUTOTUNE)
7
8for batch in dataset.take(1):
9    print(batch[:5].numpy())

The important idea is prefetching. It lets the CPU prepare the next batch while the GPU is working on the current one.

If you are decoding images or parsing large records, this matters even more. A slow pipeline can easily cap the GPU far below full utilization.

Batch Size Has a Huge Effect

A GTX 1070 is large enough that very small batches often leave it underused. Increasing the batch size increases the amount of work per training step and can improve utilization substantially, as long as memory still fits.

history = model.fit(train_ds.batch(32), epochs=3)
history = model.fit(train_ds.batch(128), epochs=3)

You should not assume “bigger is always better,” but if your batch size is tiny, low utilization is expected. The right batch size is the largest stable one that still fits memory and does not damage training dynamics.

Profile Instead of Guessing

Use TensorFlow’s profiler and nvidia-smi together. nvidia-smi shows the symptom. TensorFlow profiling shows where the time goes.

1import tensorflow as tf
2
3logdir = "./logs"
4tf.profiler.experimental.start(logdir)
5model.fit(train_ds, epochs=1)
6tf.profiler.experimental.stop()

Then inspect whether the timeline shows:

• long input stalls
• host-to-device copy bottlenecks
• many tiny operations
• expensive CPU preprocessing

That is much more actionable than staring at utilization percentages alone.

Model Structure Also Matters

Some models naturally produce lower utilization because they launch many small operations rather than a few large matrix-heavy kernels. Recurrent models, small custom ops, or models with frequent control-flow boundaries can keep utilization below the numbers you might see from large CNN or transformer workloads.

So 60 percent on a GTX 1070 may be completely reasonable for one model and a sign of pipeline inefficiency for another. The architecture changes the ceiling.

Mixed Precision and CPU Contention

On supported hardware, mixed precision can improve throughput, but a GTX 1070 will not benefit in the same way as newer Tensor Core GPUs. CPU contention also matters: if the CPU is heavily loaded by preprocessing, augmentation, or other system work, the GPU may never receive data fast enough.

That is why it is worth monitoring both CPU and GPU utilization during training rather than treating the GPU in isolation.

Common Pitfalls

• Assuming GPU utilization must be near 100 percent for the run to be healthy.
• Ignoring the input pipeline and focusing only on model code.
• Using a batch size that is too small for the device.
• Looking at nvidia-smi alone instead of profiling TensorFlow execution.
• Comparing utilization across very different model architectures as if they should behave the same.

Summary

• A GTX 1070 sitting around 60 percent utilization usually means the GPU is waiting for work, not that TensorFlow is malfunctioning.
• The most common bottlenecks are input pipelines, small batches, and CPU-side preprocessing.
• Use tf.data with parallel mapping and prefetching so the next batch is ready sooner.
• Increase batch size when memory allows, then measure again.
• Profile the training run before changing random settings, because utilization alone does not tell you the real bottleneck.