Introduction

TensorFlow often tries to reserve GPU memory aggressively, which can surprise people using Keras and lead to ResourceExhaustedError or the impression that TensorFlow is not "using memory only as needed." One common fix is to enable GPU memory growth so TensorFlow allocates memory more incrementally instead of grabbing most of it up front.

Enable Memory Growth

The standard configuration pattern is to configure physical GPU devices before the model is created.

1import tensorflow as tf
2
3gpus = tf.config.list_physical_devices("GPU")
4if gpus:
5    for gpu in gpus:
6        tf.config.experimental.set_memory_growth(gpu, True)

This tells TensorFlow to grow memory usage as required rather than preallocating the full GPU memory region immediately.

That is often the first step when developers want TensorFlow to coexist more politely with other processes on the same GPU.

What Memory Growth Does Not Solve

Memory growth changes allocation behavior, but it does not create extra GPU memory. If the model or batch size genuinely requires more memory than the device has, you can still get ResourceExhaustedError.

So there are two separate problems:

• Overeager allocation strategy.
• Real memory exhaustion.

Memory growth helps mostly with the first one.

Batch Size Is Still the First Lever

If the error happens during training, lowering the batch size is often the fastest real fix.

model.fit(x_train, y_train, batch_size=8, epochs=5)

A smaller batch reduces per-step memory demand. It is not always ideal for throughput, but it is often the simplest way to fit the model into available memory.

Model Size and Input Size Matter Too

Large inputs, big dense layers, long sequences, and high-resolution images all drive memory use up quickly. If memory growth is enabled and the error still occurs, inspect:

• Input resolution.
• Sequence length.
• Batch size.
• Model width and depth.
• Intermediate activation sizes.

The real solution may be architectural rather than purely configurational.

Clear the Old Graph State Between Experiments

In notebook-heavy workflows, stale state can also consume memory. Clearing the Keras backend between experiments can help.

import tensorflow as tf

tf.keras.backend.clear_session()

This is especially useful when you rebuild many models in one long-running process.

CPU Fallback Is a Different Tradeoff

If GPU memory is the bottleneck and performance requirements are modest, CPU execution can be a fallback. That is not a performance optimization, but it can be a practical debugging step when you want to separate a GPU-memory issue from a model-design issue.

Mixed Precision Can Also Change Memory Use

On supported hardware, mixed-precision training can reduce memory pressure for some workloads. It is not a universal fix, but it can be part of the solution when the model is close to fitting and batch-size reduction alone is not the only lever you want to use.

Common Pitfalls

• Expecting memory growth to prevent every ResourceExhaustedError.
• Configuring GPU behavior after TensorFlow has already initialized the devices.
• Ignoring batch size even though it is the biggest memory lever in many training jobs.
• Rebuilding models repeatedly in one process without clearing old Keras state.
• Treating a real out-of-memory model as if it were only an allocation-strategy issue.

Summary

• Use TensorFlow GPU memory growth when you want more incremental GPU allocation behavior.
• Configure it before building or using models.
• Memory growth helps with allocation strategy, not with absolute memory limits.
• Reduce batch size and inspect model size if ResourceExhaustedError still occurs.
• Clear Keras session state between experiments to avoid accidental memory accumulation.