Introduction

If you want to run a TensorFlow program repeatedly "from scratch", the real question is which state you need to reset. In modern TensorFlow, the usual sources of carry-over are Keras global state, random seeds, model objects kept alive in Python, and sometimes process-level resources such as GPU memory allocations.

Build the Model Inside a Function

The first rule is simple: create a new model for each run instead of reusing the old object. Wrapping the full experiment in a function gives each iteration a clean boundary.

1import numpy as np
2import tensorflow as tf
3
4
5def run_once(seed: int) -> float:
6    tf.keras.backend.clear_session()
7    tf.keras.utils.set_random_seed(seed)
8
9    x = np.array([[0.0], [1.0], [2.0], [3.0]], dtype=np.float32)
10    y = np.array([[0.0], [2.0], [4.0], [6.0]], dtype=np.float32)
11
12    model = tf.keras.Sequential([
13        tf.keras.layers.Input(shape=(1,)),
14        tf.keras.layers.Dense(1),
15    ])
16
17    model.compile(optimizer="sgd", loss="mse")
18    model.fit(x, y, epochs=50, verbose=0)
19
20    loss = model.evaluate(x, y, verbose=0)
21    return float(loss)
22
23
24for i in range(3):
25    print(i, run_once(seed=1234 + i))

This handles the most common case well. A fresh model is created each time, Keras state is cleared, and the seed is reset so runs are reproducible when that is what you want.

What clear_session() Actually Resets

tf.keras.backend.clear_session() resets Keras-managed global state such as layer name counters and references held by the Keras backend. It is especially useful when you build many models in a loop, because otherwise memory usage can grow over time.

What it does not do is magically erase every piece of state in your Python process. If you still keep references to old tensors, datasets, callbacks, or model objects, those can continue to exist until garbage collection removes them.

That means a clean rerun usually requires three things:

• create fresh model objects
• reset the seed if reproducibility matters
• avoid keeping old references in outer scopes

Use a Separate Process for Full Isolation

If you need a truly fresh run every time, process isolation is the most reliable approach. This matters when:

• GPU memory behavior becomes sticky
• native libraries hold process-level state
• one run can crash or corrupt the next
• you want strict isolation for benchmarking

A simple launcher can run the training script in a new Python process:

1import subprocess
2import sys
3
4
5for _ in range(3):
6    subprocess.run([sys.executable, "train_once.py"], check=True)

This is heavier than a function call, but it is the closest thing to "start over completely" without restarting your whole environment manually.

TensorFlow 1.x and Graph Resetting

If you are maintaining TensorFlow 1.x code or compatibility-mode graph code, graph reset also matters. In that older style, sessions and graphs were explicit runtime objects, so a fresh execution often meant both:

import tensorflow as tf

tf.compat.v1.reset_default_graph()

Then you would rebuild the graph and create a new session. In TensorFlow 2.x, eager execution and Keras-style model construction make that pattern less central, but you may still see it in legacy projects.

Decide Whether You Want Reproducibility or Diversity

Sometimes "afresh" means "independent random initialization every run." Other times it means "same starting conditions every run." Those are different goals.

For reproducible runs, set a known seed.

For genuinely different restarts, do not reuse the same seed each time.

Both are valid. The mistake is assuming that repeated calls without explicit seeding or cleanup are either fully reproducible or fully isolated. They are often neither.

Common Pitfalls

The biggest pitfall is reusing the same model instance and expecting fit() to behave like a brand-new experiment. It will continue training from the current weights.

Another common problem is forgetting that notebooks keep references alive. Even after calling clear_session(), variables in cells can still point at older objects.

GPU users often assume memory release is immediate. In practice, process-level isolation is sometimes the only dependable reset when running many experiments.

Finally, legacy TensorFlow advice about sessions and graphs is not always appropriate for TensorFlow 2.x. Make sure the reset strategy matches the programming model your code actually uses.

Summary

• Build a new model object for every run.
• Use tf.keras.backend.clear_session() when creating models in loops.
• Reset seeds explicitly if reproducibility matters.
• Use a separate process when you need strong isolation.
• Distinguish between TensorFlow 2.x Keras workflows and legacy TensorFlow 1.x graph workflows.