tensorflow
reproducibility
random seeds
machine learning
neural networks

Which seeds have to be set where to realize 100 reproducibility of training results in tensorflow?

Master System Design with Codemia

Enhance your system design skills with over 120 practice problems, detailed solutions, and hands-on exercises.

Introduction

In TensorFlow, setting one seed is not enough to make training deterministic. You need to control randomness across Python, NumPy, and TensorFlow, and even then you should be careful with hardware, versions, and nondeterministic ops. The honest answer is that you can greatly improve reproducibility, but "100 percent reproducibility" only applies when the full environment is kept stable too.

What Needs to Be Controlled

Training results can change because of:

  • Python random state
  • NumPy random state
  • TensorFlow random ops
  • data pipeline shuffling
  • nondeterministic GPU kernels
  • thread scheduling
  • different library or driver versions

That is why reproducibility is a system-level property, not just a seed-setting trick.

The Current TensorFlow Starting Point

The simplest modern setup is:

python
1import tensorflow as tf
2
3tf.keras.utils.set_random_seed(1234)
4tf.config.experimental.enable_op_determinism()

tf.keras.utils.set_random_seed sets the Python, NumPy, and TensorFlow seeds together. Enabling op determinism asks TensorFlow to prefer deterministic behavior for supported operations.

A fuller example looks like this:

python
1import os
2import tensorflow as tf
3
4os.environ["PYTHONHASHSEED"] = "1234"
5
6tf.keras.utils.set_random_seed(1234)
7tf.config.experimental.enable_op_determinism()
8
9model = tf.keras.Sequential([
10    tf.keras.layers.Dense(16, activation="relu"),
11    tf.keras.layers.Dense(1),
12])
13
14model.compile(optimizer="adam", loss="mse")

This is a strong default for reproducible Keras experiments.

Data Pipelines Matter Too

If your input pipeline shuffles data, the shuffle must also be deterministic. For example:

python
1import tensorflow as tf
2
3x = tf.constant([[1.0], [2.0], [3.0], [4.0]])
4y = tf.constant([[2.0], [4.0], [6.0], [8.0]])
5
6dataset = tf.data.Dataset.from_tensor_slices((x, y))
7dataset = dataset.shuffle(
8    buffer_size=4,
9    seed=1234,
10    reshuffle_each_iteration=False,
11).batch(2)

If you omit the seed or leave reshuffle_each_iteration=True, each epoch may see examples in a different order.

What Seeds Do Not Solve

Seeds alone do not guarantee identical training across:

  • different TensorFlow versions
  • different CUDA or cuDNN versions
  • different CPUs or GPUs
  • custom ops with nondeterministic behavior

Even memory pressure and thread scheduling can affect whether a run fails or succeeds. So if you need a truly repeatable experiment, keep the environment pinned:

  • same TensorFlow version
  • same Python version
  • same device type
  • same driver stack
  • same dataset and preprocessing code

A Practical Reproducibility Checklist

For most projects, do all of the following:

  1. call tf.keras.utils.set_random_seed(...)
  2. call tf.config.experimental.enable_op_determinism()
  3. seed dataset shuffles explicitly
  4. keep software and hardware fixed
  5. avoid nondeterministic custom ops

That gets you much closer to repeatable results than only calling tf.random.set_seed.

When Results Still Differ

If two runs still diverge, check:

  • whether the data loader uses multiprocessing or nondeterministic ordering
  • whether augmentation code calls Python or NumPy random APIs outside the seeded path
  • whether the model uses layers or ops with unsupported deterministic behavior
  • whether you changed the environment between runs

In other words, reproducibility failures are often caused by untracked side inputs, not by the seed value itself.

Common Pitfalls

The biggest mistake is setting only TensorFlow's seed and forgetting Python and NumPy. Randomness leaks in from all three.

Another mistake is assuming the same seed guarantees identical results across different machines. It does not if the runtime environment differs.

A third issue is enabling shuffle in tf.data but forgetting to make that shuffle deterministic.

Summary

  • There is no single TensorFlow seed that guarantees full reproducibility by itself.
  • Use tf.keras.utils.set_random_seed(...) to seed Python, NumPy, and TensorFlow together.
  • Enable deterministic ops with tf.config.experimental.enable_op_determinism().
  • Make dataset shuffling deterministic and keep the software and hardware environment fixed.
  • "100 percent reproducibility" requires environment control, not just seed control.

Course illustration
Course illustration

All Rights Reserved.