Introduction

If the "same" model converges in tf.keras but not in older standalone keras, the cause is usually not magic. It is almost always a difference in versions, defaults, random seeds, backend behavior, numerical precision, or data preprocessing. The fastest way to debug it is to stop treating the two environments as identical and compare them layer by layer, optimizer setting by optimizer setting, and batch by batch.

Start with the Most Likely Cause: They Are Not Actually the Same Stack

Historically, keras and tf.keras looked similar from the API level, but they were not always identical implementations. Even if the model code appears the same, the following can differ:

• optimizer defaults such as epsilon values or decay behavior
• layer implementations, especially recurrent layers and batch normalization
• random initialization paths
• backend graph execution details
• handling of channels ordering and data types
• CuDNN-accelerated kernels versus fallback kernels

So the first engineering move is not to tune blindly. It is to pin versions and print the environment clearly.

1import os
2import random
3import numpy as np
4import tensorflow as tf
5
6os.environ["PYTHONHASHSEED"] = "0"
7random.seed(0)
8np.random.seed(0)
9tf.random.set_seed(0)
10
11print("TensorFlow:", tf.__version__)
12print("Keras from tf:", tf.keras.__version__)
13print("GPU devices:", tf.config.list_physical_devices('GPU'))

If you are truly comparing standalone keras against tf.keras, record both package versions and the backend version. Without that, "exact model" is not a meaningful claim.

Compare the Training Inputs First

A large share of convergence bugs are really data bugs. If one stack shuffles differently, casts to a different dtype, normalizes in a different place, or applies augmentation in a different order, training curves can diverge fast.

Useful checks include:

• verify input tensor shapes are identical
• verify label encoding is identical
• print batch mean and standard deviation in both runs
• verify training and validation splits are the same
• disable augmentation temporarily

A simple sanity check is to overfit on a tiny subset.

1import tensorflow as tf
2from tensorflow import keras
3
4x = tf.random.normal((32, 10))
5y = tf.cast(tf.reduce_sum(x, axis=1) > 0, tf.float32)
6
7model = keras.Sequential([
8    keras.layers.Dense(16, activation='relu', input_shape=(10,)),
9    keras.layers.Dense(1, activation='sigmoid')
10])
11
12model.compile(optimizer=keras.optimizers.Adam(learning_rate=1e-3),
13              loss='binary_crossentropy',
14              metrics=['accuracy'])
15
16history = model.fit(x, y, epochs=50, verbose=0)
17print(history.history['loss'][-1], history.history['accuracy'][-1])

If one framework cannot overfit a tiny dataset and the other can, you likely have an optimizer, initialization, or numerical mismatch rather than a generalization problem.

Normalize the Optimizer and Initialization Settings

When convergence differs, I would compare these settings explicitly instead of relying on defaults:

• learning rate
• epsilon
• beta parameters for Adam-like optimizers
• gradient clipping
• kernel initializer
• bias initializer
• loss reduction mode

For example, define the optimizer manually rather than accepting library defaults:

1optimizer = tf.keras.optimizers.Adam(
2    learning_rate=1e-3,
3    beta_1=0.9,
4    beta_2=0.999,
5    epsilon=1e-7
6)

Do the same for layer initializers if reproducibility matters. Default values have changed across releases, and subtle numeric shifts can matter in unstable models.

Watch for Backend-Specific Layers

Older keras plus a TensorFlow backend and modern tf.keras may treat some layers differently under the hood. Recurrent layers, convolutions on GPU, and batch normalization have historically been sensitive areas.

Batch normalization is especially worth testing. If your model is unstable, temporarily remove it or replace it with a simpler architecture. If convergence suddenly becomes consistent, you have narrowed the problem to a layer with training-mode state or numerical sensitivity.

Also check whether one environment is silently using float64 while the other uses float32, or whether mixed precision is enabled in only one run.

Use a Minimal Reproduction Instead of the Full Pipeline

The most productive debugging workflow is:

1. fix seeds
2. use one small dataset slice
3. remove callbacks
4. remove augmentation
5. reduce the model to a few layers
6. compare losses after each epoch

At that point, compare first-batch predictions and first gradient step if needed. That sounds tedious, but it is how you separate framework differences from hidden application differences.

If the numbers diverge on the first batch, the issue is in initialization, preprocessing, or forward/backward computation. If they diverge later, look at stateful layers, callbacks, and data order.

Common Pitfalls

The biggest mistake is saying two training runs are identical because the Python model definition matches. Identical source code does not imply identical optimizer defaults, kernels, or execution behavior.

Another mistake is debugging convergence before verifying that both environments see identical input data and labels.

People also leave too many moving parts enabled at once: augmentation, callbacks, learning-rate schedules, multiprocessing data loaders, and mixed precision. Strip the pipeline down first.

Finally, do not compare an old standalone keras package to a new TensorFlow release and expect them to behave the same. Version skew alone can explain the result.

Summary

• Different convergence between keras and tf.keras usually comes from version or default-setting differences, not mystery behavior
• Verify seeds, package versions, backend versions, and device placement before tuning
• Check that preprocessing, shuffling, labels, and dtypes are actually identical
• Define optimizer and initializer settings explicitly instead of relying on defaults
• Use a tiny overfitting test and a minimal reproduction to localize the mismatch
• Treat layer implementations such as batch normalization and recurrent layers as likely sources of divergence