Introduction

Predicting from a TensorFlow SavedModel starts with knowing how the model was exported. If it was saved from Keras, tf.keras.models.load_model usually gives the simplest inference path. If it was exported as a generic SavedModel, you may instead use tf.saved_model.load and call one of its signatures directly.

Predict with a Keras SavedModel

For Keras models saved with model.save(...), loading and predicting is straightforward.

1import numpy as np
2import tensorflow as tf
3
4model = tf.keras.models.load_model("artifacts/my_keras_model")
5
6inputs = np.array([
7    [0.2, 0.5, 0.1],
8    [0.9, 0.1, 0.3],
9], dtype="float32")
10
11predictions = model.predict(inputs, verbose=0)
12print(predictions)

This works because the Keras export keeps the architecture, weights, and enough metadata for inference.

Use the Same Preprocessing as Training

Loading the model is only part of the job. The input data must be shaped and scaled the same way it was during training.

1import numpy as np
2
3raw = np.array([
4    [20.0, 50.0, 10.0],
5    [90.0, 10.0, 30.0],
6], dtype="float32")
7
8normalized = raw / 100.0
9predictions = model.predict(normalized, verbose=0)
10print(predictions)

If training used normalization, tokenization, image resizing, or feature ordering, inference must do the same. Many "wrong predictions" are really preprocessing mismatches.

Predict from a Generic SavedModel Signature

If the model was exported for serving or is not a normal Keras object, use tf.saved_model.load.

1import tensorflow as tf
2
3loaded = tf.saved_model.load("artifacts/serving_model")
4print(list(loaded.signatures.keys()))
5
6infer = loaded.signatures["serving_default"]
7
8inputs = tf.constant([[0.2, 0.5, 0.1]], dtype=tf.float32)
9outputs = infer(inputs)
10
11for name, tensor in outputs.items():
12    print(name, tensor.numpy())

The signatures dictionary tells you which callable entry points were exported. serving_default is common, but not guaranteed. Inspecting the keys first is safer than assuming the name.

That quick inspection step also helps when the model was exported by another team or by TensorFlow Serving tooling you did not write yourself.

Know the Input Names

Serving signatures can expect named tensors. When that happens, inference looks more like this:

1loaded = tf.saved_model.load("artifacts/serving_model")
2infer = loaded.signatures["serving_default"]
3
4outputs = infer(features=tf.constant([[0.2, 0.5, 0.1]], dtype=tf.float32))
5print(outputs)

The required keyword names come from the exported signature. If you pass unnamed tensors to a signature that expects named inputs, TensorFlow raises an argument error.

Batch Prediction with tf.data

For larger inference jobs, a dataset pipeline is often cleaner than calling predict on ad hoc arrays repeatedly.

1import tensorflow as tf
2
3data = tf.constant([
4    [0.2, 0.5, 0.1],
5    [0.9, 0.1, 0.3],
6    [0.4, 0.6, 0.8],
7], dtype=tf.float32)
8
9dataset = tf.data.Dataset.from_tensor_slices(data).batch(2)
10
11for batch in dataset:
12    print(model(batch, training=False).numpy())

Calling the model directly with training=False is often a good fit when you already have the loaded Keras model object in memory.

That pattern also makes it easier to keep preprocessing, batching, and postprocessing in one explicit pipeline instead of scattering small predict calls throughout an application.

Common Pitfalls

The biggest mistake is mixing up tf.keras.models.load_model and tf.saved_model.load without checking how the model was exported. Both load SavedModel directories, but they return different types and support different inference patterns.

Another issue is ignoring signatures. Generic SavedModels may expect named inputs, and their output keys may not match your assumptions.

Preprocessing mismatch is another major source of confusion. If training normalized features or resized images, the inference path must reproduce that logic exactly.

Finally, do not forget batch shape. Many models expect a leading batch dimension, so a single example often still needs shape (1, feature_count) rather than just (feature_count,).

Summary

• Use tf.keras.models.load_model for normal Keras SavedModel inference.
• Use tf.saved_model.load and exported signatures for generic or serving-oriented SavedModels.
• Inspect signature names and input keys instead of guessing them.
• Keep preprocessing identical between training and prediction.
• Remember that inference inputs usually need a batch dimension.