Introduction

Keras models already run on top of TensorFlow, so in modern TensorFlow the simplest way to make predictions is usually just model.predict(...) or model(...). But sometimes you specifically want graph-based execution for export, serving, tracing, or interoperability with lower-level TensorFlow APIs.

The important distinction is whether you mean "use the model in graph mode" or "load a serialized TensorFlow graph artifact produced from a Keras model." Both are possible, but the workflow is different.

Graph Execution in Modern TensorFlow

In TensorFlow 2, eager execution is the default. To run prediction logic as a traced graph, wrap the model call with tf.function.

1import tensorflow as tf
2import numpy as np
3
4model = tf.keras.Sequential([
5    tf.keras.layers.Dense(8, activation="relu"),
6    tf.keras.layers.Dense(1),
7])
8
9model.build((None, 4))
10
11@tf.function
12def predict_graph(x):
13    return model(x, training=False)
14
15sample = tf.constant(np.random.rand(3, 4), dtype=tf.float32)
16predictions = predict_graph(sample)
17print(predictions)

This does not require an explicit session. TensorFlow traces the computation into a graph behind the scenes.

Exporting a Keras Model as a SavedModel

If you want to persist the model in TensorFlow graph form for later inference, export it as a SavedModel:

1import tensorflow as tf
2
3model = tf.keras.Sequential([
4    tf.keras.layers.Input(shape=(4,)),
5    tf.keras.layers.Dense(8, activation="relu"),
6    tf.keras.layers.Dense(1),
7])
8
9model.save("saved_model_dir")

You can later load it with TensorFlow and call its serving signature:

1import tensorflow as tf
2import numpy as np
3
4loaded = tf.saved_model.load("saved_model_dir")
5infer = loaded.signatures["serving_default"]
6
7inputs = tf.constant(np.random.rand(2, 4), dtype=tf.float32)
8outputs = infer(inputs)
9print(outputs)

This is the modern graph-oriented deployment path derived from a Keras model.

Predicting from a Loaded SavedModel Signature

When you call loaded.signatures["serving_default"], you are invoking a concrete TensorFlow graph function. That is usually the cleanest answer to "how do I make predictions from a TensorFlow graph created from my Keras model."

If you need to inspect available outputs:

print(infer.structured_outputs)

That helps because SavedModel signatures often return dictionaries keyed by output tensor names rather than a plain tensor.

When Older Graph-and-Session Code Appears

You may still encounter older TensorFlow 1 style examples that:

• load a frozen graph
• create a tf.compat.v1.Session
• fetch input and output tensors by name
• call session.run(...)

That style is useful mainly for maintaining legacy inference pipelines. Conceptually it looks like this:

1import tensorflow as tf
2
3tf.compat.v1.disable_eager_execution()
4
5with tf.io.gfile.GFile("frozen_graph.pb", "rb") as f:
6    graph_def = tf.compat.v1.GraphDef()
7    graph_def.ParseFromString(f.read())
8
9graph = tf.Graph()
10with graph.as_default():
11    tf.import_graph_def(graph_def, name="")
12
13input_tensor = graph.get_tensor_by_name("input_1:0")
14output_tensor = graph.get_tensor_by_name("Identity:0")
15
16with tf.compat.v1.Session(graph=graph) as sess:
17    predictions = sess.run(output_tensor, feed_dict={
18        input_tensor: [[0.1, 0.2, 0.3, 0.4]]
19    })
20    print(predictions)

This is valid for legacy setups, but if you control the pipeline today, SavedModel and tf.function are usually clearer.

Choosing the Right Path

Use direct Keras prediction if:

• you already have the model object in memory
• you do not need low-level graph handling
• you just want inference in normal TensorFlow 2 code

Use tf.function if:

• you want graph execution without leaving TensorFlow 2
• you need tracing, optimization, or concrete functions

Use SavedModel loading if:

• the model has been exported already
• inference happens in another process or environment
• you need TensorFlow-native graph signatures for serving

Common Pitfalls

The biggest pitfall is mixing TensorFlow 1 session-based examples into a TensorFlow 2 workflow without realizing they solve a legacy problem.

Another issue is assuming a SavedModel signature returns a single raw tensor. It often returns a dictionary of named outputs, so inspect the structure before indexing into it blindly.

Developers also sometimes call the Keras model with training=True during inference by mistake, which can change dropout or batch normalization behavior. For predictions, use training=False.

Finally, if you export and reload the model, keep input shapes and dtypes consistent. Prediction failures after loading are often caused by mismatched input signatures rather than by the graph itself.

Summary

• In modern TensorFlow, graph-style prediction from a Keras model is usually done with tf.function or a SavedModel signature.
• 'model.predict(...) is simplest when you already have the Keras model in memory.'
• 'tf.saved_model.load(...).signatures["serving_default"] is the common path for graph-based loaded inference.'
• Legacy frozen-graph and session code still exists, but it is mainly for older TensorFlow pipelines.
• Always verify output structure, input dtype, and training=False behavior during inference.