Introduction

Google Cloud ML Engine, later called AI Platform, expects a TensorFlow SavedModel when serving predictions. If you only have a frozen .pb graph, you must wrap that graph in a serving signature so the platform knows which tensors are inputs and outputs.

What A Frozen Graph Is Missing

A frozen graph stores the computation graph and baked-in weights in a single protobuf file. That is useful for inference, but it does not give a serving system everything it needs. In particular, cloud serving expects:

• a model directory, not a single .pb file
• a serving_default signature
• named inputs and outputs
• tensor metadata that prediction tools can inspect

That is why converting to SavedModel is more than a file rename. You are packaging the graph for serving.

Identify Input And Output Tensor Names

Before conversion, confirm the tensor names from the original graph. If you guess wrong here, the exported model will load but predictions will fail.

One quick inspection script is:

1import tensorflow as tf
2
3graph_def = tf.compat.v1.GraphDef()
4with tf.io.gfile.GFile("frozen_model.pb", "rb") as f:
5    graph_def.ParseFromString(f.read())
6
7for node in graph_def.node[:20]:
8    print(node.name, node.op)

You are usually looking for placeholder tensors such as input_tensor:0 and final outputs such as probabilities:0 or predictions:0.

Export The Frozen Graph As A SavedModel

Frozen graphs are commonly from TensorFlow 1.x workflows, so the compatibility APIs are the most practical way to build the export.

1import tensorflow as tf
2
3FROZEN_GRAPH = "frozen_model.pb"
4EXPORT_DIR = "saved_model/1"
5INPUT_TENSOR_NAME = "input_tensor:0"
6OUTPUT_TENSOR_NAME = "probabilities:0"
7
8graph_def = tf.compat.v1.GraphDef()
9with tf.io.gfile.GFile(FROZEN_GRAPH, "rb") as f:
10    graph_def.ParseFromString(f.read())
11
12with tf.compat.v1.Session(graph=tf.Graph()) as sess:
13    tf.import_graph_def(graph_def, name="")
14
15    input_tensor = sess.graph.get_tensor_by_name(INPUT_TENSOR_NAME)
16    output_tensor = sess.graph.get_tensor_by_name(OUTPUT_TENSOR_NAME)
17
18    builder = tf.compat.v1.saved_model.builder.SavedModelBuilder(EXPORT_DIR)
19
20    signature = tf.compat.v1.saved_model.signature_def_utils.predict_signature_def(
21        inputs={"inputs": input_tensor},
22        outputs={"scores": output_tensor},
23    )
24
25    builder.add_meta_graph_and_variables(
26        sess,
27        [tf.saved_model.SERVING],
28        signature_def_map={"serving_default": signature},
29        clear_devices=True,
30    )
31    builder.save()
32
33print(f"SavedModel exported to {EXPORT_DIR}")

After this runs, the output directory should contain saved_model.pb plus a variables directory. Even when the original graph was frozen, TensorFlow still writes the standard serving layout.

Verify The Export Before Uploading

Do not upload the model blind. Inspect the exported signatures first:

saved_model_cli show --dir saved_model/1 --all

You should see the serving_default signature and the exact input and output tensor names you expect. If the names are wrong here, they will also be wrong in the cloud endpoint.

Match The Export To The Serving Runtime

A common source of confusion is exporting with one TensorFlow version and serving with another. Frozen graphs from old training jobs often depend on TensorFlow 1.x semantics. If the target runtime expects a significantly different version, you can get load-time failures even though the conversion code succeeded locally.

When you are working with a legacy Cloud ML Engine model, use a runtime compatible with the graph you trained. If the model is new enough to run in modern TensorFlow, consider re-exporting from the original training code instead of converting a frozen graph by hand.

Common Pitfalls

The biggest mistake is guessing tensor names instead of inspecting the graph. An incorrect signature can produce a valid-looking export that still fails at prediction time.

Another common issue is uploading the .pb file alone. Cloud serving expects a full SavedModel directory, not just a raw graph protobuf.

Developers also often export from a training graph rather than a clean inference graph, which can leave unnecessary placeholders or training-only behavior in the serving path.

Finally, keep TensorFlow version compatibility in mind. A local export that succeeds does not guarantee that an older or newer serving runtime will load it cleanly.

Summary

• A frozen .pb graph must be wrapped in a SavedModel with a serving signature for Cloud ML Engine style prediction serving.
• The most important inputs to the conversion are the correct tensor names.
• Use TensorFlow compatibility APIs for older frozen graphs, then verify the result with saved_model_cli.
• Upload the exported model directory, not the original .pb file by itself.
• Keep the serving runtime aligned with the TensorFlow version used by the graph whenever possible.