Inference using saved model in Tensorflow 2 how to control in/output?
Master System Design with Codemia
Enhance your system design skills with over 120 practice problems, detailed solutions, and hands-on exercises.
Introduction
A TensorFlow SavedModel is more than just weights on disk. It also stores callable signatures, and those signatures are what control the names, shapes, and types of the inputs and outputs you see during inference.
Two Common Loading Paths
In TensorFlow 2 there are two normal ways to load a saved model:
- '
tf.keras.models.load_model()when the artifact is a Keras model you want to use like a normal Keras object.' - '
tf.saved_model.load()when you want direct access to exported signatures.'
If your question is specifically about controlling input and output names, tf.saved_model.load() is usually the more useful API because it exposes the signature map.
Inspect the Saved Signatures
After loading, check which signatures exist and what they expect:
This tells you:
- The callable signature names, often including
serving_default. - Input argument names and tensor specs.
- Output names and shapes.
That inspection step is the fastest way to understand how to call the model correctly.
Run Inference With Named Inputs
Suppose the signature expects one input named features. You can then pass the tensor by keyword:
The return value is usually a dictionary-like mapping of output names to tensors. That is already one part of input and output control: use the exported names instead of assuming positional order.
Control the Exported Input and Output Names
If you want predictable names at inference time, define them when saving the model. The cleanest way is to export a tf.function with an explicit input signature and a dictionary return value.
Now the exported signature has a named input called features and named outputs called logits and probabilities.
Keras Models and Simpler Inference
If you do not need signature-level control, loading through Keras is simpler:
This is convenient, but you lose some of the explicit exported signature behavior. For deployment systems that depend on stable input and output names, signature-based loading is usually safer.
Shape and Dtype Still Matter
Controlling names is only half the problem. The tensor you pass must still match the saved contract. If the signature expects float32 with shape [None, 3], then int32 or [3] may fail or produce confusing errors.
A good workflow is:
- Inspect
structured_input_signature. - Build tensors with matching dtype and rank.
- Use keyword arguments with the exported names.
- Read outputs from the returned dictionary by key.
That removes most inference-time ambiguity.
Common Pitfalls
- Loading the model and guessing the input names instead of inspecting the signature.
- Assuming
SavedModeloutputs come back as a single unnamed tensor. - Exporting without an explicit signature and then being surprised by generic tensor names.
- Mixing Keras loading and signature-based loading without understanding the difference.
- Passing tensors with the wrong dtype or shape even when the names are correct.
Summary
- Use
tf.saved_model.load()when you need explicit control over inference inputs and outputs. - Inspect
loaded.signaturesandstructured_input_signaturebefore calling the model. - Export explicit signatures with named
TensorSpecinputs and dictionary outputs for predictable serving behavior. - Use keyword arguments during inference instead of relying on positional guesses.
- Correct names do not replace correct shapes and dtypes; both parts must match the exported contract.

