Understanding `frozen_inference_graph.pb` and `saved_model.pb`

When working with TensorFlow models, particularly in the context of deployment and inference, you might come across files like `frozen_inference_graph.pb` and `saved_model.pb`. While at a high level, both files are related to storing optimized TensorFlow models, they serve different purposes and have distinct characteristics.

1. Model Formats in TensorFlow

Before delving into the specifics about each file, it's essential to understand the model formats associated with TensorFlow.

• Protobuf (`.pb`) Format: This is the native format for storing serialized TensorFlow graphs and weights. The `.pb` file includes both the model graph and the weights.
• SavedModel: A comprehensive format that includes not just the computation graph and weights (similar to `.pb` files) but also additional metadata, serving configurations, and other customizable objects that make it versatile for production deployment.

2. Overview of `frozen_inference_graph.pb`

Characteristics:

• Frozen State: As the name suggests, a `frozen_inference_graph.pb` is a frozen version of a TensorFlow model. This means that the computational graph and its associated variable values (weights) are stored in a single file where variable nodes are turned into constants.
• Immutability: "Frozen" implies that the graph cannot be altered—training on this graph is not possible. It is optimized solely for inference.
• Single Graph Definition: It includes only one graph—there's no capability to manage multiple metagraphs within a single `.pb`.

Common Use Cases:

• Deployment: Primarily used for deploying pre-trained models where further customization or training is unnecessary.
• Performance Optimization: As the weights are converted into constants, the file size may be reduced, improving inference performance.

3. Overview of `saved_model.pb`

Characteristics:

• Dynamic Nature: Unlike `frozen_inference_graph.pb`, `saved_model.pb` can encapsulate multiple graph definitions (meta graphs). This feature allows it to support a more dynamic aspect of models.
• Includes Exported Functions: Apart from the weights and graph, `saved_model.pb` can include serialized signatures of exported functions—entry points for inference.
• Versatility: The `SavedModel` format is designed for ease of deployment across different environments and supports various languages like Python, C++.

Common Use Cases:

• Advanced Deployment Scenarios: Ideal for scenarios where serving needs not just the computational graph, but also additional configuration which can include pre/post-processing hooks, custom layers, and contexts.
• Model Versioning and Management: Supports rollback and multiple versions of models which is beneficial for continuous integration pipelines.

4. Technical Comparison

Below is a summary table highlighting the key similarities and differences between `frozen_inference_graph.pb` and `saved_model.pb`.

5. Practical Example with TensorFlow

Suppose you trained a TensorFlow model and now intend to deploy it:

• Option 1 - Using `frozen_inference_graph.pb`:
  • Convert the trained model to a `frozen` graph using TensorFlow utilities such as the `freeze_graph.py` script.
  • Use tools like TensorFlow Model Optimization Toolkit to further optimize for performance if needed.
• Option 2 - Using `saved_model.pb`:
  • Create a `SavedModel` using the `tf.saved_model.save` function.
  • This approach is more suited if you anticipate having future access requirements to model internals or wish to deploy using TensorFlow Serving or TensorFlow.js.