Introduction

TensorFlow Serving serves TensorFlow SavedModel artifacts. It does not natively execute PyTorch modules or scikit-learn estimators. So the honest answer is: you cannot directly point TensorFlow Serving at a .pt file or a pickled sklearn model. You either convert the model into a real TensorFlow SavedModel, or you use a serving system that supports the original framework more naturally.

Know the Constraint Up Front

TensorFlow Serving expects TensorFlow graphs and signatures. That means the deployed model must be something TensorFlow runtime can load without Python.

This rules out direct serving of:

• native PyTorch checkpoints
• pickle files from sklearn
• Python-only wrapper code using tf.py_function

If the serving path depends on Python code running arbitrary library objects, it is not a TensorFlow Serving deployment anymore.

The Practical Options

For PyTorch or sklearn, the realistic choices are:

1. convert to TensorFlow SavedModel
2. convert to ONNX and serve with an ONNX-capable server
3. use a framework-native server such as TorchServe for PyTorch

TensorFlow Serving is only the right tool if you truly end up with a valid SavedModel.

PyTorch Path: Export, Then Convert Carefully

PyTorch can export to ONNX easily:

1import torch
2import torch.nn as nn
3
4class Net(nn.Module):
5    def __init__(self):
6        super().__init__()
7        self.linear = nn.Linear(10, 2)
8
9    def forward(self, x):
10        return self.linear(x)
11
12model = Net().eval()
13dummy = torch.randn(1, 10)
14
15torch.onnx.export(
16    model,
17    dummy,
18    "model.onnx",
19    input_names=["input"],
20    output_names=["output"],
21    opset_version=17
22)

That gives you an ONNX model, not a TensorFlow Serving-ready artifact. The next step would be converting ONNX into a TensorFlow representation and then exporting a SavedModel. That path can work for some models, but it is fragile if the operator set does not translate cleanly.

So for PyTorch, the better operational question is often whether TensorFlow Serving is the right target at all.

sklearn Path: TensorFlow Serving Is Usually the Wrong Tool

scikit-learn models are usually easier to serve with:

• a small Python API
• ONNX Runtime
• MLflow
• BentoML
• KServe with a compatible predictor

You can convert many sklearn models to ONNX:

1from sklearn.linear_model import LogisticRegression
2from sklearn.datasets import load_iris
3from skl2onnx import to_onnx
4import numpy as np
5
6X, y = load_iris(return_X_y=True)
7model = LogisticRegression(max_iter=200).fit(X, y)
8
9onx = to_onnx(model, X[:1].astype(np.float32), target_opset=17)
10with open("model.onnx", "wb") as f:
11    f.write(onx.SerializeToString())

But again, ONNX is not TensorFlow SavedModel. It is usually better to serve ONNX with ONNX-capable tooling than to force an unnecessary cross-framework conversion.

What "Convert to SavedModel" Really Means

A valid TensorFlow Serving deployment needs:

• TensorFlow operations only
• a SavedModel directory
• one or more serving signatures

For a simple TensorFlow model, that looks like:

1import tensorflow as tf
2
3model = tf.keras.Sequential([
4    tf.keras.layers.Input(shape=(10,)),
5    tf.keras.layers.Dense(2)
6])
7
8tf.saved_model.save(model, "saved_model/my_model")

This is straightforward because the model is already TensorFlow. For PyTorch or sklearn, you only reach this stage if you have truly converted the computation into TensorFlow-compatible form.

If You Must Use One Serving Stack

If an organization requires one serving platform, the cleanest strategy is usually to standardize on a neutral model format and a serving system built for it. For example:

• ONNX plus ONNX Runtime Server
• KServe with multiple predictor types

That is usually cleaner than forcing every non-TensorFlow model through TensorFlow Serving-specific conversion.

Common Pitfalls

The biggest mistake is assuming TensorFlow Serving can run arbitrary Python models if you wrap them somehow. It cannot. Another is converting PyTorch or sklearn to ONNX and then assuming the job is done for TensorFlow Serving. ONNX is a separate runtime format, not a TensorFlow SavedModel. Developers also use Python callbacks such as tf.py_function in a wrapper, forgetting that TensorFlow Serving does not run arbitrary Python during inference. If the model is not truly TensorFlow by deployment time, TensorFlow Serving is the wrong endpoint.

Summary

• TensorFlow Serving natively serves TensorFlow SavedModel, not raw PyTorch or sklearn artifacts.
• PyTorch models can often export to ONNX, but that still is not directly usable by TensorFlow Serving.
• sklearn models are usually better served with ONNX-capable or Python-native serving tools.
• Use TensorFlow Serving only if the model has been genuinely converted into a TensorFlow SavedModel.
• Do not rely on Python wrappers or tf.py_function for TensorFlow Serving deployment.
• If you need one cross-framework serving platform, choose a neutral serving stack instead of forcing everything through TensorFlow Serving.