Introduction

Legacy TensorFlow 1.x services often share a tf.Session for inference, but thread safety depends on how the graph and session are used.

What Part Needs to Be Safe

There are really three different things in play:

• The TensorFlow graph definition
• The Session object used to execute that graph
• Your surrounding Python service code

In a typical inference service, the graph is built at startup, variables are restored once, and each request calls sess.run(...) with different feed values. That pattern is much safer than building or modifying graph nodes on demand during requests.

Shared Session for Read-Only Inference

The common legacy pattern is:

1import tensorflow as tf
2
3graph = tf.Graph()
4with graph.as_default():
5    x = tf.compat.v1.placeholder(tf.float32, shape=[None, 4], name="x")
6    y = tf.reduce_sum(x, axis=1, name="y")
7
8session = tf.compat.v1.Session(graph=graph)
9
10def predict(batch):
11    return session.run(y, feed_dict={x: batch})

If the graph is fixed and you are only running inference, many services do share a session across worker threads. The important constraint is that requests should not be adding ops, reinitializing variables, or otherwise mutating TensorFlow state while other threads are running.

Where Problems Usually Come From

The dangerous cases are not usually "pure inference on a frozen graph." They are things like:

• Building graph nodes lazily inside request handlers
• Reusing global default graph state without care
• Mixing training-style state updates with inference requests
• Using stateful ops that implicitly change values across runs

That is why people sometimes add a Python lock around sess.run even though the real issue is often the surrounding state model rather than the raw session call itself.

A Conservative Service Pattern

If you want the simplest correctness story, serialize session access explicitly:

1import threading
2
3lock = threading.Lock()
4
5def predict_thread_safe(batch):
6    with lock:
7        return session.run(y, feed_dict={x: batch})

This costs concurrency, but it removes many uncertainty points in older TensorFlow stacks. It is a reasonable compromise when correctness matters more than squeezing every request per second out of a legacy service.

Better Isolation: One Session Per Worker Process

For higher isolation, many production systems avoid a single shared Python process with many threads. Instead they run multiple worker processes, each with its own graph and session. That design:

• Avoids shared Python memory contention
• Keeps inference state isolated per worker
• Fits well with WSGI, Gunicorn, or dedicated model-serving workers

It uses more memory, but the operational model is often cleaner than debugging subtle thread interactions around old TensorFlow 1.x components.

Modern Context: TensorFlow 2.x

In TensorFlow 2.x, tf.Session is no longer the central execution model. New systems usually rely on eager execution, tf.function, or dedicated serving tools. If you are still asking this question, it usually means you are maintaining a legacy graph-based deployment.

That does not mean the service is wrong, but it does mean you should optimize for predictability and explicit ownership of graph/session state.

Common Pitfalls

• Mutating the graph after requests have started is much riskier than merely sharing a built session for inference.
• Relying on the global default graph in a multi-threaded service makes request handling harder to reason about.
• Assuming Python thread safety automatically covers TensorFlow runtime state is unsafe.
• Adding a coarse lock can hide deeper graph-state issues, but it is still a pragmatic short-term protection in legacy services.

Summary

• Sharing a tf.Session for inference can work if the graph is built once and treated as read-only.
• The real thread-safety problems usually come from graph mutation, global state, or stateful execution patterns.
• A lock around sess.run is a conservative legacy-service option.
• For stronger isolation, use separate worker processes or migrate away from session-based TensorFlow where possible.