Introduction
In TensorFlow, you can run part of a computation graph once (e.g., an expensive embedding or feature extraction) and then feed that result into another part of the graph repeatedly in a loop. In TF1, use session.run() to evaluate the first subgraph, then pass the result via feed_dict to the second subgraph in a loop. In TF2 (eager mode), simply call the first function once, store the result in a variable, and pass it to the second function in a loop. This pattern is common when a shared encoder processes input once and a decoder or classifier runs multiple times with different parameters.
TensorFlow 2 (Eager Mode)
1import tensorflow as tf
2
3# Expensive feature extraction — run once
4encoder = tf.keras.Sequential([
5 tf.keras.layers.Dense(256, activation='relu'),
6 tf.keras.layers.Dense(128, activation='relu'),
7])
8
9# Lightweight classifier head — run many times
10classifier = tf.keras.layers.Dense(10, activation='softmax')
11
12# Run encoder once
13input_data = tf.random.normal([1, 784])
14features = encoder(input_data) # Compute once, reuse many times
15
16# Run classifier in a loop with different parameters or inputs
17for temperature in [0.5, 1.0, 2.0, 5.0]:
18 logits = classifier(features)
19 scaled = logits / temperature
20 predictions = tf.nn.softmax(scaled)
21 print(f"Temperature {temperature}: {predictions.numpy()[:, :3]}")
TensorFlow 2 with tf.function
For performance-critical code, wrap the loop body in tf.function:
1encoder = tf.keras.Sequential([
2 tf.keras.layers.Conv2D(32, 3, activation='relu'),
3 tf.keras.layers.GlobalAveragePooling2D(),
4 tf.keras.layers.Dense(128, activation='relu'),
5])
6
7decoder = tf.keras.Sequential([
8 tf.keras.layers.Dense(256, activation='relu'),
9 tf.keras.layers.Dense(784, activation='sigmoid'),
10])
11
12@tf.function
13def decode_with_noise(features, noise_scale):
14 noisy_features = features + tf.random.normal(tf.shape(features)) * noise_scale
15 return decoder(noisy_features)
16
17# Encode once (expensive convolutions)
18images = tf.random.normal([16, 28, 28, 1])
19features = encoder(images)
20
21# Decode multiple times with different noise levels
22for scale in [0.0, 0.1, 0.5, 1.0]:
23 reconstructions = decode_with_noise(features, scale)
24 mse = tf.reduce_mean(tf.square(reconstructions - tf.reshape(images, [16, 784])))
25 print(f"Noise scale {scale}: MSE = {mse.numpy():.4f}")
TensorFlow 1.x (Session-Based)
1import tensorflow.compat.v1 as tf
2tf.disable_eager_execution()
3
4# Define the graph
5input_placeholder = tf.placeholder(tf.float32, [None, 784])
6
7# Part 1: Encoder (run once)
8with tf.variable_scope('encoder'):
9 hidden = tf.layers.dense(input_placeholder, 256, activation=tf.nn.relu)
10 features = tf.layers.dense(hidden, 128, activation=tf.nn.relu)
11
12# Part 2: Decoder (run in a loop with different inputs)
13features_placeholder = tf.placeholder(tf.float32, [None, 128])
14noise_placeholder = tf.placeholder(tf.float32, [])
15
16with tf.variable_scope('decoder'):
17 noisy = features_placeholder + tf.random.normal(tf.shape(features_placeholder)) * noise_placeholder
18 output = tf.layers.dense(noisy, 784, activation=tf.nn.sigmoid)
19
20with tf.Session() as sess:
21 sess.run(tf.global_variables_initializer())
22
23 # Run encoder once
24 input_data = [[1.0] * 784] # Example input
25 encoded = sess.run(features, feed_dict={input_placeholder: input_data})
26
27 # Run decoder in a loop with different noise levels
28 for noise in [0.0, 0.1, 0.5, 1.0]:
29 decoded = sess.run(output, feed_dict={
30 features_placeholder: encoded,
31 noise_placeholder: noise
32 })
33 print(f"Noise {noise}: output shape = {decoded.shape}")
Practical Pattern: Batch Inference with Shared Encoder
1import tensorflow as tf
2import numpy as np
3
4# Pre-trained encoder (e.g., ResNet feature extractor)
5base_model = tf.keras.applications.ResNet50(
6 weights='imagenet', include_top=False, pooling='avg'
7)
8base_model.trainable = False
9
10# Multiple task heads
11classification_head = tf.keras.layers.Dense(100, activation='softmax')
12detection_head = tf.keras.Sequential([
13 tf.keras.layers.Dense(256, activation='relu'),
14 tf.keras.layers.Dense(4) # Bounding box coordinates
15])
16
17# Extract features once (expensive)
18images = tf.random.normal([8, 224, 224, 3])
19features = base_model(images, training=False) # Run once
20
21# Run multiple heads on the same features
22class_preds = classification_head(features)
23bbox_preds = detection_head(features)
24
25print(f"Features: {features.shape}") # (8, 2048)
26print(f"Classes: {class_preds.shape}") # (8, 100)
27print(f"Bounding boxes: {bbox_preds.shape}") # (8, 4)
1# Common pattern: encode database once, search in a loop
2encoder = tf.keras.Sequential([
3 tf.keras.layers.Dense(128, activation='relu'),
4 tf.keras.layers.Dense(64),
5 tf.keras.layers.Lambda(lambda x: tf.nn.l2_normalize(x, axis=1))
6])
7
8# Encode the database once (expensive)
9database_items = tf.random.normal([10000, 784])
10database_embeddings = encoder(database_items) # Shape: (10000, 64)
11
12# Search in a loop (cheap — just dot products)
13queries = tf.random.normal([5, 784])
14for i in range(len(queries)):
15 query_embedding = encoder(queries[i:i+1])
16 similarities = tf.matmul(query_embedding, database_embeddings, transpose_b=True)
17 top_k = tf.math.top_k(similarities[0], k=5)
18 print(f"Query {i}: top matches at indices {top_k.indices.numpy()}")
Common Pitfalls
Re-running the encoder inside the loop: If you accidentally put the encoder call inside the loop, you recompute features every iteration, wasting time. Always store the encoder output in a variable before the loop and reuse it.
TF1: Using the wrong placeholder in feed_dict: In TF1, feeding encoded back requires a separate placeholder for the intermediate result (features_placeholder), not the original input_placeholder. Mixing these up feeds wrong-shaped data and causes dimension mismatch errors.
Not setting training=False on frozen encoder layers: Layers like BatchNormalization and Dropout behave differently in training vs inference mode. When reusing encoder features, pass training=False to ensure consistent outputs. Forgetting this causes different results per call due to dropout randomness.
Memory issues with large cached tensors: Caching encoder output for a large dataset (millions of items) may exceed GPU memory. Use tf.data.Dataset.map() to encode in batches and store results as NumPy arrays or on disk, not as a single tensor in memory.
Graph disconnection in TF1 when using separate placeholders: If the decoder subgraph references variables from the encoder subgraph, you must initialize both before running. Using tf.global_variables_initializer() handles this, but selectively initializing only decoder variables leaves encoder weights uninitialized.
Summary
Run the expensive part of the graph once and store the result, then feed it into the cheap part in a loop
In TF2: call the encoder function, save the output tensor, pass it to the decoder in a Python loop
In TF1: use session.run() to evaluate the encoder, then feed the result via feed_dict to the decoder
Wrap the loop body in @tf.function for graph-mode performance in TF2
Always set training=False on frozen layers when reusing intermediate features
