Introduction
In TensorFlow, variable scopes organize variables into named groups, making it possible to share variables and retrieve them by prefix. Getting all variables within a specific scope is essential for tasks like saving/restoring model subsets, freezing layers during transfer learning, computing per-scope statistics, and debugging model architecture. TensorFlow 1.x uses tf.get_collection() and tf.trainable_variables() with scope filtering, while TensorFlow 2.x uses tf.Module and Keras layer properties.
TensorFlow 1.x: Variable Scopes
1import tensorflow as tf
2
3# Create variables inside named scopes
4with tf.variable_scope("encoder"):
5 w1 = tf.get_variable("weights", shape=[784, 256])
6 b1 = tf.get_variable("bias", shape=[256])
7
8with tf.variable_scope("decoder"):
9 w2 = tf.get_variable("weights", shape=[256, 784])
10 b2 = tf.get_variable("bias", shape=[784])
11
12# Get all trainable variables in the "encoder" scope
13encoder_vars = tf.get_collection(tf.GraphKeys.TRAINABLE_VARIABLES, scope="encoder")
14print([v.name for v in encoder_vars])
15# ['encoder/weights:0', 'encoder/bias:0']
16
17# Get all variables (including non-trainable) in a scope
18all_encoder = tf.get_collection(tf.GraphKeys.GLOBAL_VARIABLES, scope="encoder")
tf.get_collection() filters variables by scope name prefix. TRAINABLE_VARIABLES includes only variables that participate in gradient computation, while GLOBAL_VARIABLES includes all variables.
Using tf.trainable_variables with Scope Filter
1import tensorflow as tf
2
3with tf.variable_scope("model/layer1"):
4 tf.get_variable("kernel", [100, 64])
5 tf.get_variable("bias", [64])
6
7with tf.variable_scope("model/layer2"):
8 tf.get_variable("kernel", [64, 10])
9 tf.get_variable("bias", [10])
10
11# Filter by scope prefix
12layer1_vars = [v for v in tf.trainable_variables() if v.name.startswith("model/layer1")]
13print([v.name for v in layer1_vars])
14# ['model/layer1/kernel:0', 'model/layer1/bias:0']
15
16# Alternative: use tf.get_collection with a regex-like scope
17model_vars = tf.get_collection(tf.GraphKeys.TRAINABLE_VARIABLES, scope="model/")
18print(len(model_vars)) # 4 — all variables under "model/"
The scope parameter in tf.get_collection() is actually a regex pattern, so scope="model/" matches all variables whose names start with model/.
TensorFlow 2.x: Keras Layer Variables
1import tensorflow as tf
2
3# Build a model with named layers
4model = tf.keras.Sequential([
5 tf.keras.layers.Dense(256, activation="relu", name="encoder_dense1"),
6 tf.keras.layers.Dense(128, activation="relu", name="encoder_dense2"),
7 tf.keras.layers.Dense(10, activation="softmax", name="output"),
8])
9
10# Build the model to create variables
11model.build(input_shape=(None, 784))
12
13# Get variables from a specific layer by name
14encoder1 = model.get_layer("encoder_dense1")
15print([v.name for v in encoder1.trainable_variables])
16# ['encoder_dense1/kernel:0', 'encoder_dense1/bias:0']
17
18# Get all variables from layers matching a prefix
19encoder_vars = []
20for layer in model.layers:
21 if layer.name.startswith("encoder"):
22 encoder_vars.extend(layer.trainable_variables)
23print(len(encoder_vars)) # 4 — kernel + bias from both encoder layers
Using tf.Module in TF2
1import tensorflow as tf
2
3class Encoder(tf.Module):
4 def __init__(self):
5 super().__init__(name="encoder")
6 self.dense1 = tf.keras.layers.Dense(256, name="dense1")
7 self.dense2 = tf.keras.layers.Dense(128, name="dense2")
8
9 def __call__(self, x):
10 x = self.dense1(x)
11 return self.dense2(x)
12
13class AutoEncoder(tf.Module):
14 def __init__(self):
15 super().__init__(name="autoencoder")
16 self.encoder = Encoder()
17 self.decoder = tf.keras.layers.Dense(784, name="decoder")
18
19ae = AutoEncoder()
20ae(tf.zeros([1, 784])) # build variables
21
22# Get all trainable variables in the encoder submodule
23print([v.name for v in ae.encoder.trainable_variables])
24# ['encoder/dense1/kernel:0', 'encoder/dense1/bias:0',
25# 'encoder/dense2/kernel:0', 'encoder/dense2/bias:0']
26
27# Get all variables in the entire model
28print(len(ae.trainable_variables)) # 6
tf.Module automatically tracks variables in submodules. Access .trainable_variables on any submodule to get its scoped variables.
Practical Use Cases
1import tensorflow as tf
2
3model = tf.keras.Sequential([
4 tf.keras.layers.Dense(128, name="feature_extractor"),
5 tf.keras.layers.Dense(10, name="classifier"),
6])
7model.build((None, 784))
8
9# 1. Freeze layers (transfer learning)
10for layer in model.layers:
11 if layer.name == "feature_extractor":
12 layer.trainable = False
13
14# 2. Apply different learning rates per scope
15feature_vars = model.get_layer("feature_extractor").trainable_variables
16classifier_vars = model.get_layer("classifier").trainable_variables
17
18optimizer1 = tf.keras.optimizers.Adam(learning_rate=1e-4)
19optimizer2 = tf.keras.optimizers.Adam(learning_rate=1e-2)
20
21# 3. Save/restore specific layers
22checkpoint = tf.train.Checkpoint(classifier=model.get_layer("classifier"))
23checkpoint.save("/tmp/classifier_ckpt")
Migrating from TF1 to TF2
1# TF1 style (deprecated in TF2):
2# encoder_vars = tf.get_collection(tf.GraphKeys.TRAINABLE_VARIABLES, scope="encoder")
3
4# TF2 equivalent using Keras:
5encoder_layer = model.get_layer("encoder")
6encoder_vars = encoder_layer.trainable_variables
7
8# TF2 equivalent using tf.Module:
9encoder_vars = model.encoder.trainable_variables
10
11# If using tf.compat.v1 for migration:
12import tensorflow.compat.v1 as tf1
13tf1.disable_eager_execution()
14encoder_vars = tf1.get_collection(tf1.GraphKeys.TRAINABLE_VARIABLES, scope="encoder")
Common Pitfalls
Using TF1 APIs in TF2 without compat mode: tf.get_collection() and tf.variable_scope() do not work in TF2 eager mode. Use tf.compat.v1 for migration or switch to tf.Module / Keras layer properties.
Forgetting to build the model before accessing variables: Variables are created lazily in TF2. Calling model.trainable_variables before passing data through the model returns an empty list. Call model.build(input_shape) or pass a dummy input first.
Scope regex matching unintended variables: The scope parameter in tf.get_collection() is a regex. scope="model" matches both model/layer1 and my_model/layer1. Use scope="model/" with a trailing slash for exact prefix matching.
Confusing trainable_variables with variables: trainable_variables excludes batch normalization running means/variances and other non-trainable state. Use variables (or non_trainable_variables) when you need the complete set for saving or inspection.
Variable name collisions across scopes: In TF1, creating variables with the same name in different scopes appends _1, _2 suffixes. In TF2 with Keras, each layer has a unique name. Always verify variable names with [v.name for v in model.trainable_variables] to confirm scope structure.
Summary
TF1: Use tf.get_collection(tf.GraphKeys.TRAINABLE_VARIABLES, scope="name") to get scoped variables
TF2 Keras: Use model.get_layer("name").trainable_variables per layer
TF2 Module: Use module.submodule.trainable_variables for hierarchical access
Build the model before accessing variables — TF2 creates them lazily
Use scoped variable access for transfer learning, per-layer learning rates, and selective checkpointing
The scope parameter is regex-based — use trailing slashes for exact prefix matching
