Practice Now
TensorFlow
variables
scope
machine learning
programming

Tensorflow get all variables in scope

ML System Design practice on Codemia

Design recommenders, ranking systems and training pipelines the way ML interviews actually ask for them, with worked solutions.

Practice ML system design
TensorFlow & KerasMachine Learning

Introduction

In TensorFlow, variable scopes organize variables into named groups, making it possible to share variables and retrieve them by prefix. Getting all variables within a specific scope is essential for tasks like saving/restoring model subsets, freezing layers during transfer learning, computing per-scope statistics, and debugging model architecture. TensorFlow 1.x uses tf.get_collection() and tf.trainable_variables() with scope filtering, while TensorFlow 2.x uses tf.Module and Keras layer properties.

TensorFlow 1.x: Variable Scopes

python
1import tensorflow as tf
2
3# Create variables inside named scopes
4with tf.variable_scope("encoder"):
5    w1 = tf.get_variable("weights", shape=[784, 256])
6    b1 = tf.get_variable("bias", shape=[256])
7
8with tf.variable_scope("decoder"):
9    w2 = tf.get_variable("weights", shape=[256, 784])
10    b2 = tf.get_variable("bias", shape=[784])
11
12# Get all trainable variables in the "encoder" scope
13encoder_vars = tf.get_collection(tf.GraphKeys.TRAINABLE_VARIABLES, scope="encoder")
14print([v.name for v in encoder_vars])
15# ['encoder/weights:0', 'encoder/bias:0']
16
17# Get all variables (including non-trainable) in a scope
18all_encoder = tf.get_collection(tf.GraphKeys.GLOBAL_VARIABLES, scope="encoder")

tf.get_collection() filters variables by scope name prefix. TRAINABLE_VARIABLES includes only variables that participate in gradient computation, while GLOBAL_VARIABLES includes all variables.

Using tf.trainable_variables with Scope Filter

python
1import tensorflow as tf
2
3with tf.variable_scope("model/layer1"):
4    tf.get_variable("kernel", [100, 64])
5    tf.get_variable("bias", [64])
6
7with tf.variable_scope("model/layer2"):
8    tf.get_variable("kernel", [64, 10])
9    tf.get_variable("bias", [10])
10
11# Filter by scope prefix
12layer1_vars = [v for v in tf.trainable_variables() if v.name.startswith("model/layer1")]
13print([v.name for v in layer1_vars])
14# ['model/layer1/kernel:0', 'model/layer1/bias:0']
15
16# Alternative: use tf.get_collection with a regex-like scope
17model_vars = tf.get_collection(tf.GraphKeys.TRAINABLE_VARIABLES, scope="model/")
18print(len(model_vars))  # 4 — all variables under "model/"

The scope parameter in tf.get_collection() is actually a regex pattern, so scope="model/" matches all variables whose names start with model/.

TensorFlow 2.x: Keras Layer Variables

python
1import tensorflow as tf
2
3# Build a model with named layers
4model = tf.keras.Sequential([
5    tf.keras.layers.Dense(256, activation="relu", name="encoder_dense1"),
6    tf.keras.layers.Dense(128, activation="relu", name="encoder_dense2"),
7    tf.keras.layers.Dense(10, activation="softmax", name="output"),
8])
9
10# Build the model to create variables
11model.build(input_shape=(None, 784))
12
13# Get variables from a specific layer by name
14encoder1 = model.get_layer("encoder_dense1")
15print([v.name for v in encoder1.trainable_variables])
16# ['encoder_dense1/kernel:0', 'encoder_dense1/bias:0']
17
18# Get all variables from layers matching a prefix
19encoder_vars = []
20for layer in model.layers:
21    if layer.name.startswith("encoder"):
22        encoder_vars.extend(layer.trainable_variables)
23print(len(encoder_vars))  # 4 — kernel + bias from both encoder layers

Using tf.Module in TF2

python
1import tensorflow as tf
2
3class Encoder(tf.Module):
4    def __init__(self):
5        super().__init__(name="encoder")
6        self.dense1 = tf.keras.layers.Dense(256, name="dense1")
7        self.dense2 = tf.keras.layers.Dense(128, name="dense2")
8
9    def __call__(self, x):
10        x = self.dense1(x)
11        return self.dense2(x)
12
13class AutoEncoder(tf.Module):
14    def __init__(self):
15        super().__init__(name="autoencoder")
16        self.encoder = Encoder()
17        self.decoder = tf.keras.layers.Dense(784, name="decoder")
18
19ae = AutoEncoder()
20ae(tf.zeros([1, 784]))  # build variables
21
22# Get all trainable variables in the encoder submodule
23print([v.name for v in ae.encoder.trainable_variables])
24# ['encoder/dense1/kernel:0', 'encoder/dense1/bias:0',
25#  'encoder/dense2/kernel:0', 'encoder/dense2/bias:0']
26
27# Get all variables in the entire model
28print(len(ae.trainable_variables))  # 6

tf.Module automatically tracks variables in submodules. Access .trainable_variables on any submodule to get its scoped variables.

Practical Use Cases

python
1import tensorflow as tf
2
3model = tf.keras.Sequential([
4    tf.keras.layers.Dense(128, name="feature_extractor"),
5    tf.keras.layers.Dense(10, name="classifier"),
6])
7model.build((None, 784))
8
9# 1. Freeze layers (transfer learning)
10for layer in model.layers:
11    if layer.name == "feature_extractor":
12        layer.trainable = False
13
14# 2. Apply different learning rates per scope
15feature_vars = model.get_layer("feature_extractor").trainable_variables
16classifier_vars = model.get_layer("classifier").trainable_variables
17
18optimizer1 = tf.keras.optimizers.Adam(learning_rate=1e-4)
19optimizer2 = tf.keras.optimizers.Adam(learning_rate=1e-2)
20
21# 3. Save/restore specific layers
22checkpoint = tf.train.Checkpoint(classifier=model.get_layer("classifier"))
23checkpoint.save("/tmp/classifier_ckpt")

Migrating from TF1 to TF2

python
1# TF1 style (deprecated in TF2):
2# encoder_vars = tf.get_collection(tf.GraphKeys.TRAINABLE_VARIABLES, scope="encoder")
3
4# TF2 equivalent using Keras:
5encoder_layer = model.get_layer("encoder")
6encoder_vars = encoder_layer.trainable_variables
7
8# TF2 equivalent using tf.Module:
9encoder_vars = model.encoder.trainable_variables
10
11# If using tf.compat.v1 for migration:
12import tensorflow.compat.v1 as tf1
13tf1.disable_eager_execution()
14encoder_vars = tf1.get_collection(tf1.GraphKeys.TRAINABLE_VARIABLES, scope="encoder")

Common Pitfalls

  • Using TF1 APIs in TF2 without compat mode: tf.get_collection() and tf.variable_scope() do not work in TF2 eager mode. Use tf.compat.v1 for migration or switch to tf.Module / Keras layer properties.
  • Forgetting to build the model before accessing variables: Variables are created lazily in TF2. Calling model.trainable_variables before passing data through the model returns an empty list. Call model.build(input_shape) or pass a dummy input first.
  • Scope regex matching unintended variables: The scope parameter in tf.get_collection() is a regex. scope="model" matches both model/layer1 and my_model/layer1. Use scope="model/" with a trailing slash for exact prefix matching.
  • Confusing trainable_variables with variables: trainable_variables excludes batch normalization running means/variances and other non-trainable state. Use variables (or non_trainable_variables) when you need the complete set for saving or inspection.
  • Variable name collisions across scopes: In TF1, creating variables with the same name in different scopes appends _1, _2 suffixes. In TF2 with Keras, each layer has a unique name. Always verify variable names with [v.name for v in model.trainable_variables] to confirm scope structure.

Summary

  • TF1: Use tf.get_collection(tf.GraphKeys.TRAINABLE_VARIABLES, scope="name") to get scoped variables
  • TF2 Keras: Use model.get_layer("name").trainable_variables per layer
  • TF2 Module: Use module.submodule.trainable_variables for hierarchical access
  • Build the model before accessing variables — TF2 creates them lazily
  • Use scoped variable access for transfer learning, per-layer learning rates, and selective checkpointing
  • The scope parameter is regex-based — use trailing slashes for exact prefix matching
Related reading
Free course
Beginner
7 lessons
2 hours
Tackling System Design Interview Problems

A short course that equips you with the skills to approach system design interviews methodically.

Start the free course
Track what you have practised

A free account saves your progress, solutions and study plan across every problem on Codemia.

ML System Design practice on Codemia

Design recommenders, ranking systems and training pipelines the way ML interviews actually ask for them, with worked solutions.

Practice ML system design
Codemia
© 2026 Codemia
Resources
Legal
Social
All Rights Reserved.