Introduction

Freezing a layer means keeping its weights fixed during training. Unfreezing means allowing those weights to be updated again. These steps are especially common in transfer learning, where you start from a pretrained model and only adapt part of it to a new task.

What Freezing Actually Does

A neural network layer has parameters such as weights and biases. During backpropagation, the optimizer updates the trainable parameters to reduce the loss. When you freeze a layer, those parameters are excluded from training updates.

In Keras, the usual mechanism is the trainable flag:

1import tensorflow as tf
2
3layer = tf.keras.layers.Dense(8)
4layer.build((None, 4))
5print(len(layer.trainable_weights))
6
7layer.trainable = False
8print(len(layer.trainable_weights))
9print(len(layer.non_trainable_weights))

After freezing, the layer still participates in the forward pass. It just stops changing.

Why People Freeze Layers

The classic use case is transfer learning. Early layers in a pretrained vision or language model often learn general patterns that are useful across tasks. For example, an image model may learn edges, textures, and shapes in early convolution blocks. If your new dataset is small, retraining all of those layers from scratch can waste compute and overfit quickly.

A typical workflow looks like this:

1. Load a pretrained base model
2. Freeze the base model
3. Add a small task-specific head
4. Train only the new head first
5. Optionally unfreeze part of the base model and fine-tune with a small learning rate

A Practical Keras Example

1import tensorflow as tf
2
3base_model = tf.keras.applications.MobileNetV2(
4    input_shape=(160, 160, 3),
5    include_top=False,
6    weights="imagenet"
7)
8base_model.trainable = False
9
10inputs = tf.keras.Input(shape=(160, 160, 3))
11x = base_model(inputs, training=False)
12x = tf.keras.layers.GlobalAveragePooling2D()(x)
13outputs = tf.keras.layers.Dense(3)(x)
14model = tf.keras.Model(inputs, outputs)
15
16model.compile(
17    optimizer=tf.keras.optimizers.Adam(),
18    loss=tf.keras.losses.SparseCategoricalCrossentropy(from_logits=True),
19    metrics=["accuracy"]
20)

Notice the training=False call when using the frozen base model. That is particularly important for layers such as batch normalization, whose training-time behavior differs from inference behavior.

What Unfreezing Means

Unfreezing is simply turning some frozen layers back into trainable ones so the pretrained representation can adapt more precisely to the new data.

1base_model.trainable = True
2
3for layer in base_model.layers[:-20]:
4    layer.trainable = False
5
6model.compile(
7    optimizer=tf.keras.optimizers.Adam(1e-5),
8    loss=tf.keras.losses.SparseCategoricalCrossentropy(from_logits=True),
9    metrics=["accuracy"]
10)

This example keeps most of the base model frozen and only fine-tunes the top 20 layers. Recompiling is important after changing trainability so the optimizer sees the correct set of trainable weights.

Why Not Unfreeze Everything Immediately

If you start by training the whole pretrained model on a small dataset, the random head layers can send large gradients backward and damage useful pretrained features. Freezing first gives the new output layers time to settle.

Then, if validation results plateau, selective unfreezing can improve accuracy by adapting higher-level features to the new task. This second stage is usually done with a lower learning rate because the goal is refinement, not wholesale relearning.

The Batch Normalization Detail

Batch normalization deserves special care. In Keras, freezing a BatchNormalization layer is not just about gradients. It also changes how that layer behaves during the forward pass, because it uses stored moving statistics rather than batch statistics.

That is why transfer-learning examples often call the base model with training=False even when the outer model is being trained. Without that, you can get unstable results and silently shift the statistics inside the frozen base.

Common Pitfalls

The biggest mistake is freezing or unfreezing layers and forgetting to recompile the model. The optimizer state and trainable-weight list need to be rebuilt.

Another mistake is unfreezing too much too soon on a small dataset. That often leads to overfitting or catastrophic forgetting of useful pretrained features.

A third problem is confusing trainable=False with inference-only use of the entire model. Frozen layers still run during prediction and training; they simply stop updating their parameters.

Summary

• Freezing a layer means its weights are not updated during training
• Unfreezing a layer makes those weights trainable again
• The common workflow is freeze first, train a new head, then fine-tune part of the base model
• Recompile after changing trainability so the optimizer sees the correct parameters
• Batch normalization layers need extra care during transfer learning because their runtime behavior is special