Introduction

Keras lets you mark whole variables or whole layers as trainable or non-trainable, but it does not natively support "half of this kernel tensor is frozen and half is trainable" as a built-in setting. If you want that behavior, the clean solutions are either to split the convolution into separate frozen and trainable branches or to keep one layer and mask the gradients manually during training.

Why layer.trainable = False Is Not Enough

For a convolution layer, the kernel weights usually live in one variable tensor. Keras tracks trainability at the variable level, not at the slice level.

So this works:

layer.trainable = False

but this kind of idea is not a supported Keras API:

# This is conceptually what people want, but Keras does not expose it directly.
# layer.kernel[:, :, :, :half].trainable = False

The kernel is one trainable variable. You cannot mark only half of that variable as non-trainable through standard layer configuration.

The Practical Solution: Split the Filters Into Two Layers

The simplest design is to replace one Conv2D with two parallel Conv2D layers:

• one frozen branch with half the filters,
• one trainable branch with the other half,
• then concatenate their outputs along the channel axis.

1import tensorflow as tf
2
3inputs = tf.keras.Input(shape=(64, 64, 3))
4
5frozen_conv = tf.keras.layers.Conv2D(
6    filters=16,
7    kernel_size=3,
8    padding="same",
9    use_bias=False,
10    name="frozen_conv",
11)
12trainable_conv = tf.keras.layers.Conv2D(
13    filters=16,
14    kernel_size=3,
15    padding="same",
16    use_bias=False,
17    name="trainable_conv",
18)
19
20frozen_output = frozen_conv(inputs)
21trainable_output = trainable_conv(inputs)
22outputs = tf.keras.layers.Concatenate(axis=-1)([frozen_output, trainable_output])
23
24model = tf.keras.Model(inputs, outputs)
25frozen_conv.trainable = False
26
27model.summary()

Functionally, this gives you a 32-filter result while freezing half the filters cleanly.

Initialize the Frozen Half

If the frozen filters should come from a pretrained layer, copy those weights into the frozen branch before training:

1import numpy as np
2
3dummy_kernel = np.random.randn(3, 3, 3, 16).astype("float32")
4frozen_conv.build((None, 64, 64, 3))
5frozen_conv.set_weights([dummy_kernel])
6frozen_conv.trainable = False

The trainable branch can then learn the remaining filters independently.

Alternative: Mask Gradients in a Custom Training Step

If you truly must keep one physical convolution kernel, the advanced approach is gradient masking. You compute gradients normally, then zero out the gradient slice corresponding to the filters you want frozen.

1import tensorflow as tf
2
3layer = tf.keras.layers.Conv2D(32, 3, padding="same", use_bias=False)
4x = tf.random.normal((2, 32, 32, 3))
5
6with tf.GradientTape() as tape:
7    y = layer(x)
8    loss = tf.reduce_mean(y)
9
10grads = tape.gradient(loss, layer.trainable_variables)
11kernel_grad = grads[0]
12
13mask = tf.concat(
14    [
15        tf.zeros_like(kernel_grad[:, :, :, :16]),
16        tf.ones_like(kernel_grad[:, :, :, 16:]),
17    ],
18    axis=-1,
19)
20
21masked_grad = kernel_grad * mask
22
23optimizer = tf.keras.optimizers.Adam()
24optimizer.apply_gradients([(masked_grad, layer.trainable_variables[0])])

This works, but it pushes you into a custom training loop. That is why the split-layer design is usually easier to maintain.

Which Approach Should You Prefer

Prefer split layers when:

• architecture clarity matters,
• you want ordinary model.fit,
• or the frozen and trainable channels can reasonably be modeled as separate branches.

Prefer gradient masking when:

• you must keep one combined kernel variable,
• or the model architecture is tightly tied to a single layer interface.

The second approach is more flexible but also more fragile.

Watch Out for BatchNorm and Downstream Layers

Even if half the filters are frozen, downstream layers may still adapt to the combined output. That is normal, but it means "half the filters are frozen" does not imply "half the representation stays unchanged end to end."

If the frozen filters are followed by batch normalization or other trainable transformations, those later transformations can still alter the effective behavior of the frozen branch.

Common Pitfalls

The biggest pitfall is assuming Keras supports slice-level trainability flags on a kernel tensor. It does not in the standard layer API.

Another mistake is freezing half the convolution filters conceptually while forgetting that associated bias terms, normalization layers, or downstream weights may still change the overall behavior.

Developers also often choose gradient masking immediately when a split-layer design would be simpler, easier to debug, and compatible with normal Keras workflows.

Finally, if you copy pretrained filters into the frozen branch, verify the channel order and weight shapes carefully before training.

Summary

• Keras trainability is defined per variable, not per slice of a variable tensor.
• To freeze half the filters cleanly, split the convolution into frozen and trainable branches and concatenate the outputs.
• Gradient masking is the advanced alternative when you must keep one kernel variable.
• Split layers usually work better with standard model.fit workflows.
• Remember that downstream trainable layers can still change the overall effect of a partially frozen representation.