Introduction

Feature-wise Linear Modulation, often called FiLM, applies per-feature scaling and shifting conditioned on another input. In Keras, this is useful for multi-modal models where one branch modulates another branch. A correct implementation must align tensor shapes and keep conditioning networks stable.

Core Sections

FiLM Concept in One Equation

FiLM transforms feature tensor x using learned parameters gamma and beta.

FiLM(x) = gamma * x + beta

Gamma and beta are produced by a conditioning network, often from text, metadata, or task context.

Build a Simple FiLM Layer in Keras

Create a custom layer that receives feature tensor and conditioning vector.

1import tensorflow as tf
2
3class FiLMLayer(tf.keras.layers.Layer):
4    def __init__(self, feature_dim, **kwargs):
5        super().__init__(**kwargs)
6        self.gamma_dense = tf.keras.layers.Dense(feature_dim)
7        self.beta_dense = tf.keras.layers.Dense(feature_dim)
8
9    def call(self, inputs):
10        x, cond = inputs
11        gamma = self.gamma_dense(cond)
12        beta = self.beta_dense(cond)
13        return gamma * x + beta

This version targets two-dimensional feature tensors of shape batch by feature.

Integrate FiLM into a Multi-input Model

1feature_input = tf.keras.Input(shape=(64,), name="features")
2cond_input = tf.keras.Input(shape=(32,), name="conditioning")
3
4x = tf.keras.layers.Dense(64, activation="relu")(feature_input)
5x = FiLMLayer(64)([x, cond_input])
6x = tf.keras.layers.Dense(32, activation="relu")(x)
7out = tf.keras.layers.Dense(1, activation="sigmoid")(x)
8
9model = tf.keras.Model([feature_input, cond_input], out)
10model.compile(optimizer="adam", loss="binary_crossentropy", metrics=["accuracy"])

This pattern is a good baseline for conditional prediction tasks.

Handle Shape Alignment Carefully

If feature tensors are rank three or higher, reshape gamma and beta for broadcasting.

1# Example for sequence features: x shape batch, time, channels
2gamma = tf.expand_dims(gamma, axis=1)
3beta = tf.expand_dims(beta, axis=1)
4modulated = gamma * x + beta

Shape mismatches are the most common FiLM implementation bug.

Stabilize Training

Large unconstrained gamma values can destabilize optimization. Common tricks include initializing gamma near one, using regularization, and limiting conditioning branch depth.

You can also apply normalization before FiLM to keep modulation magnitudes predictable.

Evaluate Modulation Impact

Run ablation experiments with and without FiLM to verify benefit. If performance does not improve, conditioning signal may be weak or merged at wrong network depth.

Sequence and Vision FiLM Variants

FiLM is not limited to flat features. For sequence models, conditioning can modulate each time step channel-wise. For vision models, conditioning can modulate feature maps channel by channel.

1# x shape: batch, height, width, channels
2cond = tf.keras.layers.Dense(128, activation="relu")(cond_input)
3gamma = tf.keras.layers.Dense(channels)(cond)
4beta = tf.keras.layers.Dense(channels)(cond)
5
6gamma = tf.reshape(gamma, (-1, 1, 1, channels))
7beta = tf.reshape(beta, (-1, 1, 1, channels))
8modulated = gamma * x + beta

Shape management is critical when moving from dense to spatial features.

Regularization and Monitoring

Because FiLM introduces extra capacity, overfitting can increase on small datasets. Add dropout or weight decay in conditioning branches, and monitor validation performance closely.

Track gamma and beta statistics during training. Extremely large magnitudes often indicate unstable conditioning behavior and may require lower learning rates or normalization.

Careful experiment tracking is essential so FiLM gains are distinguished from unrelated training changes such as optimizer or data preprocessing adjustments.

When FiLM is applied in deeper layers, verify gradient flow with simple diagnostics to ensure conditioning signal remains effective throughout the network depth.

Consistent experiment logging helps teams iterate on FiLM designs with confidence.

Common Pitfalls

• Forgetting to align gamma and beta shapes with feature tensor dimensions.
• Using overly deep conditioning branches and causing unstable training.
• Applying FiLM at arbitrary layers without ablation validation.
• Ignoring initialization and letting gamma explode early.
• Passing unnormalized conditioning features and adding noisy modulation.

Summary

• FiLM applies feature-wise scaling and shifting conditioned on context.
• Keras implementation requires careful tensor shape management.
• Multi-input models are natural places to use FiLM.
• Stabilize gamma and beta learning with sensible initialization and normalization.
• Validate FiLM value through controlled ablation experiments.