Why keras use call instead of __call__?

Keras

call method

__call__ method

Python

deep learning

Why keras use call instead of call?

Master System Design with Codemia

Enhance your system design skills with over 120 practice problems, detailed solutions, and hands-on exercises.

Start Practicing Learn More

Introduction

Keras layers use a call() method instead of __call__() because the base Layer class defines __call__() to handle essential bookkeeping — building the layer on first use, tracking input shapes, managing training vs inference mode, applying regularization, and recording the computation graph. Your custom call() method contains only the forward-pass logic, while __call__() wraps it with all the framework plumbing. This separation keeps custom layer code clean and ensures consistent behavior across all layers.

How It Works Internally

When you write output = my_layer(input), Python calls my_layer.__call__(input). Inside Layer.__call__():

python

1# Simplified version of what Keras does in Layer.__call__
2class Layer:
3    def __call__(self, inputs, *args, **kwargs):
4        # 1. Build the layer (create weights) on first call
5        if not self.built:
6            self._maybe_build(inputs)
7
8        # 2. Handle training/inference mode
9        training = kwargs.get("training", None)
10
11        # 3. Cast inputs to the layer's dtype
12        inputs = self._maybe_cast_inputs(inputs)
13
14        # 4. Call the user's forward pass logic
15        outputs = self.call(inputs, *args, **kwargs)
16
17        # 5. Track the computation for the graph
18        self._set_connectivity_metadata(inputs, outputs)
19
20        # 6. Apply activity regularization
21        self._handle_activity_regularization(inputs, outputs)
22
23        return outputs

You only implement step 4. Steps 1-3 and 5-6 are handled automatically.

Writing a Custom Layer

python

1import tensorflow as tf
2
3class LinearLayer(tf.keras.layers.Layer):
4    def __init__(self, units, **kwargs):
5        super().__init__(**kwargs)
6        self.units = units
7
8    def build(self, input_shape):
9        # Called once on first __call__ — create weights here
10        self.w = self.add_weight(
11            shape=(input_shape[-1], self.units),
12            initializer="random_normal",
13            trainable=True,
14        )
15        self.b = self.add_weight(
16            shape=(self.units,),
17            initializer="zeros",
18            trainable=True,
19        )
20
21    def call(self, inputs):
22        # Forward pass only — no weight creation, no bookkeeping
23        return tf.matmul(inputs, self.w) + self.b
24
25# Usage — Python calls __call__, which calls build() then call()
26layer = LinearLayer(64)
27output = layer(tf.random.normal([32, 128]))  # __call__ -> build -> call
28print(output.shape)  # (32, 64)

What call Does That call Does Not

Responsibility	Handled by `__call__`	Your `call`
Build weights on first use	Yes	No
Input shape validation	Yes	No
Dtype casting	Yes	No
Training/inference flag	Yes (passed through)	Receives it
Activity regularization	Yes	No
Graph connectivity tracking	Yes	No
Eager vs graph execution	Yes	No
Masking support	Yes	No

The training Argument

__call__ passes the training flag to your call method:

python

1class DropoutLayer(tf.keras.layers.Layer):
2    def __init__(self, rate, **kwargs):
3        super().__init__(**kwargs)
4        self.rate = rate
5
6    def call(self, inputs, training=None):
7        if training:
8            return tf.nn.dropout(inputs, rate=self.rate)
9        return inputs
10
11layer = DropoutLayer(0.5)
12
13# During training — dropout is applied
14train_output = layer(x, training=True)
15
16# During inference — dropout is skipped
17infer_output = layer(x, training=False)

You do not need to handle the training flag plumbing — __call__ manages it.

Comparison with PyTorch

PyTorch uses the same pattern but with forward() instead of call():

python

1import torch
2import torch.nn as nn
3
4class MyModule(nn.Module):
5    def __init__(self, in_features, out_features):
6        super().__init__()
7        self.linear = nn.Linear(in_features, out_features)
8
9    def forward(self, x):  # Equivalent to Keras call()
10        return self.linear(x)
11
12# __call__ invokes hooks, then forward()
13module = MyModule(128, 64)
14output = module(torch.randn(32, 128))  # __call__ -> forward

Both frameworks use the same design: __call__ handles framework logic, and users override only the forward pass method.

Why Not Override call Directly

python

1# BAD: Overriding __call__ breaks Keras internals
2class BrokenLayer(tf.keras.layers.Layer):
3    def __call__(self, inputs):
4        # Skips weight building, dtype casting, graph tracking, etc.
5        return inputs * 2
6
7# This layer:
8# - Never calls build()
9# - Is not tracked in model.layers properly
10# - Does not support training/inference mode
11# - Breaks model.save() and model serialization

Custom Layer with Multiple Inputs

python

1class AttentionLayer(tf.keras.layers.Layer):
2    def __init__(self, units, **kwargs):
3        super().__init__(**kwargs)
4        self.units = units
5
6    def build(self, input_shape):
7        # input_shape is a list when call receives multiple inputs
8        self.W = self.add_weight(shape=(input_shape[0][-1], self.units))
9
10    def call(self, inputs):
11        query, key = inputs
12        scores = tf.matmul(query, key, transpose_b=True)
13        weights = tf.nn.softmax(scores)
14        return tf.matmul(weights, key)
15
16layer = AttentionLayer(64)
17output = layer([query_tensor, key_tensor])

Common Pitfalls

Overriding __call__ instead of call: Overriding __call__ bypasses weight building, input validation, regularization, and graph tracking. The layer may appear to work in simple tests but fails during model saving, distributed training, or when used with model.fit().
Creating weights inside call instead of build: Weights created in call are recreated on every forward pass. Create them in build() (called once) and use them in call(). The add_weight() method registers them for training and serialization.
Calling self.call() directly instead of self(): Calling self.call(inputs) skips the __call__ bookkeeping. If a layer internally uses another layer, call it as self.sub_layer(inputs), not self.sub_layer.call(inputs).
Forgetting to call super().__init__(**kwargs) in __init__: Without calling the parent constructor, the layer is not properly registered. Features like layer.name, layer.dtype, and layer.trainable will not work.
Not accepting **kwargs in __init__: Keras passes configuration arguments (like name and dtype) through **kwargs. Without accepting them, MyLayer(name="my_layer") raises a TypeError.

Summary

Keras uses call() so that __call__() can handle weight building, dtype casting, graph tracking, and regularization
Override call() for forward pass logic, build() for weight creation
Never override __call__() — it breaks the framework's internals
PyTorch uses the same pattern with forward() instead of call()
Always call layers as layer(inputs), not layer.call(inputs), to trigger the full lifecycle