Introduction

add_loss in Keras is useful when part of your training objective comes from intermediate tensors, regularization terms, or custom constraints that are not tied directly to the main target passed to fit. The confusing part is that Keras aggregates those losses into the total training loss by default. If you want to see each component separately, you need to expose them as metrics or return them from a custom training step.

What add_loss Actually Does

When a layer or model calls self.add_loss, Keras stores that tensor as an extra contribution to the optimization objective.

1import tensorflow as tf
2
3class ActivityPenalty(tf.keras.layers.Layer):
4    def call(self, inputs):
5        penalty = 0.01 * tf.reduce_mean(tf.square(inputs))
6        self.add_loss(penalty)
7        return inputs

When this layer participates in a model, its penalty becomes part of the total loss used during training. Keras does not automatically create a named output column for each extra term.

Use add_metric to Report Each Loss Component

If you want separate visibility in training logs, add a metric with a stable name alongside the loss.

1import tensorflow as tf
2
3class ActivityPenalty(tf.keras.layers.Layer):
4    def call(self, inputs):
5        penalty = 0.01 * tf.reduce_mean(tf.square(inputs))
6        self.add_loss(penalty)
7        self.add_metric(penalty, name="activity_penalty")
8        return inputs

Now the model can log both the total loss and the named penalty metric during fit. This is usually the simplest answer when the question is really about output, not optimization.

Example with Multiple Added Losses

You can add more than one extra loss term from different layers.

1inputs = tf.keras.Input(shape=(4,))
2x = tf.keras.layers.Dense(8, activation="relu")(inputs)
3x = ActivityPenalty()(x)
4x = tf.keras.layers.Dense(8, activation="relu")(x)
5x = ActivityPenalty()(x)
6outputs = tf.keras.layers.Dense(1)(x)
7
8model = tf.keras.Model(inputs, outputs)
9model.compile(optimizer="adam", loss="mse")

During training, Keras sums the base mse loss and both added penalties into one total loss value. The individual penalty values can still appear in logs if you registered them as metrics.

Inspect Added Losses Programmatically

At call time, a model exposes the currently collected extra losses in model.losses.

1import tensorflow as tf
2
3x = tf.ones((2, 4))
4_ = model(x)
5print(model.losses)

This is useful for debugging, but it is not a full training report by itself. The entries are tensors, not automatically formatted history columns with stable names.

It is also important to remember that model.losses is populated during a forward call. If you inspect it before the model has processed input, it may look empty even though your layers do call add_loss.

Use a Custom train_step for Full Control

If you need to log, weight, or print each loss component exactly the way you want, override train_step.

1class CustomModel(tf.keras.Model):
2    def train_step(self, data):
3        x, y = data
4        with tf.GradientTape() as tape:
5            y_pred = self(x, training=True)
6            base_loss = self.compiled_loss(y, y_pred)
7            extra_loss = tf.add_n(self.losses) if self.losses else 0.0
8            total_loss = base_loss + extra_loss
9
10        grads = tape.gradient(total_loss, self.trainable_variables)
11        self.optimizer.apply_gradients(zip(grads, self.trainable_variables))
12        return {
13            "loss": total_loss,
14            "base_loss": base_loss,
15            "extra_loss": extra_loss,
16        }

This is the right approach when metrics need custom grouping or when some losses should be reported separately but combined differently for optimization.

That matters in multi-objective training, where you may want one number for optimization, one number for regularization pressure, and another number for a task-specific penalty that you are still monitoring separately.

Common Pitfalls

• Expecting every add_loss term to appear automatically as a separate named value in training history.
• Forgetting that Keras sums all extra losses into the total objective.
• Using model.losses for debugging and assuming it is a permanent log format.
• Adding custom losses without also exposing metrics when observability matters.
• Overriding train_step without keeping the total loss aligned with the optimization objective.

Summary

• 'add_loss contributes extra terms to the total training loss.'
• Keras does not automatically label each extra loss as a separate history field.
• Use add_metric when you want named logging for individual loss components.
• Inspect model.losses when debugging what is currently attached.
• Override train_step when you need full control over reporting and aggregation.