Introduction

Keras and TensorFlow expose several cross-entropy losses because classification problems come in different shapes. The names look similar, but each loss assumes something specific about the labels, the model outputs, and whether classes are mutually exclusive or independent.

Start with the Problem Shape

The first distinction is not between APIs. It is between problem types.

Use these mental buckets:

• binary classification: one yes-or-no label
• multi-class classification: exactly one class out of many
• multi-label classification: several classes can be true at once

Once that is clear, the loss choice usually becomes straightforward.

binary_crossentropy

binary_crossentropy is for binary classification and also for multi-label classification where each output unit is an independent yes-or-no decision.

Typical output layer:

1import tensorflow as tf
2
3model = tf.keras.Sequential([
4    tf.keras.layers.Dense(16, activation="relu"),
5    tf.keras.layers.Dense(1, activation="sigmoid"),
6])
7
8model.compile(
9    optimizer="adam",
10    loss="binary_crossentropy",
11    metrics=["accuracy"],
12)

Here the model predicts a probability for a single binary label.

For multi-label classification, the final layer may have multiple sigmoid outputs:

1model = tf.keras.Sequential([
2    tf.keras.layers.Dense(16, activation="relu"),
3    tf.keras.layers.Dense(4, activation="sigmoid"),
4])

That still uses binary cross-entropy because each output dimension is treated independently.

categorical_crossentropy

Use categorical_crossentropy when each example belongs to exactly one class and the true labels are one-hot encoded.

Example label format:

1y_true = [
2    [1, 0, 0],
3    [0, 1, 0],
4    [0, 0, 1],
5]

Typical model:

1model = tf.keras.Sequential([
2    tf.keras.layers.Dense(32, activation="relu"),
3    tf.keras.layers.Dense(3, activation="softmax"),
4])
5
6model.compile(
7    optimizer="adam",
8    loss="categorical_crossentropy",
9    metrics=["accuracy"],
10)

The softmax output means all class probabilities compete with each other and sum to 1. That matches the assumption that exactly one class is correct.

sparse_categorical_crossentropy

This loss is for the same multi-class situation as categorical_crossentropy, but the labels are integer class ids instead of one-hot vectors.

Example labels:

y_true = [0, 1, 2, 1]

Typical compile step:

1model.compile(
2    optimizer="adam",
3    loss="sparse_categorical_crossentropy",
4    metrics=["accuracy"],
5)

The model output is still usually a softmax over classes. The difference is only how the true labels are encoded.

This is often the most convenient choice when your dataset already stores labels as integers.

Logits vs Probabilities

Another axis of confusion is whether the model output is already passed through sigmoid or softmax, or whether it is still raw logits. Keras loss objects let you say this explicitly with from_logits=True.

Binary example:

1loss = tf.keras.losses.BinaryCrossentropy(from_logits=True)
2
3model = tf.keras.Sequential([
4    tf.keras.layers.Dense(16, activation="relu"),
5    tf.keras.layers.Dense(1),
6])

Multi-class example:

1loss = tf.keras.losses.SparseCategoricalCrossentropy(from_logits=True)
2
3model = tf.keras.Sequential([
4    tf.keras.layers.Dense(32, activation="relu"),
5    tf.keras.layers.Dense(3),
6])

This matters because applying softmax in the model and also telling the loss that the outputs are logits is wrong. The loss needs to know which representation it is receiving.

TensorFlow tf.nn Versions

TensorFlow also exposes lower-level functions such as:

• 'tf.nn.softmax_cross_entropy_with_logits'
• 'tf.nn.sigmoid_cross_entropy_with_logits'

These operate directly on tensors and are often used in custom training loops or lower-level model code.

Example:

1import tensorflow as tf
2
3logits = tf.constant([[2.0, 0.5, -1.0]])
4labels = tf.constant([[1.0, 0.0, 0.0]])
5
6loss = tf.nn.softmax_cross_entropy_with_logits(labels=labels, logits=logits)
7print(loss.numpy())

These are not fundamentally different learning objectives. They are lower-level interfaces to the same core ideas.

A Practical Decision Table

Use this rule set:

• one binary label: binary_crossentropy
• several independent binary labels: binary_crossentropy
• one class out of many with one-hot labels: categorical_crossentropy
• one class out of many with integer labels: sparse_categorical_crossentropy
• raw model outputs instead of probabilities: use the matching loss with from_logits=True

Most confusion comes from mixing up label encoding with task type.

Common Pitfalls

The most common mistake is using categorical_crossentropy with integer labels instead of one-hot labels, when sparse_categorical_crossentropy is the correct match. Another is using binary_crossentropy for a mutually exclusive multi-class problem that should use softmax-based categorical loss. Developers also often confuse logits and probabilities and end up applying softmax or sigmoid twice. A final issue is assuming the low-level tf.nn cross-entropy functions represent different learning objectives when they are often just lower-level forms of the same core losses.

Summary

• Choose the cross-entropy loss based on task shape and label encoding.
• 'binary_crossentropy is for binary and independent multi-label outputs.'
• 'categorical_crossentropy is for one-hot multi-class labels.'
• 'sparse_categorical_crossentropy is for integer-coded multi-class labels.'
• Pay close attention to whether your model outputs logits or probabilities.