Introduction

In TensorFlow, from_logits tells the loss function whether the model output is raw unnormalized scores or already normalized probabilities. That sounds small, but it changes how the loss interprets the last layer and whether TensorFlow applies softmax internally. If you set it incorrectly, training can become unstable or simply optimize the wrong mathematical quantity.

What Logits Actually Are

Logits are the raw numbers produced by the final model layer before softmax. They can be any real values, such as:

[2.3, -1.1, 0.7]

These are not probabilities. They do not sum to 1, and they can be negative.

Probabilities are what you get after applying softmax to those logits. For a multi-class classifier, softmax converts the raw scores into values between 0 and 1 that sum to 1.

That distinction is exactly what from_logits communicates to the loss function.

from_logits=True

Use from_logits=True when the model outputs raw scores with no softmax activation in the final layer.

Example:

1import tensorflow as tf
2
3model = tf.keras.Sequential([
4    tf.keras.layers.Dense(32, activation="relu"),
5    tf.keras.layers.Dense(10)  # no softmax here
6])
7
8loss_fn = tf.keras.losses.SparseCategoricalCrossentropy(from_logits=True)

In this setup, TensorFlow knows the outputs are logits and handles the internal softmax step in a numerically stable way while computing the cross-entropy loss.

This is often the preferred configuration because the combined logits-plus-loss path is implemented carefully for numerical stability.

from_logits=False

Use from_logits=False when the model already outputs probabilities, which usually means the final layer applies softmax.

1import tensorflow as tf
2
3model = tf.keras.Sequential([
4    tf.keras.layers.Dense(32, activation="relu"),
5    tf.keras.layers.Dense(10, activation="softmax")
6])
7
8loss_fn = tf.keras.losses.SparseCategoricalCrossentropy(from_logits=False)

Now the loss expects probabilities and does not apply softmax internally again.

This configuration is also correct, but only if the model really is outputting probabilities.

Why the Setting Matters

If you mismatch the final layer and from_logits, the math becomes wrong.

Two common bad cases are:

• model outputs logits, but from_logits=False
• model outputs softmax probabilities, but from_logits=True

In the first case, the loss treats unnormalized scores as if they were probabilities. In the second case, the loss behaves as though it still needs to apply a softmax-like interpretation even though probabilities were already produced.

Either mistake can hurt convergence and make debugging harder because the code still runs.

sparse_categorical_crossentropy Also Expects Integer Labels

The word sparse means the labels are integer class IDs rather than one-hot vectors.

For example, these are valid sparse labels:

y_true = tf.constant([0, 2, 1, 3])

The predictions, meanwhile, still have one score per class. So if your model has 4 classes, the output shape is typically (batch_size, 4).

This label format is independent of from_logits, but both pieces must be correct at the same time.

Recommended Pairings

A simple rule of thumb is:

• no final softmax: from_logits=True
• final softmax present: from_logits=False

These pairings are the ones you should deliberately aim for.

In many TensorFlow examples, developers prefer the first form:

1model = tf.keras.Sequential([
2    tf.keras.layers.Dense(64, activation="relu"),
3    tf.keras.layers.Dense(5)
4])
5
6model.compile(
7    optimizer="adam",
8    loss=tf.keras.losses.SparseCategoricalCrossentropy(from_logits=True),
9    metrics=["accuracy"]
10)

That keeps the final layer simple and lets the loss function own the stable logits-to-loss calculation.

How to Check Your Model

If you are unsure what your model outputs, inspect the last layer. A final Dense(num_classes) with no activation usually means logits. A final Dense(num_classes, activation="softmax") usually means probabilities.

You can also print predictions from a small batch and inspect whether rows sum to 1.

preds = model(tf.random.normal((2, 20)))
print(preds)
print(tf.reduce_sum(preds, axis=-1))

If the outputs already look like probability distributions, from_logits=False is the likely match.

Common Pitfalls

The most common mistake is adding a softmax activation to the final layer and still setting from_logits=True. That creates a mismatch between model output and loss expectation.

Another mistake is using raw logits with from_logits=False, which makes the loss interpret arbitrary scores as if they were probabilities.

Developers also focus on from_logits while overlooking label format. SparseCategoricalCrossentropy still expects integer labels, not one-hot encoded ones.

Summary

• 'from_logits tells TensorFlow whether model outputs are raw scores or already normalized probabilities.'
• Use from_logits=True when the final layer has no softmax.
• Use from_logits=False when the final layer already applies softmax.
• A mismatch can make training behave incorrectly even if the code still runs.
• For sparse categorical loss, the labels should be integer class IDs regardless of the logits setting.