Introduction

In Keras, loss and accuracy are related but they do not mean the same thing and they are not used the same way during training. Loss is the quantity the optimizer tries to minimize, while accuracy is usually a reporting metric that tells you how often predictions match labels according to a task-specific rule.

What Loss Means in Keras

The loss function converts model predictions and true targets into a number that represents how wrong the model is. During training, Keras computes gradients from the loss and updates weights to reduce it.

For example, a binary classifier often uses binary_crossentropy:

1import tensorflow as tf
2
3model = tf.keras.Sequential([
4    tf.keras.layers.Dense(8, activation="relu"),
5    tf.keras.layers.Dense(1, activation="sigmoid"),
6])
7
8model.compile(
9    optimizer="adam",
10    loss="binary_crossentropy",
11    metrics=["accuracy"],
12)

Here, the optimizer works on binary_crossentropy, not directly on accuracy.

That matters because loss contains richer information than a simple correct or incorrect label. A prediction of 0.51 for the correct class and a prediction of 0.99 for the correct class may both count as "accurate," but the loss will usually reward the more confident correct prediction more strongly.

What Accuracy Means in Keras

Accuracy is a metric that checks how often predictions match the target under a defined rule. The specific rule depends on the metric:

• 'BinaryAccuracy compares binary predictions against a threshold'
• 'CategoricalAccuracy compares the index of the highest predicted class with one-hot labels'
• 'SparseCategoricalAccuracy does the same for integer class labels'

Example:

1import tensorflow as tf
2
3y_true = tf.constant([1, 0, 1, 1], dtype=tf.float32)
4y_pred = tf.constant([0.9, 0.4, 0.2, 0.8], dtype=tf.float32)
5
6metric = tf.keras.metrics.BinaryAccuracy(threshold=0.5)
7metric.update_state(y_true, y_pred)
8
9print(metric.result().numpy())

With a threshold of 0.5, the predictions become [1, 0, 0, 1], so the accuracy is 0.75.

Why Loss and Accuracy Can Move Differently

A common surprise is seeing loss decrease while accuracy stays flat, or accuracy improve while loss gets worse. This is not a Keras bug.

It happens because:

• accuracy is coarse and thresholded
• loss is continuous and sensitive to confidence

Suppose a binary model changes from predicting 0.51 to 0.90 for correct positive examples. Accuracy may stay the same because both are above the threshold, but loss improves because the model is more confidently correct.

The reverse can also happen. A few predictions may cross the accuracy threshold and improve accuracy, while other predictions become much worse and increase average loss.

What Appears in history

When you call model.fit(), Keras reports epoch-level aggregates for the configured loss and metrics:

history = model.fit(x_train, y_train, epochs=5, validation_data=(x_val, y_val))

print(history.history.keys())

Typical keys include:

• 'loss'
• 'accuracy'
• 'val_loss'
• 'val_accuracy'

These values are aggregated over batches for each epoch. They are useful diagnostics, but only the loss participates directly in gradient-based optimization unless you build a custom training loop.

Picking the Right Pair

Loss and metric should match the task and label format.

Examples:

• binary classification: binary_crossentropy with BinaryAccuracy
• multi-class one-hot labels: categorical_crossentropy with CategoricalAccuracy
• multi-class integer labels: sparse_categorical_crossentropy with SparseCategoricalAccuracy
• regression: mean_squared_error or mean_absolute_error, where plain accuracy is usually not meaningful

Choosing the wrong metric can produce numbers that look valid but mean very little.

Common Pitfalls

The most common mistake is assuming Keras "optimizes accuracy." In standard training, it optimizes the loss. Accuracy is reported for you, but it is not normally the differentiable objective driving the weight update.

Another mistake is using generic "accuracy" without understanding how Keras maps that string to a concrete metric based on output shape and targets. This convenience is helpful, but explicit metrics are easier to reason about.

People also compare loss values across different loss functions as though they were directly comparable. A cross-entropy value and an MSE value live on different scales.

Finally, high accuracy can hide serious problems in imbalanced datasets. In those cases, precision, recall, AUC, or F1-style metrics may be more informative than raw accuracy.

Summary

• In Keras, loss is the optimization target and accuracy is usually a reporting metric.
• Loss measures how wrong predictions are, including confidence, not just correctness.
• Accuracy depends on task-specific matching rules such as thresholds or argmax.
• Loss and accuracy can move in different directions without contradiction.
• Use loss and metrics that match the model output format and the business goal.