Introduction

model.evaluate and model.predict are not supposed to disagree about accuracy if you run them on the same data and compute the metric correctly. When they do differ in a multi-class NLP task, the cause is usually label encoding, preprocessing mismatch, dataset ordering, or an error in how predictions are turned into class labels.

What Each Method Actually Does

model.evaluate runs the model on a dataset and computes the compiled loss and metrics. If you compiled with metrics=["accuracy"], Keras applies the metric implementation that matches your targets, such as categorical accuracy or sparse categorical accuracy.

model.predict only returns model outputs, usually class probabilities or logits. You must convert those outputs into predicted labels yourself before computing accuracy.

1import numpy as np
2import tensorflow as tf
3
4model = tf.keras.Sequential(
5    [
6        tf.keras.layers.Input(shape=(100,)),
7        tf.keras.layers.Dense(32, activation="relu"),
8        tf.keras.layers.Dense(3, activation="softmax"),
9    ]
10)
11
12model.compile(
13    optimizer="adam",
14    loss="sparse_categorical_crossentropy",
15    metrics=["accuracy"],
16)
17
18x_test = np.random.rand(8, 100).astype("float32")
19y_test = np.array([0, 1, 2, 1, 0, 2, 1, 0])
20
21loss, eval_accuracy = model.evaluate(x_test, y_test, verbose=0)
22pred_probs = model.predict(x_test, verbose=0)
23pred_labels = np.argmax(pred_probs, axis=1)
24manual_accuracy = np.mean(pred_labels == y_test)
25
26print(eval_accuracy, manual_accuracy)

If the model output and labels are interpreted correctly, those two accuracy values should match or be extremely close.

Common Reasons They Differ

The biggest source of confusion is target format. If your labels are one-hot encoded, you should compare argmax of the predictions against argmax of the true labels. If your labels are integer class ids, compare against the raw integer targets instead.

Another common mistake is using a different dataset path for each method. For example, evaluate may run on a batched dataset with one preprocessing pipeline, while predict runs on raw text or on a generator that has already been partially consumed.

Here is the correct pattern for one-hot targets:

1import numpy as np
2
3pred_probs = model.predict(x_test, verbose=0)
4pred_labels = np.argmax(pred_probs, axis=1)
5true_labels = np.argmax(y_test_one_hot, axis=1)
6
7accuracy = np.mean(pred_labels == true_labels)
8print(accuracy)

NLP Pipelines Make Mismatches Easier

In text classification, the preprocessing stack often includes tokenization, padding, vocabulary lookup, and possibly label encoding. If evaluate sees tokenized and padded arrays but predict is run on differently prepared inputs, you are not comparing the same experiment.

This is especially easy to miss when using TextVectorization, custom generators, or label encoders outside the model graph. A reliable debugging step is to save the exact tensor batch sent to evaluate and then run predict on that same tensor batch.

Batching and ordering can also matter. If you shuffle the evaluation dataset before calling predict, but compare the results to labels in the original order, your manual accuracy will be wrong even though the model outputs are fine.

A Good Debugging Checklist

Use this sequence when the numbers disagree:

1. Confirm whether labels are sparse integers or one-hot vectors.
2. Confirm whether model outputs are probabilities or logits.
3. Make sure predict and evaluate use the exact same input tensors.
4. Check that labels are aligned with predictions after batching and shuffling.
5. Recompute manual accuracy on a tiny batch you can inspect by hand.

That process usually exposes the mismatch quickly.

Common Pitfalls

• Comparing argmax(predictions) to one-hot labels directly instead of to argmax of the labels.
• Forgetting that accuracy in Keras depends on the target format chosen at compile time.
• Running predict on differently preprocessed text than the data used by evaluate.
• Misaligning labels and predictions after shuffling, batching, or partial generator consumption.
• Interpreting logits as probabilities without checking the model output layer and loss configuration.

Summary

• 'model.evaluate computes metrics internally, while model.predict only returns outputs.'
• Accuracy should match when both use the same data and the same label interpretation.
• Multi-class NLP pipelines often fail because labels or preprocessing differ between the two paths.
• Use argmax correctly based on whether targets are sparse or one-hot encoded.
• Debug with a tiny fixed batch when the numbers do not line up.