Introduction

A confusion matrix shows how often a classifier predicts each class versus the true label. In TensorFlow, the standard tool is tf.math.confusion_matrix, and the main job is converting your model outputs into class IDs before you build the matrix.

Create class IDs from labels and predictions

tf.math.confusion_matrix expects integer class labels. If your model outputs probabilities or logits, convert them with tf.argmax.

1import tensorflow as tf
2
3y_true = tf.constant([0, 1, 2, 1, 0])
4y_pred_probs = tf.constant([
5    [0.90, 0.05, 0.05],
6    [0.10, 0.80, 0.10],
7    [0.20, 0.10, 0.70],
8    [0.15, 0.55, 0.30],
9    [0.40, 0.30, 0.30],
10])
11
12y_pred = tf.argmax(y_pred_probs, axis=1, output_type=tf.int32)
13cm = tf.math.confusion_matrix(y_true, y_pred, num_classes=3)
14
15print(cm.numpy())

For this example, the result is a 3 x 3 matrix where rows represent true classes and columns represent predicted classes.

Building the matrix from a trained model

A typical workflow with a Keras model looks like this:

1import tensorflow as tf
2
3logits = model.predict(x_test, verbose=0)
4y_pred = tf.argmax(logits, axis=1, output_type=tf.int32)
5
6if len(y_test.shape) > 1:
7    y_true = tf.argmax(y_test, axis=1, output_type=tf.int32)
8else:
9    y_true = tf.cast(y_test, tf.int32)
10
11cm = tf.math.confusion_matrix(y_true, y_pred, num_classes=num_classes)
12print(cm.numpy())

The if block handles both sparse labels such as [0, 2, 1] and one-hot labels such as [0, 1, 0].

Visualizing the confusion matrix

The raw matrix is useful, but plotting it makes patterns easier to spot:

1import matplotlib.pyplot as plt
2
3cm_np = cm.numpy()
4
5fig, ax = plt.subplots(figsize=(5, 5))
6image = ax.imshow(cm_np, cmap="Blues")
7ax.set_xlabel("Predicted label")
8ax.set_ylabel("True label")
9ax.set_title("Confusion Matrix")
10
11for row in range(cm_np.shape[0]):
12    for col in range(cm_np.shape[1]):
13        ax.text(col, row, cm_np[row, col], ha="center", va="center")
14
15fig.colorbar(image)
16plt.show()

This quickly highlights which classes the model confuses most often.

If you have human-readable class names, label the axes so the plot is easier to interpret during model reviews:

class_names = ["cat", "dog", "horse"]
ax.set_xticks(range(len(class_names)), class_names, rotation=45, ha="right")
ax.set_yticks(range(len(class_names)), class_names)

That small addition turns the chart from a debugging artifact into something you can actually show in an evaluation report.

Normalize when raw counts hide the real pattern

If some classes are much more common than others, raw counts can be misleading. A normalized matrix shows the error rate within each true class:

1cm_float = tf.cast(cm, tf.float32)
2row_sums = tf.reduce_sum(cm_float, axis=1, keepdims=True)
3normalized_cm = tf.math.divide_no_nan(cm_float, row_sums)
4
5print(normalized_cm.numpy())

That makes it easier to compare minority classes against majority classes.

Common Pitfalls

The first pitfall is feeding probabilities directly into tf.math.confusion_matrix. The function wants class IDs, not vectors of scores, so multiclass predictions must be reduced with argmax.

Another frequent mistake is mixing one-hot labels and sparse labels without converting them into the same representation. If y_true and y_pred do not use the same class encoding, the matrix is meaningless even if TensorFlow builds it successfully.

Be careful with axis interpretation as well. In TensorFlow's confusion matrix, rows are true labels and columns are predicted labels. People often reverse those mentally and misread the result.

Finally, set num_classes explicitly when possible. It guarantees a consistent matrix shape even if one class is missing from the current evaluation batch.

It is also worth separating validation and test confusion matrices. A strong-looking matrix on the validation set does not prove final model quality if you tuned hyperparameters against that same split.

Keeping those evaluations separate gives you a more trustworthy picture of model generalization.

Summary

• Use tf.math.confusion_matrix to build a confusion matrix from true and predicted class IDs.
• Convert logits or probabilities to class IDs with tf.argmax.
• Convert one-hot encoded labels to sparse IDs before comparing them to predictions.
• Plot the matrix when you want to inspect class-specific errors visually.
• Normalize the matrix when class imbalance makes raw counts hard to interpret.