tensorflow
model saving
validation accuracy
machine learning
neural networks

tensorflowCan save best model only with val_acc available, skipping

ML System Design practice on Codemia

Design recommenders, ranking systems and training pipelines the way ML interviews actually ask for them, with worked solutions.

Practice ML system design

Introduction

The message Can save best model only with val_acc available, skipping comes from Keras ModelCheckpoint when the callback is told to watch a metric that never appears in the training logs. In practice, that usually means the monitor name is outdated, validation data was not provided, or the metric was never compiled into the model in the first place.

Why the Callback Skips Saving

ModelCheckpoint(save_best_only=True) works by comparing the current epoch value of one metric against the best value seen so far. If the callback looks for val_acc and the logs only contain val_accuracy, it has nothing to compare and prints the skipping message instead of saving.

That behavior is important because it means the callback is not broken. It is protecting you from saving based on a metric that does not exist.

A minimal example of the correct modern pattern looks like this:

python
1import numpy as np
2import tensorflow as tf
3
4x = np.random.rand(200, 4).astype("float32")
5y = (x.sum(axis=1) > 2.0).astype("float32")
6
7model = tf.keras.Sequential([
8    tf.keras.layers.Input(shape=(4,)),
9    tf.keras.layers.Dense(8, activation="relu"),
10    tf.keras.layers.Dense(1, activation="sigmoid"),
11])
12
13model.compile(
14    optimizer="adam",
15    loss="binary_crossentropy",
16    metrics=["accuracy"],
17)
18
19checkpoint = tf.keras.callbacks.ModelCheckpoint(
20    filepath="best.keras",
21    monitor="val_accuracy",
22    mode="max",
23    save_best_only=True,
24)
25
26history = model.fit(
27    x,
28    y,
29    validation_split=0.2,
30    epochs=3,
31    callbacks=[checkpoint],
32    verbose=0,
33)

With this configuration, val_accuracy exists, so the checkpoint callback can compare epochs and save the best model.

Old Metric Names Versus New Metric Names

A large share of these errors comes from copying code written for older Keras versions. Older tutorials often used:

  • 'acc'
  • 'val_acc'

Modern tf.keras usually logs:

  • 'accuracy'
  • 'val_accuracy'

That small naming change is enough to trigger the warning. If the code says monitor="val_acc" but your history contains val_accuracy, the callback will skip every epoch.

The fastest way to stop guessing is to print the actual history keys:

python
print(history.history.keys())

Typical output looks like this:

python
dict_keys(['loss', 'accuracy', 'val_loss', 'val_accuracy'])

Use one of those exact strings for monitor.

Validation Data Must Exist

The prefix val_ means Keras computed the metric on validation data. If you do not pass validation_data or validation_split, there is no validation metric at all.

python
history = model.fit(x, y, epochs=3, verbose=0)
print(history.history.keys())

In that case, the logs will usually contain only training metrics such as loss and accuracy. Monitoring val_accuracy is impossible because it was never produced.

If you intentionally do not have validation data, monitor a training metric instead:

python
1checkpoint = tf.keras.callbacks.ModelCheckpoint(
2    filepath="best.keras",
3    monitor="accuracy",
4    mode="max",
5    save_best_only=True,
6)

That is valid, but it changes the meaning of “best” from best validation result to best training result.

The Metric Must Be Compiled

The metric also needs to be part of compile. If you monitor val_accuracy but compile only a loss, Keras will not log accuracy at all.

python
1model.compile(
2    optimizer="adam",
3    loss="binary_crossentropy",
4    metrics=["accuracy"],
5)

The same rule applies to custom metrics. If you define tf.keras.metrics.AUC(name="auc"), then the validation metric will typically be named val_auc, not val_accuracy or val_acc.

python
1auc = tf.keras.metrics.AUC(name="auc")
2
3model.compile(
4    optimizer="adam",
5    loss="binary_crossentropy",
6    metrics=[auc],
7)
8
9checkpoint = tf.keras.callbacks.ModelCheckpoint(
10    filepath="best.keras",
11    monitor="val_auc",
12    mode="max",
13    save_best_only=True,
14)

Again, the key point is exact name matching.

Match the mode to the Metric

The mode argument matters too. Accuracy-like metrics should usually use mode="max", because bigger is better. Loss metrics should usually use mode="min", because smaller is better.

python
1loss_checkpoint = tf.keras.callbacks.ModelCheckpoint(
2    filepath="best-loss.keras",
3    monitor="val_loss",
4    mode="min",
5    save_best_only=True,
6)

A wrong mode does not cause the “available, skipping” message, but it can still make checkpoint selection incorrect.

A Reliable Debugging Routine

When ModelCheckpoint behaves strangely, use this checklist:

  1. print history.history.keys()
  2. confirm validation data exists if the monitor starts with val_
  3. confirm the metric appears in compile(metrics=...)
  4. confirm mode matches the direction of improvement

This is much faster than changing the callback arguments at random.

Common Pitfalls

  • Monitoring val_acc in modern tf.keras where the real key is val_accuracy.
  • Expecting validation metrics without passing validation_data or validation_split.
  • Compiling the model without the metric you want to monitor.
  • Using mode="max" for val_loss or mode="min" for accuracy.
  • Debugging the callback before inspecting history.history.keys().

Summary

  • The skipping message means the monitored metric is missing from the logs.
  • In modern tf.keras, val_accuracy is usually the right replacement for val_acc.
  • Validation metrics exist only when validation data is actually provided.
  • The monitored metric must also be present in compile(metrics=...).
  • Print the history keys first and make the callback match them exactly.

Related reading
Free course
Beginner
7 lessons
2 hours
Tackling System Design Interview Problems

A short course that equips you with the skills to approach system design interviews methodically.

Start the free course
Track what you have practised

A free account saves your progress, solutions and study plan across every problem on Codemia.

ML System Design practice on Codemia

Design recommenders, ranking systems and training pipelines the way ML interviews actually ask for them, with worked solutions.

Practice ML system design

All Rights Reserved.