Introduction

If you are only getting about 10% accuracy on CIFAR-100, the first question is whether the training pipeline is correct before you start chasing exotic model ideas. Random guessing on CIFAR-100 is about 1%, so 10% means the model is learning something, but it is still far below what a solid modern baseline should achieve.

Start With Data and Labels

CIFAR-100 has 100 classes and tiny 32x32 color images, so preprocessing mistakes matter a lot. Before tuning the network, verify:

• images are scaled consistently
• labels match the loss function
• train and test sets are not mixed
• the final layer has 100 outputs

If you use integer labels, pair them with sparse categorical loss. If you one-hot encode labels, use categorical loss instead.

1import tensorflow as tf
2
3(x_train, y_train), (x_test, y_test) = tf.keras.datasets.cifar100.load_data()
4
5x_train = x_train.astype("float32") / 255.0
6x_test = x_test.astype("float32") / 255.0
7
8model = tf.keras.Sequential([
9    tf.keras.layers.Input(shape=(32, 32, 3)),
10    tf.keras.layers.Flatten(),
11    tf.keras.layers.Dense(100, activation="softmax"),
12])
13
14model.compile(
15    optimizer="adam",
16    loss="sparse_categorical_crossentropy",
17    metrics=["accuracy"],
18)

If the output layer or loss is mismatched, training quality can collapse even though the code still runs.

Use a Stronger Baseline Model

A tiny fully connected network usually performs poorly on CIFAR-100 because it ignores image structure. Convolutional models are the normal baseline, and residual networks are often much better than plain stacks of conv layers.

Here is a small convolutional baseline:

1import tensorflow as tf
2
3model = tf.keras.Sequential([
4    tf.keras.layers.Input(shape=(32, 32, 3)),
5    tf.keras.layers.Conv2D(64, 3, padding="same", activation="relu"),
6    tf.keras.layers.BatchNormalization(),
7    tf.keras.layers.Conv2D(64, 3, padding="same", activation="relu"),
8    tf.keras.layers.MaxPooling2D(),
9    tf.keras.layers.Conv2D(128, 3, padding="same", activation="relu"),
10    tf.keras.layers.BatchNormalization(),
11    tf.keras.layers.Conv2D(128, 3, padding="same", activation="relu"),
12    tf.keras.layers.MaxPooling2D(),
13    tf.keras.layers.GlobalAveragePooling2D(),
14    tf.keras.layers.Dropout(0.3),
15    tf.keras.layers.Dense(100, activation="softmax"),
16])

That will not be state of the art, but it is much closer to a useful baseline than a plain multilayer perceptron.

Add Data Augmentation

CIFAR-100 is small, so augmentation often makes a visible difference. Basic random flips and small translations help the model generalize instead of memorizing.

1data_augmentation = tf.keras.Sequential([
2    tf.keras.layers.RandomFlip("horizontal"),
3    tf.keras.layers.RandomTranslation(0.1, 0.1),
4    tf.keras.layers.RandomRotation(0.05),
5])
6
7inputs = tf.keras.Input(shape=(32, 32, 3))
8x = data_augmentation(inputs)
9x = tf.keras.layers.Conv2D(64, 3, padding="same", activation="relu")(x)
10x = tf.keras.layers.GlobalAveragePooling2D()(x)
11outputs = tf.keras.layers.Dense(100, activation="softmax")(x)
12
13model = tf.keras.Model(inputs, outputs)

Even simple augmentation can outperform endless random hyperparameter changes when the original input pipeline is too weak.

Tune the Training Process

Once the model and data pipeline are reasonable, the next levers are:

• learning rate
• optimizer choice
• weight decay or regularization
• number of epochs
• batch size
• learning rate schedule

A high constant learning rate can trap the model at poor accuracy. On the other hand, training for too few epochs can make a decent model look broken. For CIFAR-100, a schedule that reduces the learning rate over time is often much stronger than a fixed default optimizer setting.

Watch the Curves, Not Just One Number

Training accuracy and validation accuracy tell you what kind of problem you have. If both are low, the model or optimization setup is weak. If training accuracy is high but validation accuracy is poor, the model is overfitting and needs augmentation, regularization, or a better architecture.

That is why accuracy alone is not enough. The training curves tell you whether to focus on model capacity, optimization, or generalization.

Common Pitfalls

The first mistake is using the wrong loss for the label format. Sparse labels with categorical loss, or one-hot labels with sparse loss, can quietly damage training.

Another common issue is using a model that is too weak for CIFAR-100. A simple dense network can learn something, but it usually leaves a lot of accuracy on the table.

Developers also sometimes skip normalization and augmentation, which makes the model work harder than necessary on a difficult 100-class dataset.

Finally, do not judge the run too early. CIFAR-100 can need more careful optimization than CIFAR-10 because the classification task is much finer-grained.

Summary

• '10% accuracy usually means the pipeline is learning, but the baseline is still weak.'
• Verify label format, loss function, preprocessing, and output size before deeper tuning.
• Use a convolutional or residual model instead of a plain dense network.
• Add basic data augmentation because CIFAR-100 benefits from it.
• Inspect training and validation curves so you know whether the problem is underfitting or overfitting.