Introduction

Visualizing learned weights is one of the simplest ways to inspect what a neural network has actually stored during training. With MNIST, this works especially well because each input image is small, grayscale, and has a fixed 28 x 28 structure. For some layers, you can turn the weights back into images and inspect which pixel patterns a neuron is responding to.

The most important detail is that not every layer can be visualized the same way. A dense layer connected directly to pixels can often be reshaped into an image. A convolutional layer already stores spatial kernels, so you visualize filters instead. Once you know which tensor you are looking at, the plots become much easier to interpret.

Start with a Simple Dense Model

The cleanest example is a small fully connected model that receives flattened MNIST images. In that setup, each neuron in the first hidden layer has one weight per input pixel.

1import tensorflow as tf
2from tensorflow import keras
3
4(x_train, y_train), _ = keras.datasets.mnist.load_data()
5
6x_train = x_train.astype("float32") / 255.0
7x_train = x_train.reshape(-1, 28 * 28)
8
9model = keras.Sequential([
10    keras.layers.Input(shape=(28 * 28,)),
11    keras.layers.Dense(16, activation="relu", name="hidden"),
12    keras.layers.Dense(10, activation="softmax", name="output"),
13])
14
15model.compile(
16    optimizer="adam",
17    loss="sparse_categorical_crossentropy",
18    metrics=["accuracy"],
19)
20
21model.fit(x_train, y_train, epochs=3, batch_size=128, verbose=0)

After training, the first dense layer has a kernel of shape 784 x 16. That means:

• 784 input pixels per neuron
• 16 hidden neurons

Each column of that matrix is one neuron's learned view of the input image.

Extract the Learned Weights

Keras exposes layer weights through get_weights(). For a dense layer, the result is usually:

1. the kernel matrix
2. the bias vector

1kernel, bias = model.get_layer("hidden").get_weights()
2
3print(kernel.shape)  # (784, 16)
4print(bias.shape)    # (16,)

If you want to visualize neuron i, select column i from the kernel and reshape it back into the original image size:

image_like_weights = kernel[:, 0].reshape(28, 28)
print(image_like_weights.shape)

That gives a 28 x 28 array that can be plotted with matplotlib.

Plot Weight Maps as Images

The following example shows all 16 hidden neurons in a 4 x 4 grid:

1import matplotlib.pyplot as plt
2
3fig, axes = plt.subplots(4, 4, figsize=(8, 8))
4
5for i, ax in enumerate(axes.flat):
6    weight_image = kernel[:, i].reshape(28, 28)
7    ax.imshow(weight_image, cmap="seismic")
8    ax.set_title(f"Neuron {i}")
9    ax.axis("off")
10
11plt.tight_layout()
12plt.show()

A diverging colormap such as seismic, coolwarm, or bwr is usually better than a grayscale map for weights. Positive and negative values matter in different ways, so it helps to show both directions clearly.

What you often see are blurry stroke-like templates rather than whole digits. That is expected. Early hidden units usually learn fragments such as vertical bars, loops, edge regions, or diagonal strokes.

What These Images Actually Mean

This is where people often over-interpret the plots. A dense neuron's weight image is not the same thing as "the digit the neuron likes." It is more accurate to think of it as a learned linear pattern over the input pixels before the activation function is applied.

For example:

• bright positive areas mean those pixels push the neuron upward
• dark negative areas mean those pixels suppress the neuron
• near-zero areas contribute little

So if a neuron has a bright vertical band in the center, it may respond strongly to digits containing a central stroke. But the final behavior still depends on the rest of the network, not just that one image.

It is also worth noting that the output layer is different. Its weights connect hidden activations to class scores, not raw pixels, so reshaping those weights into 28 x 28 images usually does not make sense.

Convolutional Models Should Be Visualized Differently

If you train a convolutional network on MNIST, the first layer does not store a 784 x N matrix. Instead, it stores small filters such as 3 x 3 x 1 x 8, which means:

• kernel height 3
• kernel width 3
• one input channel
• eight output filters

Here is a small convolutional example:

1conv_model = keras.Sequential([
2    keras.layers.Input(shape=(28, 28, 1)),
3    keras.layers.Conv2D(8, kernel_size=3, activation="relu", name="conv1"),
4    keras.layers.Flatten(),
5    keras.layers.Dense(10, activation="softmax"),
6])
7
8conv_kernel, conv_bias = conv_model.get_layer("conv1").get_weights()
9print(conv_kernel.shape)  # (3, 3, 1, 8)

To plot those filters, select each output channel:

1fig, axes = plt.subplots(1, 8, figsize=(12, 2))
2
3for i, ax in enumerate(axes):
4    ax.imshow(conv_kernel[:, :, 0, i], cmap="seismic")
5    ax.set_title(f"F{i}")
6    ax.axis("off")
7
8plt.tight_layout()
9plt.show()

In a convolutional model, these filter plots are usually more meaningful than trying to force everything into full-image weight maps.

Common Pitfalls

The most common mistake is reshaping the wrong tensor. Only layers directly connected to flattened pixel inputs have weights that naturally reshape back to 28 x 28.

Another issue is plotting untrained weights and expecting structure. Before training, the images mostly show random initialization noise.

A third mistake is ignoring scaling. If the colormap range changes from plot to plot, visual comparisons become misleading. When comparing neurons, it can help to fix vmin and vmax across all subplots.

Finally, weight visualization is not the same as activation visualization. Weights tell you what was learned in the parameters. Activations tell you how a specific input image excites the network. Both are useful, but they answer different questions.

Summary

• For a dense MNIST model, first-layer weights can often be reshaped into 28 x 28 images.
• get_weights() returns the kernel matrix and bias vector for a layer.
• Each dense neuron corresponds to one column of the kernel matrix.
• Convolutional models should usually be visualized through their learned filters instead.
• Weight plots are a helpful inspection tool, but they should be interpreted as parameter patterns, not literal digit templates.