Introduction

If you already have MNIST on disk, the correct loading method depends on the exact files you downloaded. Some code expects a packaged mnist.npz file, while other code expects the raw IDX files such as train-images-idx3-ubyte.

That distinction matters more than the directory path itself. Before writing any loader code, confirm which file format you actually have. Once the format is clear, loading from a specific folder is straightforward.

Identify the MNIST Format First

MNIST is commonly stored in one of these forms:

• a Keras-style packaged file such as mnist.npz
• raw IDX files such as train-images-idx3-ubyte and train-labels-idx1-ubyte
• compressed .gz archives containing those raw IDX files

Different libraries know how to read different formats. Many "file not found" or "bad magic number" errors come from using the wrong loader for the files on disk.

Load a Local mnist.npz File With Keras

If you have the Keras-style packaged file already downloaded, tensorflow.keras.datasets.mnist.load_data can load it directly from a path:

1from tensorflow.keras.datasets import mnist
2
3(x_train, y_train), (x_test, y_test) = mnist.load_data(
4    path="data/mnist.npz"
5)
6
7print(x_train.shape, y_train.shape)
8print(x_test.shape, y_test.shape)

This is the easiest case because the file format matches what Keras expects. The path can be relative or absolute.

Load Raw IDX Files Manually

If you downloaded the original IDX files, you can parse them directly with Python and NumPy:

1import struct
2import numpy as np
3
4
5def read_idx_images(path: str) -> np.ndarray:
6    with open(path, "rb") as f:
7        magic, count, rows, cols = struct.unpack(">IIII", f.read(16))
8        if magic != 2051:
9            raise ValueError("not an MNIST image file")
10        data = np.frombuffer(f.read(), dtype=np.uint8)
11        return data.reshape(count, rows, cols)
12
13
14def read_idx_labels(path: str) -> np.ndarray:
15    with open(path, "rb") as f:
16        magic, count = struct.unpack(">II", f.read(8))
17        if magic != 2049:
18            raise ValueError("not an MNIST label file")
19        return np.frombuffer(f.read(), dtype=np.uint8)
20
21
22x_train = read_idx_images("mnist/train-images-idx3-ubyte")
23y_train = read_idx_labels("mnist/train-labels-idx1-ubyte")
24
25print(x_train.shape, y_train.shape)

This approach is useful when you downloaded the dataset from a mirror or unpacked the files yourself.

Handle Compressed Downloads

If the dataset files still end in .gz, decompress them first or write a loader that reads from gzip streams. A simple decompression step looks like this:

1import gzip
2import shutil
3
4
5def decompress_gzip(source: str, target: str) -> None:
6    with gzip.open(source, "rb") as src, open(target, "wb") as dst:
7        shutil.copyfileobj(src, dst)

Many manual-loading failures happen because the code expects plain IDX files while the directory still contains compressed archives.

Inspect and Normalize the Data After Loading

No matter how you load MNIST, verify the shapes and normalize the pixel values before training:

1x_train = x_train.astype("float32") / 255.0
2x_test = x_test.astype("float32") / 255.0
3
4print(x_train.min(), x_train.max())

Checking shape and value range immediately is a good way to catch a wrong loader, damaged files, or an unexpected directory layout before those issues leak into training code.

Common Pitfalls

The most common mistake is using the wrong loader for the file format. A raw IDX directory and a packaged mnist.npz file are not interchangeable.

Another common issue is pointing Keras at a directory when the API expects a filename, or pointing a manual IDX loader at compressed .gz archives instead of decompressed files.

People also assume every library uses the same folder structure. Keras, PyTorch, and custom loaders all have slightly different expectations.

Finally, do not skip the sanity checks. If the array shape or value range is wrong, you want to find that out immediately instead of during model training.

Summary

• First identify whether your local MNIST data is mnist.npz, raw IDX files, or compressed archives.
• Use mnist.load_data(path=...) when you already have a local Keras-style mnist.npz.
• Parse raw IDX files manually when you downloaded the original dataset files.
• Decompress .gz archives if your loader expects plain IDX files.
• Inspect shapes and normalize pixel ranges before using the dataset for training.