tensorflow
image_dataset_from_directory
error
machine learning
image processing

Error in loading image_dataset_from_directory in tensorflow?

ML System Design practice on Codemia

Design recommenders, ranking systems and training pipelines the way ML interviews actually ask for them, with worked solutions.

Practice ML system design

Introduction

image_dataset_from_directory is convenient, but it is strict about directory layout and label handling. When it fails, the cause is usually not TensorFlow itself but a mismatch between your folder structure, your loader arguments, and what the model expects next.

Start with the Expected Directory Structure

The most important assumption is one subdirectory per class.

text
1data/
2  cats/
3    cat1.jpg
4    cat2.jpg
5  dogs/
6    dog1.jpg
7    dog2.jpg

If images are placed directly inside data/ with no class folders, labels="inferred" will not behave the way many people expect.

A minimal working example looks like this:

python
1import tensorflow as tf
2
3dataset = tf.keras.utils.image_dataset_from_directory(
4    "data",
5    labels="inferred",
6    label_mode="int",
7    image_size=(224, 224),
8    batch_size=32,
9    shuffle=True,
10    seed=42,
11)
12
13for images, labels in dataset.take(1):
14    print(images.shape, labels.shape)

If this fails, fix the basic input layout before changing model code.

Check Label Mode Against Class Count

A very common error is using the wrong label_mode.

Good rules:

  • use label_mode="binary" only for exactly two classes
  • use label_mode="int" for integer class indices
  • use label_mode="categorical" for one-hot labels

If you have three or more folders and still use binary, downstream shapes and loss functions will not match.

Validation Split Must Match in Both Calls

When creating training and validation datasets from the same directory, both calls must use the same validation_split and seed.

python
1train_ds = tf.keras.utils.image_dataset_from_directory(
2    "data",
3    validation_split=0.2,
4    subset="training",
5    seed=42,
6    image_size=(224, 224),
7    batch_size=32,
8)
9
10val_ds = tf.keras.utils.image_dataset_from_directory(
11    "data",
12    validation_split=0.2,
13    subset="validation",
14    seed=42,
15    image_size=(224, 224),
16    batch_size=32,
17)

If those parameters differ, the split becomes inconsistent and results can be confusing or invalid.

Corrupt or Unsupported Files

The directory may contain bad files, hidden files, or images with unsupported content. That can cause loading errors that look like TensorFlow bugs.

You can scan the files with Pillow:

python
1from pathlib import Path
2from PIL import Image
3
4for path in Path("data").rglob("*"):
5    if path.suffix.lower() in {".jpg", ".jpeg", ".png"}:
6        try:
7            Image.open(path).verify()
8        except Exception as exc:
9            print("Bad image:", path, exc)

This is a practical way to find broken files before training starts.

Confirm Class Names and Shapes Early

After the dataset loads, inspect the inferred classes immediately:

python
1dataset = tf.keras.utils.image_dataset_from_directory(
2    "data",
3    image_size=(224, 224),
4    batch_size=16,
5)
6
7print(dataset.class_names)

Then inspect one batch:

python
1for images, labels in dataset.take(1):
2    print(images.dtype)
3    print(images.shape)
4    print(labels.shape)

This catches many errors early:

  • wrong image size
  • wrong label shape
  • unexpected class ordering

Color Mode and Model Input Must Match

If the loader outputs RGB images but the model expects grayscale, you will hit a later shape error.

python
1dataset = tf.keras.utils.image_dataset_from_directory(
2    "data",
3    color_mode="rgb",
4    image_size=(128, 128),
5    batch_size=16,
6)

For grayscale workflows, set color_mode="grayscale" and update the model input shape accordingly.

A Good Debugging Sequence

When this function fails, avoid changing five parameters at once. A better sequence is:

  1. confirm folder structure
  2. load with minimal arguments
  3. print class_names
  4. inspect one batch shape
  5. only then add splitting, augmentation, caching, or prefetching

That isolates the real error quickly instead of layering new variables on top of it.

Common Pitfalls

One common mistake is assuming the root directory itself is a class folder. It is not. The classes are the subdirectories.

Another issue is using incompatible label mode and loss-function combinations, such as binary labels with a multiclass model.

A third pitfall is optimizing the pipeline before confirming the data is valid. Prefetching and augmentation do not fix structural dataset errors.

Summary

  • 'image_dataset_from_directory expects one subdirectory per class when labels are inferred.'
  • Match label_mode to the actual number of classes and training setup.
  • Use the same split settings and seed for training and validation datasets.
  • Scan for corrupt image files if loading fails unexpectedly.
  • Print class names and batch shapes early before debugging model code.

Related reading
Free course
Beginner
7 lessons
2 hours
Tackling System Design Interview Problems

A short course that equips you with the skills to approach system design interviews methodically.

Start the free course
Track what you have practised

A free account saves your progress, solutions and study plan across every problem on Codemia.

ML System Design practice on Codemia

Design recommenders, ranking systems and training pipelines the way ML interviews actually ask for them, with worked solutions.

Practice ML system design

All Rights Reserved.