Introduction

When people ask how to split a TensorFlow dataset into training, validation, and test sets using the Keras preprocessing API, the important answer is that Keras preprocessing can create training and validation splits directly, but test data usually needs its own separate handling. If you try to force all three partitions out of one convenience call, the API shape often becomes the real source of confusion.

The Core Split: Training and Validation

For image directories, the current Keras-friendly pattern is image_dataset_from_directory with validation_split, subset, and a fixed seed.

1import tensorflow as tf
2
3train_ds = tf.keras.utils.image_dataset_from_directory(
4    "images",
5    validation_split=0.2,
6    subset="training",
7    seed=42,
8    image_size=(224, 224),
9    batch_size=32,
10)
11
12val_ds = tf.keras.utils.image_dataset_from_directory(
13    "images",
14    validation_split=0.2,
15    subset="validation",
16    seed=42,
17    image_size=(224, 224),
18    batch_size=32,
19)

The shared seed and validation_split keep the two subsets aligned so they partition the same directory consistently.

That solves only two splits:

• training
• validation

There is no automatic third "test" subset in that call.

Where the Test Set Should Come From

The cleanest approach is to keep test data separate from the start. For example:

1dataset/
2  train_val/
3    cats/
4    dogs/
5  test/
6    cats/
7    dogs/

Then use preprocessing to split only train_val:

1test_ds = tf.keras.utils.image_dataset_from_directory(
2    "dataset/test",
3    shuffle=False,
4    image_size=(224, 224),
5    batch_size=32,
6)

This mirrors good machine-learning practice. The test set should be untouched by training-time decisions, so keeping it physically separate is often the least error-prone design.

If You Only Have One Directory

Sometimes you start with one directory and still need three partitions. In that case, use preprocessing to get two splits first, then split one of those again with tf.data.

Example idea:

1. create a train+validation and test split outside the preprocessing helper, or
2. load everything and manually slice the resulting dataset

Here is a tf.data style example using cardinality:

1import tensorflow as tf
2
3all_ds = tf.keras.utils.image_dataset_from_directory(
4    "images",
5    shuffle=True,
6    seed=42,
7    image_size=(224, 224),
8    batch_size=32,
9)
10
11dataset_size = tf.data.experimental.cardinality(all_ds).numpy()
12test_batches = int(0.2 * dataset_size)
13val_batches = int(0.2 * dataset_size)
14
15test_ds = all_ds.take(test_batches)
16remaining = all_ds.skip(test_batches)
17val_ds = remaining.take(val_batches)
18train_ds = remaining.skip(val_batches)

This works, but it is less explicit than a directory-level split, and the split is based on batches rather than raw examples unless you take extra care.

That is why a separate test directory is usually cleaner.

Older keras.preprocessing.image.ImageDataGenerator

Older tutorials often use ImageDataGenerator with validation_split:

1from tensorflow.keras.preprocessing.image import ImageDataGenerator
2
3datagen = ImageDataGenerator(rescale=1.0 / 255, validation_split=0.2)
4
5train_gen = datagen.flow_from_directory(
6    "images",
7    target_size=(224, 224),
8    batch_size=32,
9    subset="training",
10)
11
12val_gen = datagen.flow_from_directory(
13    "images",
14    target_size=(224, 224),
15    batch_size=32,
16    subset="validation",
17)

That API has the same limitation: it gives you training and validation, not a magical third test split. If you need a test set, keep another directory or perform a separate split yourself.

Reproducibility Matters

Whenever you rely on automatic splitting, keep the split reproducible:

• set a fixed seed
• use the same split parameters in both calls
• avoid reshuffling differently between train and validation creation

Without those, the train and validation subsets may not partition the source data the way you think they do.

A Practical Recommendation

For maintainable pipelines, use this rule:

• training and validation can come from the same source directory using a split helper
• test data should usually live in a separate directory or separate manifest

That keeps evaluation honest and makes the pipeline easier to explain to other people on the project.

If you later need cross-validation or more elaborate experimental design, convenience splitting inside preprocessing is often too narrow anyway.

Common Pitfalls

The biggest mistake is assuming Keras preprocessing has a built-in three-way training/validation/test subset switch. It does not.

Another issue is creating training and validation datasets with different seeds or different split parameters, which breaks the intended partitioning.

Developers also often split after batching without realizing they are splitting batches rather than exact example counts. That may be acceptable, but it should be intentional.

Finally, do not let test data participate in tuning decisions. If the test set comes from the same convenience split loop you repeatedly rerun while experimenting, it is no longer a trustworthy final evaluation set.

Summary

• Keras preprocessing helpers can directly create training and validation splits.
• Test data is usually best kept separate rather than generated from the same helper call.
• Use the same seed, validation_split, and source directory when building matching training and validation datasets.
• Manual tf.data splitting is possible, but it is less explicit and can be batch-based.
• A physically separate test set is usually the cleanest and safest design.