Introduction

In tf.data, preprocessing is usually applied as elements are consumed, not once up front for the entire dataset. That means a map transformation typically runs for each element as iteration pulls data through the pipeline. The main exceptions are when you explicitly cache or materialize results so later iterations can reuse previously computed values.

The Default Model Is Per-Element, On Demand

A TensorFlow dataset is a pipeline, not a precomputed array of transformed samples. If you define:

1import tensorflow as tf
2
3ds = tf.data.Dataset.range(5)
4ds = ds.map(lambda x: x * 10)

the multiplication is conceptually part of the pipeline. It is applied when elements are read, not eagerly for the whole dataset at pipeline construction time.

If you iterate element by element:

for item in ds:
    print(item.numpy())

the mapped transformation is evaluated as those items flow through the iterator.

iterator.next() Pulls the Next Processed Element

Each call to iterator.next() or each loop step requests the next element from the pipeline. Any upstream map, filter, or parsing logic needed for that element is executed as part of producing that next value.

A small demonstration:

1import tensorflow as tf
2
3def preprocess(x):
4    tf.print("processing", x)
5    return x * 2
6
7ds = tf.data.Dataset.range(3).map(preprocess)
8it = iter(ds)
9
10print(next(it).numpy())
11print(next(it).numpy())

You will see the preprocessing output associated with the elements being requested. The work happens during iteration, not once for the entire dataset definition.

Repeating the Dataset Repeats the Preprocessing

If you iterate again over the same uncached pipeline, the preprocessing runs again for the new pass.

1import tensorflow as tf
2
3def preprocess(x):
4    tf.print("processing", x)
5    return x + 1
6
7ds = tf.data.Dataset.range(2).map(preprocess)
8
9for item in ds:
10    print("first pass", item.numpy())
11
12for item in ds:
13    print("second pass", item.numpy())

Without caching, both passes trigger the map function again.

This is one reason why expensive preprocessing can noticeably increase epoch time when the dataset is revisited many times during training.

Use cache() If You Want Work Done Once and Reused

If preprocessing is deterministic and you want it computed only once, insert .cache() after the expensive step.

1import tensorflow as tf
2
3def preprocess(x):
4    tf.print("processing", x)
5    return x * x
6
7ds = tf.data.Dataset.range(3).map(preprocess).cache()
8
9for item in ds:
10    print("first pass", item.numpy())
11
12for item in ds:
13    print("second pass", item.numpy())

On the first pass, TensorFlow computes and stores the processed elements. On later passes, it reuses the cached results instead of rerunning the mapping logic.

This is the closest behavior to "preprocessing once for the whole dataset," but it only happens because you explicitly added caching.

Prefetch Changes Timing, Not Semantics

prefetch can make it look like preprocessing happens ahead of time, and in a sense it does. TensorFlow may prepare future elements in the background while the current batch is being consumed. But the semantic model is still per-element pipeline execution.

ds = ds.map(preprocess).batch(32).prefetch(tf.data.AUTOTUNE)

This improves throughput, but it does not turn the whole dataset into a one-time preprocessed block by itself.

Think in Terms of Pipeline Stages

A useful mental model is:

1. dataset source provides raw elements
2. transformations process those elements
3. iteration pulls elements through the pipeline
4. optional caching stores processed outputs for reuse

That model explains why TensorFlow can handle datasets larger than memory. It does not need to preprocess everything eagerly before iteration starts.

When Whole-Dataset Preprocessing Happens Outside tf.data

Sometimes teams preprocess data once offline and save the results to TFRecord, NumPy files, or other formats. In that setup, the expensive work happened before the training pipeline started. But that is an application-level design choice, not the default behavior of tf.data.Dataset.map.

If the question is specifically about tf.data transformations, the default answer is still per-element, per-consumption, unless caching or prior materialization changes that behavior.

Common Pitfalls

The biggest mistake is assuming map runs once at dataset definition time. Another is forgetting that a second epoch over an uncached dataset will rerun the preprocessing. Developers also confuse prefetch with permanent caching, even though prefetch only overlaps work and does not guarantee reuse across passes. Expensive deterministic preprocessing often belongs before a .cache() stage if memory or disk caching is acceptable. If the transform includes randomness, caching may also change the intended training behavior, so it has to be used deliberately.

Summary

• In tf.data, preprocessing usually happens per element as the iterator requests data.
• 'iterator.next() pulls the next element through the transformation pipeline.'
• Reiterating over an uncached dataset reruns the preprocessing.
• 'cache() is the main way to compute results once and reuse them across passes.'
• 'prefetch improves overlap and throughput, but it does not by itself make preprocessing one-time.'
• Whole-dataset preprocessing is typically an explicit offline design, not the default tf.data behavior.