Introduction

When your training data is spread across many CSV files, the goal is usually the same: stream rows efficiently, parse them into tensors, batch them, and feed the model without loading everything into memory. TensorFlow's tf.data API is built for exactly that workflow.

The core pattern is list_files, then interleave or flat_map to read each file, then map to parse each line, followed by batch and prefetch. Once you understand that pipeline, CSV input becomes predictable and scalable.

Build a File-Based Input Pipeline

Suppose you have many files matching data/train-*.csv, each with a header and rows shaped like feature1,feature2,label.

1import tensorflow as tf
2
3AUTOTUNE = tf.data.AUTOTUNE
4
5file_pattern = "data/train-*.csv"
6batch_size = 32

Start by listing files:

files = tf.data.Dataset.list_files(file_pattern, shuffle=True)

files is a dataset of file paths, not CSV rows. Next, convert each path into a line dataset and combine them:

1dataset = files.interleave(
2    lambda path: tf.data.TextLineDataset(path).skip(1),
3    cycle_length=4,
4    num_parallel_calls=AUTOTUNE,
5    deterministic=False,
6)

Using skip(1) removes the header line from every file. interleave reads from several files concurrently, which often improves throughput.

Parse Each CSV Row with map

Now define a parser. tf.io.decode_csv turns a comma-separated line into typed tensors.

1record_defaults = [0.0, 0.0, 0]
2
3def parse_csv_line(line: tf.Tensor):
4    feature1, feature2, label = tf.io.decode_csv(
5        line,
6        record_defaults=record_defaults,
7    )
8
9    features = tf.stack([feature1, feature2])
10    label = tf.cast(label, tf.int32)
11    return features, label

Map the parser across the dataset:

dataset = dataset.map(parse_csv_line, num_parallel_calls=AUTOTUNE)

At this point, each element is a pair of tensors shaped like (features, label).

Batch, Shuffle, and Prefetch

After parsing, add the training-oriented transformations:

dataset = dataset.shuffle(10_000)
dataset = dataset.batch(batch_size)
dataset = dataset.prefetch(AUTOTUNE)

That gives you batches ready for model training:

for features, labels in dataset.take(1):
    print(features.shape)
    print(labels.shape)

A complete training example looks like this:

1model = tf.keras.Sequential([
2    tf.keras.layers.Input(shape=(2,)),
3    tf.keras.layers.Dense(16, activation="relu"),
4    tf.keras.layers.Dense(1, activation="sigmoid"),
5])
6
7model.compile(optimizer="adam", loss="binary_crossentropy", metrics=["accuracy"])
8model.fit(dataset, epochs=5)

When make_csv_dataset Is Simpler

If your CSV structure is conventional and you want TensorFlow to handle more of the parsing, tf.data.experimental.make_csv_dataset can be easier:

1dataset = tf.data.experimental.make_csv_dataset(
2    file_pattern=file_pattern,
3    batch_size=32,
4    label_name="label",
5    num_epochs=1,
6    header=True,
7    shuffle=True,
8    num_parallel_reads=AUTOTUNE,
9)

This returns batches of feature dictionaries plus labels. It is convenient, but the manual TextLineDataset plus map approach gives you finer control and teaches the underlying mechanics.

Why map Matters

map is where feature engineering usually lives. You can normalize columns, cast dtypes, derive labels, or combine fields before the batch reaches the model.

1def preprocess(features, label):
2    normalized = (features - tf.constant([10.0, 5.0])) / tf.constant([2.0, 1.5])
3    return normalized, label
4
5dataset = dataset.map(preprocess, num_parallel_calls=AUTOTUNE)

This keeps preprocessing close to the data pipeline and lets TensorFlow parallelize it.

Common Pitfalls

• Calling batch before parsing lines usually makes the parser logic harder because each element becomes a batch of strings.
• Forgetting skip(1) when files contain headers causes CSV parsing failures.
• Using the wrong record_defaults types can silently coerce data incorrectly or fail at runtime.
• Shuffling with too small a buffer reduces randomness, especially across many files.
• Building the dataset without prefetch can leave the model waiting on input instead of training continuously.

Summary

• Use Dataset.list_files to discover many CSV files.
• Use interleave and TextLineDataset to stream lines from multiple files efficiently.
• Parse each line with map and tf.io.decode_csv.
• Add shuffle, batch, and prefetch for training performance.
• Reach for make_csv_dataset when you want a higher-level CSV loader with less manual control.