tensorflow
tf.data
dataset manipulation
data indexing
data preprocessing

accessing indexes of tf.data.Dataset for deleting and appending data elements

Master System Design with Codemia

Enhance your system design skills with over 120 practice problems, detailed solutions, and hands-on exercises.

Introduction

tf.data.Dataset is not a mutable Python list, so there is no direct API for "delete element at index 5" or "append one item to the end" in place. The normal pattern is to build a new dataset from transformations such as enumerate, filter, concatenate, or by materializing the data elsewhere if true random mutation is required.

Why Direct Index Mutation Does Not Exist

tf.data.Dataset is designed as a streaming, transformation-based pipeline. It is optimized for:

  • sequential iteration
  • lazy evaluation
  • composable transformations

It is not designed as an editable container with list-like mutation semantics.

That means operations such as delete and append are expressed by creating a new dataset, not by modifying the original one in place.

Simulate Indexes with enumerate

If you need index-aware filtering, start by enumerating the dataset.

python
1import tensorflow as tf
2
3ds = tf.data.Dataset.from_tensor_slices([10, 20, 30, 40, 50])
4indexed = ds.enumerate()
5
6for idx, value in indexed:
7    print(int(idx.numpy()), int(value.numpy()))

This gives you index-value pairs that you can then filter.

Delete an Element by Index

To remove the element at index 2:

python
1import tensorflow as tf
2
3ds = tf.data.Dataset.from_tensor_slices([10, 20, 30, 40, 50])
4
5filtered = (
6    ds.enumerate()
7      .filter(lambda idx, value: idx != 2)
8      .map(lambda idx, value: value)
9)
10
11print(list(filtered.as_numpy_iterator()))

This creates a new dataset without the unwanted element.

Append Elements by Concatenation

To append values, build another dataset and concatenate it.

python
1import tensorflow as tf
2
3left = tf.data.Dataset.from_tensor_slices([10, 20, 30])
4right = tf.data.Dataset.from_tensor_slices([40, 50])
5
6combined = left.concatenate(right)
7print(list(combined.as_numpy_iterator()))

This is the dataset equivalent of appending, but again the result is a new dataset pipeline.

Replace Rather Than Mutate

If your real goal is to replace one item by index, combine filtering and concatenation logic or map over the enumerated dataset.

python
1import tensorflow as tf
2
3ds = tf.data.Dataset.from_tensor_slices([10, 20, 30, 40])
4
5updated = (
6    ds.enumerate()
7      .map(lambda idx, value: tf.cond(idx == 1, lambda: 999, lambda: value))
8)
9
10print(list(updated.as_numpy_iterator()))

This is often a better mental model than asking how to mutate the dataset directly.

When to Materialize Instead

If the dataset is small and true random editing matters more than streaming performance, it may be simpler to convert it to a Python or NumPy structure, edit it there, and recreate the dataset.

python
1import tensorflow as tf
2
3ds = tf.data.Dataset.from_tensor_slices([10, 20, 30, 40])
4values = list(ds.as_numpy_iterator())
5
6del values[1]
7values.append(99)
8
9new_ds = tf.data.Dataset.from_tensor_slices(values)
10print(list(new_ds.as_numpy_iterator()))

This is perfectly reasonable for small in-memory data, but it defeats the main benefits of tf.data for large streaming pipelines.

Choose Based on Scale

Use dataset transformations when:

  • the data is large
  • the pipeline should stay lazy
  • the data source may not fit in memory

Materialize outside tf.data when:

  • the dataset is small
  • random editing is the real requirement
  • simplicity matters more than streaming behavior

That tradeoff is the real design decision.

Common Pitfalls

The biggest mistake is expecting list-like random-access mutation from tf.data.Dataset. That is not what the API is built for.

Another issue is forcing complex index-based edits inside tf.data when the data is small enough that a plain Python list would be simpler.

A third problem is forgetting to remove the index again after enumerate, which leaves your downstream pipeline working with (index, value) pairs unexpectedly.

Summary

  • 'tf.data.Dataset does not support in-place delete or append by index.'
  • Use enumerate plus filter to remove items by position.
  • Use concatenate to append additional dataset elements.
  • Use mapping logic when you want replacement instead of deletion.
  • For small editable data, materializing to Python and rebuilding the dataset is often simpler.

Course illustration
Course illustration

All Rights Reserved.