Introduction

Building a NumPy array from a generator is possible, but the best method depends on what the generator yields. If it yields a flat stream of scalar values, np.fromiter is usually the cleanest and most efficient option. If it yields lists, tuples, or other row-like structures, converting to list first and then calling np.array is often simpler.

The key questions are whether the output is one-dimensional, whether the dtype is known, and whether you need to preserve streaming behavior as long as possible.

Use np.fromiter for 1D Scalar Output

If the generator yields one scalar at a time, use np.fromiter:

1import numpy as np
2
3def numbers():
4    for i in range(5):
5        yield i * 2
6
7arr = np.fromiter(numbers(), dtype=np.int64)
8print(arr)

This produces a one-dimensional NumPy array without first materializing a Python list of all values.

dtype is required because NumPy needs to know what kind of array to allocate.

Build Multi-Dimensional Arrays with np.array(list(...))

If the generator yields rows rather than scalars, np.fromiter is not the right fit. In that case, collect the rows and pass them to np.array:

1import numpy as np
2
3def rows():
4    for i in range(3):
5        yield [i, i + 1, i + 2]
6
7arr = np.array(list(rows()), dtype=np.int64)
8print(arr)

This is often the most readable approach for two-dimensional data.

It is less streaming-friendly because the list is built first, but it matches how NumPy expects shaped row data.

Know the Trade-Off

The difference between the two methods is mainly about shape and memory behavior.

Use np.fromiter when:

• the generator yields scalar values
• you want a 1D array
• you know the dtype in advance

Use np.array(list(generator)) when:

• the generator yields rows or nested structures
• you want NumPy to infer a 2D or higher-dimensional shape
• clarity matters more than keeping the generator lazy until the last moment

In other words, fromiter is specialized and efficient, while array(list(...)) is more general. Picking the right one early saves you from awkward reshaping code later. It also makes it easier to reason about memory use before the dataset gets large.

Count Can Help Performance

If you know how many scalar elements the generator will produce, pass count to np.fromiter:

arr = np.fromiter(numbers(), dtype=np.int64, count=5)

This lets NumPy allocate more efficiently because it knows the final size ahead of time.

It is optional, but useful when the size is known cheaply.

Remember That Generators Are One-Shot

A generator is consumed as it is iterated. Once NumPy has built the array, you usually cannot iterate the same generator again and expect the same data:

1g = numbers()
2a = np.fromiter(g, dtype=np.int64)
3b = np.fromiter(g, dtype=np.int64)
4print(a)
5print(b)  # empty, because the generator is exhausted

That behavior surprises people more often than the NumPy API itself.

Common Pitfalls

• Using np.fromiter on a generator of lists when it is meant for scalar iteration.
• Forgetting to provide dtype to np.fromiter.
• Expecting to reuse the same generator after it has already been consumed.
• Materializing a huge generator into a list unnecessarily when a 1D fromiter solution would work.

Summary

• Use np.fromiter(generator, dtype=...) for generators that yield scalar values.
• Use np.array(list(generator)) when the generator yields rows or nested structures.
• Provide count to np.fromiter if you know the final number of elements.
• Remember that generators are consumed once.
• Choose the method based on output shape, dtype knowledge, and memory needs.