Introduction

Summing a list of tensors is a common TensorFlow task in model code, gradient aggregation, and feature combination pipelines. The best approach depends on whether the tensors have the same shape and whether you want elementwise addition or a scalar total. This article shows the idiomatic TensorFlow options and explains when each one is appropriate.

Elementwise Sum of Same-Shaped Tensors

If every tensor has the same shape and dtype, tf.add_n is usually the cleanest solution. It performs elementwise addition across the list.

1import tensorflow as tf
2
3tensors = [
4    tf.constant([1.0, 2.0, 3.0]),
5    tf.constant([4.0, 5.0, 6.0]),
6    tf.constant([7.0, 8.0, 9.0]),
7]
8
9result = tf.add_n(tensors)
10print(result.numpy())  # [12. 15. 18.]

This is not a Python-side sum of individual numbers. It adds corresponding tensor elements and returns another tensor with the same shape.

Use tf.stack Plus tf.reduce_sum When You Need an Extra Axis

Another common pattern is stacking tensors and summing along the new axis.

1import tensorflow as tf
2
3tensors = [
4    tf.constant([[1, 2], [3, 4]]),
5    tf.constant([[10, 20], [30, 40]]),
6    tf.constant([[100, 200], [300, 400]]),
7]
8
9stacked = tf.stack(tensors, axis=0)
10result = tf.reduce_sum(stacked, axis=0)
11
12print(stacked.shape)   # (3, 2, 2)
13print(result.numpy())

This is useful if you also want access to the stacked representation for debugging or additional reductions such as mean or max.

Scalar Total Over All Elements in All Tensors

Sometimes you do not want an elementwise tensor result. You want one scalar containing the sum of every element across every tensor.

1import tensorflow as tf
2
3tensors = [
4    tf.constant([1, 2, 3]),
5    tf.constant([4, 5]),
6    tf.constant([6]),
7]
8
9total = tf.add_n([tf.reduce_sum(t) for t in tensors])
10print(total.numpy())  # 21

Here each tensor is reduced to a scalar first, then those scalars are added together.

What Happens with Python sum

Python sum can work in simple eager-mode cases, but it is usually not the best TensorFlow idiom.

1import tensorflow as tf
2
3tensors = [
4    tf.constant([1.0, 2.0]),
5    tf.constant([3.0, 4.0]),
6]
7
8result = sum(tensors)
9print(result.numpy())

This may behave correctly for small examples, but tf.add_n is clearer and better signals intent to readers. It also avoids relying on Python's accumulation behavior when graph tracing or mixed types are involved.

Lists of Variable Shapes Need a Different Strategy

tf.add_n and tf.stack require compatible shapes. If your list contains tensors with different lengths, decide which kind of sum you actually need.

If you want a scalar total, reduce each tensor separately:

1import tensorflow as tf
2
3tensors = [
4    tf.constant([1.0, 2.0]),
5    tf.constant([3.0, 4.0, 5.0]),
6]
7
8total = tf.add_n([tf.reduce_sum(t) for t in tensors])
9print(total.numpy())  # 15.0

If you want elementwise addition, you must first pad or otherwise align the shapes.

Example Inside a Training Step

Gradient aggregation is a realistic use case. The example below sums gradient tensors from two steps before applying them.

1import tensorflow as tf
2
3g1 = [tf.constant([0.1, 0.2]), tf.constant([0.3])]
4g2 = [tf.constant([0.4, 0.5]), tf.constant([0.6])]
5
6summed_gradients = [tf.add_n(parts) for parts in zip(g1, g2)]
7
8for g in summed_gradients:
9    print(g.numpy())

Each position in the gradient list is summed independently. This is the same shape discipline you need when aggregating gradients across replicas or mini-batches.

Behavior Inside tf.function

These operations work well inside graph-traced functions too.

1import tensorflow as tf
2
3@tf.function
4def sum_tensors(tensors):
5    return tf.add_n(tensors)
6
7result = sum_tensors([
8    tf.constant([1.0, 2.0]),
9    tf.constant([3.0, 4.0]),
10])
11
12print(result)

If you are building production TensorFlow code, prefer TensorFlow ops over Python-side list arithmetic so tracing remains predictable.

Choosing the Right Method

Use this rule of thumb:

• 'tf.add_n for elementwise sum across same-shaped tensors'
• 'tf.stack plus tf.reduce_sum when you also need a stacked axis'
• 'tf.reduce_sum on each tensor first when shapes differ and you want one scalar total'

That small distinction prevents many shape errors.

Common Pitfalls

• Using tf.add_n on tensors with incompatible shapes.
• Expecting tf.reduce_sum on a Python list to automatically do the right thing.
• Confusing elementwise tensor addition with a scalar total over all elements.
• Relying on Python sum in code that should stay clearly TensorFlow-native.
• Forgetting that gradient lists must be aggregated position by position.

Summary

• 'tf.add_n is the standard way to sum a list of same-shaped tensors elementwise.'
• 'tf.stack plus tf.reduce_sum is useful when you need the intermediate stacked dimension.'
• For variable-sized tensors, reduce each tensor first if you want a scalar total.
• Keep TensorFlow arithmetic in TensorFlow ops when writing traced or production code.
• Always decide up front whether you want elementwise output or one scalar total.