Introduction

Reducing a tensor by sum along a certain dimension means summing values across one or more axes while leaving the other axes intact. The operation is simple once you remember one rule: the axis you pass is the axis that disappears unless you explicitly keep it. Most confusion comes from mixing up rows and columns or from expecting the output shape to stay the same.

The Basic Idea of reduce_sum

In TensorFlow, tf.reduce_sum sums elements across an axis:

1import tensorflow as tf
2
3x = tf.constant([
4    [1, 2, 3],
5    [4, 5, 6],
6], dtype=tf.int32)
7
8print(tf.reduce_sum(x))

Output:

tf.Tensor(21, shape=(), dtype=int32)

No axis was provided, so TensorFlow summed every element in the tensor and returned a scalar.

Sum Along One Axis

Now sum by a specific dimension:

1import tensorflow as tf
2
3x = tf.constant([
4    [1, 2, 3],
5    [4, 5, 6],
6], dtype=tf.int32)
7
8print(tf.reduce_sum(x, axis=0))  # column-wise
9print(tf.reduce_sum(x, axis=1))  # row-wise

Output:

tf.Tensor([5 7 9], shape=(3,), dtype=int32)
tf.Tensor([ 6 15], shape=(2,), dtype=int32)

Interpretation:

• 'axis=0 collapses rows and sums each column'
• 'axis=1 collapses columns and sums each row'

That is the core pattern for most reduce_sum questions.

Think in Terms of Shape

Shape reasoning makes the axis argument much easier to understand.

If x has shape (2, 3):

• reduce on axis=0 and the 0 axis disappears, leaving shape (3,)
• reduce on axis=1 and the 1 axis disappears, leaving shape (2,)

So a good debugging habit is:

print(x.shape)
print(tf.reduce_sum(x, axis=0).shape)
print(tf.reduce_sum(x, axis=1).shape)

When the output shape surprises you, the axis choice is usually the reason.

Keep the Reduced Dimension with keepdims=True

Sometimes you want the reduced axis to remain as a size-1 dimension, especially for broadcasting:

1import tensorflow as tf
2
3x = tf.constant([
4    [1.0, 2.0, 3.0],
5    [4.0, 5.0, 6.0],
6])
7
8result = tf.reduce_sum(x, axis=1, keepdims=True)
9print(result)
10print(result.shape)

Output:

tf.Tensor(
[[ 6.]
 [15.]], shape=(2, 1), dtype=float32)

Without keepdims=True, the shape would be (2,). Keeping dimensions is often helpful in normalization or broadcasting workflows.

Reduce Across Multiple Dimensions

You can reduce across more than one axis at once:

1import tensorflow as tf
2
3x = tf.ones((2, 3, 4), dtype=tf.int32)
4result = tf.reduce_sum(x, axis=[1, 2])
5
6print(result)
7print(result.shape)

Output:

tf.Tensor([12 12], shape=(2,), dtype=int32)

Here TensorFlow sums across the last two dimensions and keeps only the first one.

This is common in batch-oriented code, where you want one summary value per batch item.

Negative Axes Also Work

TensorFlow lets you refer to axes from the end:

1import tensorflow as tf
2
3x = tf.constant([
4    [1, 2, 3],
5    [4, 5, 6],
6])
7
8print(tf.reduce_sum(x, axis=-1))

axis=-1 means "the last dimension," so this gives the same result as axis=1 for a rank-2 tensor.

Negative axes are handy when the exact rank may vary but the last feature dimension still has the same meaning.

A Practical Example

Suppose you have a batch of model outputs shaped:

(batch_size, timesteps, features)

and you want the sum across timesteps:

1import tensorflow as tf
2
3x = tf.ones((2, 4, 3))
4result = tf.reduce_sum(x, axis=1)
5print(result.shape)

The result shape is:

(2, 3)

because the time dimension was reduced away.

This kind of shape reasoning is more important than memorizing examples.

Common Pitfalls

The biggest mistake is mixing up axis=0 and axis=1. In a matrix, axis=0 reduces rows and leaves columns, while axis=1 reduces columns and leaves rows.

Another issue is forgetting that the reduced dimension disappears unless keepdims=True is used. That often causes unexpected broadcasting or shape-mismatch problems later.

Developers also sometimes pass data shaped differently than they assume. When the input shape is wrong, the axis argument can look wrong even though the real issue is the tensor layout.

Finally, reducing all dimensions by accident can return a scalar when you expected a vector or matrix. If that happens, check whether you omitted axis entirely.

Summary

• 'reduce_sum sums values across the axis or axes you specify.'
• The reduced axis disappears unless you set keepdims=True.
• 'axis=0 and axis=1 have different meanings because they reduce different dimensions.'
• Output shape is usually the best guide to whether the reduction axis was chosen correctly.
• For debugging, print both the input shape and the reduced output shape before assuming the math is wrong.