tf.reduce_sum
TensorFlow
axis parameter
deep learning
reduce operation

What does tf.reduce_sum do with axis -1?

Master System Design with Codemia

Enhance your system design skills with over 120 practice problems, detailed solutions, and hands-on exercises.

Introduction

In TensorFlow, axis=-1 means "use the last dimension." So tf.reduce_sum(..., axis=-1) sums across the last axis of the tensor, no matter how many dimensions the tensor has. That is why negative axes are handy: they let you refer to dimensions relative to the end instead of counting from the front.

A Simple Two-Dimensional Example

Start with a matrix:

python
1import tensorflow as tf
2
3x = tf.constant([
4    [1, 2, 3],
5    [4, 5, 6],
6])
7
8result = tf.reduce_sum(x, axis=-1)
9print(result.numpy())

Output:

python
[ 6 15]

The last dimension of x is each row's columns. So TensorFlow sums each row across its columns:

  • '1 + 2 + 3 = 6'
  • '4 + 5 + 6 = 15'

That is why the result shape becomes (2,).

Why -1 Means the Last Axis

TensorFlow supports negative axis values so you can count backward.

For a tensor with rank n:

  • 'axis=0 means the first dimension'
  • 'axis=1 means the second dimension'
  • 'axis=-1 means the last dimension'
  • 'axis=-2 means the second-to-last dimension'

This is especially useful in code where the leading dimensions may vary but the final dimension always represents features, channels, or per-item values.

A Three-Dimensional Example

Consider a tensor with shape (2, 2, 3):

python
1import tensorflow as tf
2
3x = tf.constant([
4    [[1, 2, 3], [4, 5, 6]],
5    [[7, 8, 9], [10, 11, 12]],
6])
7
8result = tf.reduce_sum(x, axis=-1)
9print(result.numpy())
10print(result.shape)

Output:

python
[[ 6 15]
 [24 33]]
(2, 2)

Again, the last axis contains groups of three values, so each inner triplet is summed.

Comparing axis=-1 to Other Axes

Using a different axis changes which dimension gets collapsed.

python
1import tensorflow as tf
2
3x = tf.constant([
4    [1, 2, 3],
5    [4, 5, 6],
6])
7
8print(tf.reduce_sum(x, axis=0).numpy())
9print(tf.reduce_sum(x, axis=1).numpy())
10print(tf.reduce_sum(x, axis=-1).numpy())

Output:

python
[5 7 9]
[ 6 15]
[ 6 15]

Here axis=1 and axis=-1 are the same because the tensor is two-dimensional and the last axis is also axis 1.

Using keepdims=True

By default, the reduced dimension disappears. If you want to keep it as size 1, use keepdims=True.

python
1import tensorflow as tf
2
3x = tf.constant([
4    [1, 2, 3],
5    [4, 5, 6],
6])
7
8result = tf.reduce_sum(x, axis=-1, keepdims=True)
9print(result.numpy())
10print(result.shape)

Output:

python
[[ 6]
 [15]]
(2, 1)

This is helpful when later tensor operations expect the original rank to stay intact.

Why axis=-1 Is So Common in ML Code

Machine learning tensors often store features in the final dimension. For example:

  • logits per class
  • embedding coordinates
  • color channels after a reshape
  • sequence features at each time step

Using axis=-1 means the code says "sum over the features" without hard-coding the earlier dimensions such as batch size.

That makes the code more reusable across different batch shapes.

Common Pitfalls

A common mistake is assuming axis=-1 means "sum everything." It does not. It only sums the last dimension.

Another issue is losing track of the output shape after reduction. Summing over one axis removes that axis unless keepdims=True is set.

Developers also sometimes confuse axis=-1 with axis=0. They do opposite things in many common tensor layouts.

Finally, always verify which dimension actually stores the feature values in your tensor. axis=-1 is convenient only when the last dimension is the one you intend to reduce.

Summary

  • 'axis=-1 in tf.reduce_sum means sum across the last dimension.'
  • Negative axes count from the end of the tensor shape.
  • The reduced axis disappears unless keepdims=True is used.
  • 'axis=-1 is common when the final dimension stores features or channels.'
  • Check tensor shapes carefully so you reduce the intended dimension.

Course illustration
Course illustration

All Rights Reserved.