What does tf.reduce_sum do with axis -1?
ML System Design practice on Codemia
Design recommenders, ranking systems and training pipelines the way ML interviews actually ask for them, with worked solutions.
Introduction
In TensorFlow, axis=-1 means "use the last dimension." So tf.reduce_sum(..., axis=-1) sums across the last axis of the tensor, no matter how many dimensions the tensor has. That is why negative axes are handy: they let you refer to dimensions relative to the end instead of counting from the front.
A Simple Two-Dimensional Example
Start with a matrix:
Output:
The last dimension of x is each row's columns. So TensorFlow sums each row across its columns:
- '
1 + 2 + 3 = 6' - '
4 + 5 + 6 = 15'
That is why the result shape becomes (2,).
Why -1 Means the Last Axis
TensorFlow supports negative axis values so you can count backward.
For a tensor with rank n:
- '
axis=0means the first dimension' - '
axis=1means the second dimension' - '
axis=-1means the last dimension' - '
axis=-2means the second-to-last dimension'
This is especially useful in code where the leading dimensions may vary but the final dimension always represents features, channels, or per-item values.
A Three-Dimensional Example
Consider a tensor with shape (2, 2, 3):
Output:
Again, the last axis contains groups of three values, so each inner triplet is summed.
Comparing axis=-1 to Other Axes
Using a different axis changes which dimension gets collapsed.
Output:
Here axis=1 and axis=-1 are the same because the tensor is two-dimensional and the last axis is also axis 1.
Using keepdims=True
By default, the reduced dimension disappears. If you want to keep it as size 1, use keepdims=True.
Output:
This is helpful when later tensor operations expect the original rank to stay intact.
Why axis=-1 Is So Common in ML Code
Machine learning tensors often store features in the final dimension. For example:
- logits per class
- embedding coordinates
- color channels after a reshape
- sequence features at each time step
Using axis=-1 means the code says "sum over the features" without hard-coding the earlier dimensions such as batch size.
That makes the code more reusable across different batch shapes.
Common Pitfalls
A common mistake is assuming axis=-1 means "sum everything." It does not. It only sums the last dimension.
Another issue is losing track of the output shape after reduction. Summing over one axis removes that axis unless keepdims=True is set.
Developers also sometimes confuse axis=-1 with axis=0. They do opposite things in many common tensor layouts.
Finally, always verify which dimension actually stores the feature values in your tensor. axis=-1 is convenient only when the last dimension is the one you intend to reduce.
Summary
- '
axis=-1intf.reduce_summeans sum across the last dimension.' - Negative axes count from the end of the tensor shape.
- The reduced axis disappears unless
keepdims=Trueis used. - '
axis=-1is common when the final dimension stores features or channels.' - Check tensor shapes carefully so you reduce the intended dimension.
Related reading
- What does the error Loaded runtime CuDNN library 5005 but source was compiled with 5103 mean?
- What does the filter parameter mean in Conv2d layer?
- what does the tf.nn.lrn method do?
- What does the use_multiprocessing input argument in keras mode.fit do?
- What does tf.strided_slice do?
- What does tf.train.get_global_step do in TensorFlow?
- What does trainingTrue mean when calling a TensorFlow Keras model?
- What does unsqueeze do in Pytorch?
.png&w=3840&q=75)
Tackling System Design Interview Problems
A short course that equips you with the skills to approach system design interviews methodically.
Start the free courseTrack what you have practised
A free account saves your progress, solutions and study plan across every problem on Codemia.
ML System Design practice on Codemia
Design recommenders, ranking systems and training pipelines the way ML interviews actually ask for them, with worked solutions.