Introduction

In exact arithmetic, addition is associative and commutative, so reordering a sum does not change the answer. In floating-point arithmetic, the hardware is working with rounded approximations, so intermediate rounding can make different summation orders produce different results.

Why Exact Math and Computer Math Diverge

Real-number arithmetic assumes infinite precision. Computers do not have that luxury. A double stores a finite number of bits, so most decimal fractions are approximated.

When you add two floating-point values, the processor aligns their exponents and rounds the result back into the available precision. If one number is much larger than the other, the smaller number can partly or completely disappear during that rounding step.

That means:

• the mathematical rule still exists
• the machine approximation no longer behaves exactly like that rule

So the question is not “why did algebra stop working” but “where did the rounding happen.”

A Small Example

Python makes the issue easy to see:

1values = [1e16, 1.0, -1e16]
2
3left_to_right = ((values[0] + values[1]) + values[2])
4right_grouped = (values[0] + (values[1] + values[2]))
5
6print(left_to_right)
7print(right_grouped)

One grouping prints 0.0, while the other prints 0.0 or 1.0 depending on the exact evaluation path and platform behavior. The core point is that the tiny 1.0 can be lost when combined with 1e16 too early.

A clearer demonstration is to compare naive summation with a more careful routine:

1import math
2
3values = [1e16, 1.0, 1.0, 1.0, -1e16]
4
5print(sum(values))
6print(sum(reversed(values)))
7print(math.fsum(values))

sum depends on order because it adds sequentially. math.fsum uses a more stable algorithm and is much closer to the mathematically expected result.

What Causes the Difference

Two effects matter most.

First, rounding error accumulates. Each intermediate result is rounded, so a long chain of additions is really a long chain of approximate additions.

Second, cancellation makes things worse. If two nearly equal large values with opposite signs are added, the remaining digits may contain mostly rounding noise. After that, adding small values can produce noticeably different answers depending on when the cancellation occurred.

This is why summing from smallest magnitude to largest often behaves better than summing in arbitrary order. Small values get combined with each other before they are swamped by very large ones.

Better Ways to Sum Floating-Point Data

The simplest improvement is to use a numerically stable summation routine when your language provides one. In Python, that is math.fsum. In numeric libraries, pairwise or tree-based summation is common because it reduces error growth compared with a strict left-to-right loop.

If you need decimal-style exactness for financial quantities, a binary floating-point type may be the wrong tool altogether. Python's Decimal type is slower but can represent decimal arithmetic more predictably.

1from decimal import Decimal
2
3values = [Decimal("10000000000000000"), Decimal("1.0"), Decimal("-10000000000000000")]
4print(sum(values))

That example is not a universal replacement for floating point, but it shows that the data type matters as much as the loop order.

Common Pitfalls

A common mistake is treating floating-point differences as proof of a compiler bug. Most of the time the compiler and CPU are behaving as specified.

Another mistake is comparing floating-point results with exact equality after a long computation. Use tolerances such as abs(a - b) < epsilon when appropriate.

People also assume parallel reduction is harmless. It is faster, but it changes the grouping of additions, so the final low-order bits can differ from a single-threaded sum.

Summary

• In exact math, sum order does not matter; in floating-point arithmetic, rounding makes order matter.
• Large and small magnitudes mixed together are especially sensitive to summation order.
• Cancellation can amplify small rounding differences into visible result changes.
• Stable routines such as math.fsum reduce the problem significantly.
• If exact decimal behavior matters, choose a more appropriate numeric type instead of relying on naive floating-point addition.