Introduction

tf.matmul looks simple, but it does not follow the same intuition as elementwise TensorFlow ops such as + or *. Matrix multiplication has strict dimension rules, and any extra leading dimensions are treated as batch dimensions rather than as free-form broadcasting targets.

What tf.matmul Actually Multiplies

For ordinary rank-2 tensors, tf.matmul(a, b) multiplies a matrix of shape (m, n) by a matrix of shape (n, p) and returns shape (m, p).

1import tensorflow as tf
2
3a = tf.constant([[1.0, 2.0], [3.0, 4.0]])
4b = tf.constant([[5.0], [6.0]])
5
6result = tf.matmul(a, b)
7print(result)

The important rule is that the inner dimensions must match. If they do not, TensorFlow raises an error instead of trying to guess your intent.

Batch Dimensions Are Not Elementwise Broadcasting

When tensors have rank greater than 2, the last two dimensions still represent matrices. The earlier dimensions represent batches of matrices.

1import tensorflow as tf
2
3a = tf.ones((2, 3, 4))
4b = tf.ones((2, 4, 5))
5c = tf.matmul(a, b)
6
7print(c.shape)  # (2, 3, 5)

Here TensorFlow performs two matrix multiplications, one for each batch item. This is often called batch matrix multiplication, but it is not the same mental model as NumPy-style elementwise broadcasting over arbitrary axes. The batch dimensions are expected to line up with the multiplication you want to perform.

Why People Expect Broadcasting

Elementwise ops make broadcasting feel universal:

1import tensorflow as tf
2
3x = tf.ones((3, 4))
4y = tf.ones((1, 4))
5print((x + y).shape)  # (3, 4)

That works because addition is elementwise. tf.matmul is different because the last two axes have a structural meaning: rows and columns of matrices. TensorFlow cannot reinterpret mismatched matrix axes as a harmless broadcast without changing the math.

How to Fix a Shape Mismatch

If one input should conceptually be reused across several batch items, reshape or broadcast it explicitly before calling tf.matmul. Making that step explicit keeps the code honest and easier to debug.

1import tensorflow as tf
2
3left = tf.ones((1, 3, 4))
4right = tf.ones((2, 4, 5))
5
6left_expanded = tf.broadcast_to(left, (2, 3, 4))
7result = tf.matmul(left_expanded, right)
8print(result.shape)

This says exactly what you mean: reuse the same left matrix for two batch entries, then perform batch matrix multiplication.

Alternatives for Different Goals

Sometimes tf.matmul is the wrong tool:

• use * for elementwise multiplication
• use tf.einsum when you need a custom tensor contraction
• use tf.tensordot when you want to sum over named axes rather than matrix-only semantics

Choosing the correct operator matters because TensorFlow shape rules are part of the API contract, not just a runtime inconvenience.

Debugging Shape Problems

When a tf.matmul call fails, inspect both tensor shapes before changing code:

print(left.shape)
print(right.shape)

Then ask two separate questions:

• do the inner matrix dimensions match
• do the batch dimensions describe the same number of matrix multiplications

That split is usually enough to identify whether you need a transpose, a reshape, or an explicit broadcast.

Common Pitfalls

• Expecting tf.matmul to behave like an elementwise broadcasted multiply.
• Forgetting that the last two axes are matrix dimensions, not generic dimensions.
• Trying to fix a mismatch by reshaping blindly instead of checking the math.
• Using tf.matmul when * or tf.einsum better matches the intended operation.
• Reading a batch-dimension error as if it were an ordinary broadcasting error.

Summary

• 'tf.matmul is matrix multiplication, so its shape rules are stricter than elementwise ops.'
• The last two dimensions define matrices; leading dimensions represent batches.
• Batch multiplication is not the same as arbitrary elementwise broadcasting.
• If you need reuse across batches, broadcast or reshape explicitly before multiplication.
• Check shapes first, then decide whether you need tf.matmul, elementwise multiplication, or a different tensor operator.