Introduction

torch.stack and torch.cat both combine tensors, but they solve different shape problems. cat joins tensors along an existing dimension, while stack creates a new dimension and therefore increases the tensor rank by one.

This difference sounds small, but it changes the meaning of the result. Many model-shape bugs in PyTorch come from using stack when the code really needed cat, or vice versa.

cat Joins Along an Existing Dimension

Use torch.cat when the tensors already have the right rank and you want to extend one of their existing axes:

1import torch
2
3a = torch.tensor([[1, 2], [3, 4]])
4b = torch.tensor([[5, 6], [7, 8]])
5
6cat0 = torch.cat([a, b], dim=0)  # shape [4, 2]
7cat1 = torch.cat([a, b], dim=1)  # shape [2, 4]
8
9print(cat0.shape)
10print(cat1.shape)

Here the tensors stay rank-2. Only the chosen dimension grows.

stack Creates a New Dimension

Use torch.stack when you want to bundle same-shaped tensors into a new axis:

1import torch
2
3a = torch.tensor([[1, 2], [3, 4]])
4b = torch.tensor([[5, 6], [7, 8]])
5
6stack0 = torch.stack([a, b], dim=0)  # shape [2, 2, 2]
7stack1 = torch.stack([a, b], dim=1)  # shape [2, 2, 2]
8
9print(stack0.shape)
10print(stack1.shape)

The rank increases from 2 to 3 because a new dimension is inserted.

One useful mental model is that stack is like applying unsqueeze to each tensor first and then concatenating.

Know the Input Rules

The compatibility rules differ:

• 'cat requires all shapes to match except on the concatenation dimension'
• 'stack requires all tensors to have exactly the same shape'

1x = torch.randn(2, 3)
2y = torch.randn(4, 3)
3
4print(torch.cat([x, y], dim=0).shape)
5
6try:
7    torch.stack([x, y], dim=0)
8except RuntimeError as exc:
9    print("stack failed:", exc)

This explains why stack is often used to build batches from equal-sized samples, while cat is more common for feature fusion.

Common Deep Learning Use Cases

Batch assembly from equal-size samples:

samples = [torch.randn(3, 224, 224) for _ in range(8)]
batch = torch.stack(samples, dim=0)  # [8, 3, 224, 224]
print(batch.shape)

Channel-wise feature fusion:

1f1 = torch.randn(16, 32, 64, 64)
2f2 = torch.randn(16, 16, 64, 64)
3merged = torch.cat([f1, f2], dim=1)  # [16, 48, 64, 64]
4print(merged.shape)

If you used stack in that second case, you would introduce an unwanted extra dimension instead of expanding the channel axis.

Think About Performance and Memory

Both operations allocate a new tensor. That means repeatedly calling cat or stack inside a loop is usually inefficient:

rows = [torch.randn(1, 10) for _ in range(1000)]
out = torch.cat(rows, dim=0)

This is much better than growing the result incrementally with one concatenation per iteration. The same advice applies to stack: collect first, merge once.

Also remember that merged tensors must agree on device and dtype. Shape compatibility alone is not enough.

Common Pitfalls

The biggest mistake is choosing based on what "seems to work" rather than on the shape contract of the next layer. Always check the output shape explicitly.

Another common issue is using stack to fuse features when the correct operation was concatenation along an existing channel or time dimension.

People also build tensors incrementally in loops and pay repeated allocation costs. Both cat and stack are much happier when called once on a collected list.

Finally, debug merge problems by inspecting shape, dtype, and device together. A mismatch in any one of those can cause a runtime failure.

Summary

• 'torch.cat joins tensors along an existing dimension.'
• 'torch.stack creates a new dimension and increases rank.'
• Use stack for batch assembly from equal-shaped samples.
• Use cat for extending channels, time steps, or other existing axes.
• Collect tensors first and merge once to avoid unnecessary allocation overhead.