Introduction

In TensorFlow ResNet implementations, fixed padding usually means explicitly padding the input tensor yourself and then running a VALID convolution. The reason is not that padding="same" is always wrong. The reason is that explicit padding gives predictable behavior for strided convolutions and keeps the spatial math consistent with the reference ResNet design.

What Fixed Padding Means

A 3 x 3 convolution needs one pixel of padding on each side if you want to preserve the center alignment of the kernel. With stride 1, padding="same" already does the obvious thing. The subtle case is stride greater than 1.

In TensorFlow, padding="same" with stride 2 computes output shapes by framework rules and may place asymmetric padding depending on the input size. Reference ResNet code often avoids that ambiguity by padding explicitly first and then using padding="valid".

1import tensorflow as tf
2
3
4def fixed_padding(inputs, kernel_size):
5    pad_total = kernel_size - 1
6    pad_beg = pad_total // 2
7    pad_end = pad_total - pad_beg
8    return tf.pad(inputs, [[0, 0], [pad_beg, pad_end], [pad_beg, pad_end], [0, 0]])
9
10
11x = tf.random.normal((1, 32, 32, 3))
12x = fixed_padding(x, 3)
13conv = tf.keras.layers.Conv2D(16, 3, strides=2, padding="valid")
14y = conv(x)
15print(y.shape)

This makes the padding amount explicit and independent of TensorFlow's internal same padding logic.

Why ResNet Implementations Often Prefer It

The main reason is reproducibility of shape behavior. In a deep architecture with many residual blocks, tiny differences in downsampling rules can cause skip-path mismatches or inconsistent output shapes across ports.

Fixed padding helps because it gives you these properties:

• the padding amount is visible in code
• the effective receptive field alignment is predictable
• strided convolutions behave the same way for a given kernel size
• ports from reference implementations are easier to compare

That is especially useful when reproducing a paper or matching an existing pretrained checkpoint.

The Real Difference Shows Up with Stride Greater Than 1

For a plain stride-1 convolution, same padding is usually fine and much simpler.

1import tensorflow as tf
2
3x = tf.random.normal((1, 32, 32, 3))
4layer = tf.keras.layers.Conv2D(16, 3, strides=1, padding="same")
5print(layer(x).shape)

The reason older ResNet code goes out of its way to define a fixed_padding helper is the downsampling path. Once you introduce stride 2, explicit padding plus VALID makes the boundary behavior easier to reason about.

In residual networks, that matters because the main branch and shortcut branch need compatible spatial dimensions. If your downsampling math is opaque, debugging shape mismatches becomes harder.

Why Not Always Use same

Using padding="same" is not a bug. Many Keras models use it successfully. The tradeoff is that you are delegating padding details to the framework.

That is perfectly acceptable when:

• you are building a custom model from scratch
• exact compatibility with a reference implementation is not required
• you value simpler code over strict control of boundary behavior

But if you are copying a ResNet family implementation from a paper or an official model definition, fixed padding is often part of that recipe.

A Minimal Residual Block Example

1import tensorflow as tf
2
3
4def fixed_padding(inputs, kernel_size):
5    pad_total = kernel_size - 1
6    pad_beg = pad_total // 2
7    pad_end = pad_total - pad_beg
8    return tf.pad(inputs, [[0, 0], [pad_beg, pad_end], [pad_beg, pad_end], [0, 0]])
9
10
11def conv3x3(inputs, filters, stride):
12    if stride > 1:
13        inputs = fixed_padding(inputs, 3)
14        padding = "valid"
15    else:
16        padding = "same"
17    return tf.keras.layers.Conv2D(filters, 3, strides=stride, padding=padding, use_bias=False)(inputs)
18
19
20x = tf.random.normal((1, 32, 32, 16))
21y = conv3x3(x, 16, stride=2)
22print(y.shape)

This pattern is common because it keeps the stride-1 case simple while making the downsampling case explicit.

Common Pitfalls

The biggest mistake is thinking fixed padding improves accuracy by itself. It is mostly about architecture consistency and predictable shape behavior, not a magic accuracy switch.

Another mistake is mixing fixed_padding and padding="same" arbitrarily across residual blocks. If the model family expects one convention, stay consistent.

A third problem is forgetting that the shortcut path must downsample in a compatible way. Many residual shape bugs blamed on padding are really mismatches between the main path and projection path.

Summary

• Fixed padding means explicitly padding first and then using VALID convolution
• It is most useful for strided convolutions in ResNet-style downsampling blocks
• The main benefit is predictable, reference-friendly shape behavior
• 'padding="same" is still fine for many models, especially stride-1 layers'
• Use one consistent padding convention across the residual block design