Introduction

Patch extraction is useful for denoising, style transfer, local attention, and custom convolution pipelines, but extracting patches is only half the problem. Reconstructing the original image is harder because overlapping patches write back into the same pixels. In TensorFlow, there is no simple built-in inverse for extract_image_patches, so reconstruction is usually implemented as a fold-and-average operation.

Understand What the Patch Tensor Contains

Older TensorFlow code often uses extract_image_patches, while newer code uses tf.image.extract_patches. Both produce a tensor where each spatial location stores a flattened patch.

For a single grayscale image, a 3 x 3 patch with stride 1 taken from a 5 x 5 image produces a patch grid of shape 1 x 3 x 3 x 9. That means:

• there are 3 x 3 patch positions
• each patch contains 3 x 3 = 9 values
• overlapping regions appear in many different patches

Because of that overlap, reconstruction is not just reshaping. You must place each patch back into the output image and keep track of how many times each pixel has been written.

A Simple Reconstruction Strategy

The standard approach is:

1. create an output image initialized to zeros
2. create a second array of counts initialized to zeros
3. add each patch into the correct output region
4. add 1 into the count array for the same region
5. divide the summed image by the counts

That final division averages overlapping contributions and restores the original values when the extracted patches came from the original image without modification.

Runnable NumPy Example

The following example extracts overlapping 3 x 3 patches from a small image and reconstructs it by averaging overlaps.

1import numpy as np
2
3
4def extract_patches(image, patch_size, stride):
5    h, w = image.shape
6    ph, pw = patch_size
7    patches = []
8
9    for i in range(0, h - ph + 1, stride):
10        for j in range(0, w - pw + 1, stride):
11            patches.append(image[i:i + ph, j:j + pw])
12
13    return np.array(patches)
14
15
16def reconstruct_from_patches(patches, image_shape, patch_size, stride):
17    h, w = image_shape
18    ph, pw = patch_size
19    image = np.zeros((h, w), dtype=np.float32)
20    counts = np.zeros((h, w), dtype=np.float32)
21
22    index = 0
23    for i in range(0, h - ph + 1, stride):
24        for j in range(0, w - pw + 1, stride):
25            image[i:i + ph, j:j + pw] += patches[index]
26            counts[i:i + ph, j:j + pw] += 1.0
27            index += 1
28
29    return image / counts
30
31
32original = np.arange(1, 26, dtype=np.float32).reshape(5, 5)
33patches = extract_patches(original, patch_size=(3, 3), stride=1)
34reconstructed = reconstruct_from_patches(patches, original.shape, (3, 3), 1)
35
36print(original)
37print(reconstructed)
38print(np.allclose(original, reconstructed))

This prints True at the end because the reconstruction logic correctly averages the overlaps.

Relating It to TensorFlow

If your patches come from tf.image.extract_patches, the reconstruction idea is the same. You usually reshape the last dimension back into (patch_height, patch_width, channels), then scatter or add each patch into the output tensor.

1import tensorflow as tf
2
3image = tf.reshape(tf.range(1, 26, dtype=tf.float32), (1, 5, 5, 1))
4patches = tf.image.extract_patches(
5    images=image,
6    sizes=[1, 3, 3, 1],
7    strides=[1, 1, 1, 1],
8    rates=[1, 1, 1, 1],
9    padding="VALID",
10)
11
12print(patches.shape)

For this example, the shape is (1, 3, 3, 9). Before reconstruction, each flattened patch must be reshaped back into 3 x 3 x 1.

In production code, you often convert the patch tensor to NumPy for inspection first, because the most common bugs are indexing mistakes rather than TensorFlow math errors.

Handle Edges and Padding Deliberately

Reconstruction is simplest when extraction used padding="VALID" and the stride divides the valid patch grid cleanly. If you extracted with padding="SAME", edge patches may include padded values, so your inverse logic needs to mirror that choice.

Color images add one more dimension, but the idea is unchanged: accumulate patches into the output image and divide by per-pixel counts for each channel.

Common Pitfalls

• Trying to reconstruct by reshaping the patch tensor without folding overlaps back into the image.
• Forgetting to divide by the overlap count, which makes central pixels too bright.
• Ignoring the channel dimension when reshaping TensorFlow patches.
• Mixing extraction settings such as VALID and SAME between forward and inverse logic.
• Assuming extract_image_patches has a direct built-in inverse in TensorFlow.

Summary

• Reconstructing from image patches is a fold-and-average problem, not a simple reshape.
• Overlapping patches must be accumulated into the output image.
• A second count array is needed so you can average each pixel correctly.
• 'tf.image.extract_patches flattens each patch, so reshape it before writing it back.'
• Match your reconstruction logic to the original stride, patch size, and padding settings.