Is there a function to extract image patches in PyTorch?
ML System Design practice on Codemia
Design recommenders, ranking systems and training pipelines the way ML interviews actually ask for them, with worked solutions.
Introduction
Yes. In PyTorch, the usual way to extract image patches is to use Tensor.unfold or torch.nn.Unfold. Both expose sliding local windows over image tensors, which makes them useful for patch-based models, local feature extraction, and vision-transformer style preprocessing.
Using unfold on the Tensor
For a simple tensor-level solution, unfold works directly on the height and width dimensions.
If x has shape (batch, channels, height, width), then:
- the first
unfoldslices along height - the second
unfoldslices along width
For this example, the result shape is:
(1, 1, 2, 2, 2, 2)
That means:
- 1 batch
- 1 channel
- 2 patch positions vertically
- 2 patch positions horizontally
- each patch is
2 x 2
Making the Patches Easier to Use
The raw unfold output is often correct but awkward. A reshape makes the patches more convenient:
Now each patch is an independent tensor of shape (1, 2, 2), and the total batch of patches is easier to feed into later code.
Using torch.nn.Unfold
For many model pipelines, torch.nn.Unfold is even cleaner because it behaves like an image-to-patch operator.
This returns shape (batch, channels * kernel_height * kernel_width, number_of_patches).
For the example above, that becomes:
- batch = 1
- flattened patch size = 4
- number of patches = 4
That format is especially useful for:
- feeding local windows into linear layers
- tokenizing images for transformer-style models
- preparing patches for custom operations
Overlapping Patches
If you want overlap, use a stride smaller than the patch size.
With kernel_size=3 and stride=1, every patch overlaps heavily with its neighbors. That is common in classical image-processing pipelines and some dense prediction tasks.
Padding for Border Coverage
If the image size is not divisible by the patch geometry, you may want padding before extraction:
Padding is often necessary when you need a complete grid of equally sized patches.
Common Pitfalls
The most common mistake is forgetting the tensor layout. PyTorch image tensors usually follow (N, C, H, W), so unfolding the wrong dimensions produces confusing shapes.
Another issue is assuming unfold returns ready-to-use image batches. It returns a structured view or flattened patch representation, so reshaping is often part of the workflow.
A third pitfall is ignoring stride. A stride equal to patch size gives non-overlapping patches, while smaller strides produce overlap. The difference changes both the patch count and the computational cost.
Finally, if you want exact border handling, think about padding explicitly. Otherwise the final rows or columns may be dropped when the dimensions do not fit the patch geometry cleanly.
Summary
- PyTorch can extract image patches with
Tensor.unfoldortorch.nn.Unfold. - '
unfoldis flexible and works directly on tensor dimensions.' - '
torch.nn.Unfoldis often more convenient for model pipelines.' - Stride controls whether patches overlap.
- Padding may be needed when the image size does not divide evenly into patches.
Related reading
- Is there a tensorflow equivalent to np.empty?
- Is there a way of determining how much GPU memory is in use by TensorFlow?
- Is there a way to check if mxnet uses my gpu?
- Is there a way to determine where messages came from in a Kafka topic?
- k-fold cross validation using DataLoaders in PyTorch
- KL Divergence for two probability distributions in PyTorch
- Is there a way to get the color of a recognized object inside a picture?
- Is there an algorithm to determine contiguous colored regions in a grid?
.png&w=3840&q=75)
Tackling System Design Interview Problems
A short course that equips you with the skills to approach system design interviews methodically.
Start the free courseTrack what you have practised
A free account saves your progress, solutions and study plan across every problem on Codemia.
ML System Design practice on Codemia
Design recommenders, ranking systems and training pipelines the way ML interviews actually ask for them, with worked solutions.