Introduction

Interpolated sampling is what you need when you want pixel values at non-integer coordinates instead of exact grid locations. This comes up in image warping, optical flow, spatial transformers, and any pipeline where points land between pixels rather than directly on them.

Why Interpolation Is Needed

If you sample only integer coordinates, you can use tf.gather_nd directly. The problem appears when your point is something like row 10.3, column 42.7. Real image transformations produce those coordinates all the time.

Nearest-neighbor sampling would round to one pixel, but bilinear interpolation is smoother because it blends the four surrounding pixels according to distance.

A Simple Bilinear Sampler in TensorFlow

The following example samples arbitrary y, x points from a single image tensor of shape [height, width, channels].

1import tensorflow as tf
2
3
4def bilinear_sample(image: tf.Tensor, points: tf.Tensor) -> tf.Tensor:
5    image = tf.convert_to_tensor(image, dtype=tf.float32)
6    points = tf.convert_to_tensor(points, dtype=tf.float32)
7
8    height = tf.shape(image)[0]
9    width = tf.shape(image)[1]
10
11    y = tf.clip_by_value(points[:, 0], 0.0, tf.cast(height - 1, tf.float32))
12    x = tf.clip_by_value(points[:, 1], 0.0, tf.cast(width - 1, tf.float32))
13
14    y0 = tf.cast(tf.floor(y), tf.int32)
15    x0 = tf.cast(tf.floor(x), tf.int32)
16    y1 = tf.minimum(y0 + 1, height - 1)
17    x1 = tf.minimum(x0 + 1, width - 1)
18
19    y0f = tf.cast(y0, tf.float32)
20    x0f = tf.cast(x0, tf.float32)
21    dy = y - y0f
22    dx = x - x0f
23
24    top_left = tf.gather_nd(image, tf.stack([y0, x0], axis=1))
25    top_right = tf.gather_nd(image, tf.stack([y0, x1], axis=1))
26    bottom_left = tf.gather_nd(image, tf.stack([y1, x0], axis=1))
27    bottom_right = tf.gather_nd(image, tf.stack([y1, x1], axis=1))
28
29    wa = (1.0 - dy) * (1.0 - dx)
30    wb = (1.0 - dy) * dx
31    wc = dy * (1.0 - dx)
32    wd = dy * dx
33
34    return (
35        top_left * wa[:, None]
36        + top_right * wb[:, None]
37        + bottom_left * wc[:, None]
38        + bottom_right * wd[:, None]
39    )
40
41
42image = tf.reshape(tf.range(16, dtype=tf.float32), (4, 4, 1))
43points = tf.constant([
44    [0.5, 0.5],
45    [1.25, 2.0],
46    [3.0, 1.0],
47], dtype=tf.float32)
48
49samples = bilinear_sample(image, points)
50print(samples.numpy().squeeze())

This works by:

• finding the four surrounding pixels
• computing fractional offsets within that cell
• weighting each corner according to distance

The output has shape [num_points, channels].

When tf.image.resize Is Enough

If your sampling points form a regular grid, you often do not need a custom sampler at all. tf.image.resize already performs interpolation efficiently.

1import tensorflow as tf
2
3image = tf.random.uniform((1, 128, 128, 3))
4resized = tf.image.resize(image, size=(256, 256), method="bilinear")
5print(resized.shape)

Use tf.image.resize for full-image resizing. Use a point sampler when the coordinates are irregular or derived from another model.

Batch Dimensions and Coordinate Conventions

One subtle part of image sampling is coordinate order. Many computer vision libraries use x, y, while tensor indexing is usually row, column, which corresponds to y, x. The example above expects y, x.

Another subtlety is batching. Real model pipelines often work with images shaped [batch, height, width, channels] and points shaped [batch, num_points, 2]. The same bilinear logic still applies, but you need to include the batch index when gathering pixels.

If you sample points from model outputs, document the coordinate convention early. Silent x, y versus y, x mix-ups are extremely common.

Clipping and Boundary Behavior

The example clips coordinates into the valid image range. That is a reasonable default, but it is not the only possible behavior. Depending on the application, you might prefer:

• zero padding outside the image
• reflection at boundaries
• wrapping for tiled textures

Clipping is simple and keeps the example runnable, but boundary handling should match the semantics of your model or image transform.

Common Pitfalls

• Using tf.gather_nd directly on non-integer coordinates. Tensor indexing requires integer locations.
• Mixing up x, y and y, x ordering.
• Forgetting batch dimensions when moving from a toy example to real model inputs.
• Ignoring boundary conditions and getting unexpected edge values.
• Using a custom point sampler when tf.image.resize would already solve a regular-grid resize problem.

Summary

• Interpolated sampling is needed for non-integer image coordinates.
• Bilinear interpolation blends the four surrounding pixels smoothly.
• A custom TensorFlow sampler is useful for irregular point queries.
• Be explicit about coordinate order and boundary handling.
• Use tf.image.resize when the problem is full-image resampling on a regular grid.