Introduction

In image segmentation, a void label marks pixels that should not contribute to training or evaluation. These are often uncertain, unlabeled, or intentionally ignored regions, commonly encoded as a value such as 255.

The correct handling strategy is usually not to remap void pixels into a real class. Instead, keep them as ignore labels and mask them out of the loss and metrics. That preserves the training signal from valid pixels without teaching the model that “void” is a meaningful semantic class.

Why Void Labels Need Special Handling

If you treat void pixels as an ordinary class, the model learns from noisy or undefined targets. That can distort both training and evaluation.

The standard segmentation rule is:

• valid labels contribute to loss
• void labels do not

This means your pipeline must build a boolean mask identifying non-void pixels.

A Simple Masked Loss Pattern

Assume your labels use 255 for void and your model outputs per-pixel logits.

1import tensorflow as tf
2
3VOID_LABEL = 255
4
5
6def masked_sparse_ce(y_true, y_pred):
7    y_true = tf.cast(y_true, tf.int32)
8    valid_mask = tf.not_equal(y_true, VOID_LABEL)
9
10    safe_labels = tf.where(valid_mask, y_true, tf.zeros_like(y_true))
11    per_pixel = tf.keras.losses.sparse_categorical_crossentropy(
12        safe_labels,
13        y_pred,
14        from_logits=True,
15    )
16
17    valid_mask = tf.cast(valid_mask, per_pixel.dtype)
18    per_pixel = per_pixel * valid_mask
19
20    return tf.reduce_sum(per_pixel) / tf.reduce_sum(valid_mask)

This computes cross-entropy only over valid pixels.

Why the Label Is Replaced Before Loss Computation

Most sparse losses still expect label values to be inside the valid class range. If 255 is outside the model’s class count, you must replace it with a safe placeholder before calling the loss.

The important detail is that the replacement value does not matter as long as the corresponding pixel is multiplied by zero in the mask afterward.

That is why tf.where is used before the loss and the mask is applied after the loss.

Metrics Need the Same Treatment

If your loss ignores void pixels but your accuracy metric does not, the training loop still reports misleading numbers.

A masked accuracy example:

1def masked_accuracy(y_true, y_pred):
2    y_true = tf.cast(y_true, tf.int32)
3    predictions = tf.argmax(y_pred, axis=-1, output_type=tf.int32)
4    valid_mask = tf.not_equal(y_true, VOID_LABEL)
5
6    matches = tf.equal(y_true, predictions)
7    matches = tf.logical_and(matches, valid_mask)
8
9    valid_mask = tf.cast(valid_mask, tf.float32)
10    matches = tf.cast(matches, tf.float32)
11
12    return tf.reduce_sum(matches) / tf.reduce_sum(valid_mask)

Loss and metrics should agree about which pixels count.

Data Pipeline Considerations

In a tf.data pipeline, keep the void label unchanged until the training step unless you have a deliberate preprocessing reason to remap it.

A typical pipeline might:

• load image and mask
• resize with the correct interpolation rules
• preserve the integer label map
• pass the mask unchanged into the loss

Be careful with resizing segmentation masks. Use nearest-neighbor interpolation so the void label and class ids do not get mixed into fractional nonsense.

Should Void Be an Extra Output Class?

Usually no. Only make void a real class if the model is intentionally supposed to predict “unknown” or “ignore” as a semantic output category.

That is a different modeling decision from simply ignoring unlabeled pixels.

For ordinary segmentation datasets with ignore regions, masking is the better default.

Common Pitfalls

A common mistake is treating the void label as an ordinary class id. That injects noise into training and makes metrics misleading.

Another mistake is masking the loss but not masking the evaluation metrics, which produces inconsistent training feedback.

Developers also forget to replace the out-of-range void label before calling sparse cross-entropy, which can trigger invalid-label errors.

Finally, resizing segmentation masks with bilinear interpolation can corrupt class ids and void regions. Use nearest-neighbor for masks.

Summary

• Void labels should usually be ignored, not learned as a normal class.
• Build a valid-pixel mask and apply it to both loss and metrics.
• Replace out-of-range void values with a safe label before calling sparse loss functions.
• Preserve label integrity through the input pipeline, especially during resizing.
• The key rule is consistency: if a pixel is ignored in training, it should be ignored in evaluation too.