Introduction

A Fully Convolutional Network, or FCN, is a neural network for dense prediction tasks such as semantic segmentation. Unlike a classification CNN, an FCN does not end with fully connected layers that collapse spatial structure into one label. Instead, it preserves spatial reasoning and produces a per-pixel output map. In TensorFlow, this is straightforward to build with convolution, pooling, and upsampling layers.

What Makes an FCN Different

A standard image classifier turns an image into one class prediction. An FCN turns an image into another image-shaped tensor, typically a segmentation map.

That leads to a few defining properties:

• no dense classifier head at the end
• output is spatial, not a single label
• the final tensor is often shaped like height x width x classes

For segmentation, each pixel receives a class score instead of the whole image receiving one label.

A Small FCN in TensorFlow Keras

Here is a compact example:

1import tensorflow as tf
2from tensorflow.keras import layers, Model
3
4def build_fcn(input_shape=(128, 128, 3), num_classes=3):
5    inputs = layers.Input(shape=input_shape)
6
7    # Encoder
8    x = layers.Conv2D(32, 3, padding="same", activation="relu")(inputs)
9    x = layers.MaxPool2D()(x)   # 64x64
10    x = layers.Conv2D(64, 3, padding="same", activation="relu")(x)
11    x = layers.MaxPool2D()(x)   # 32x32
12    x = layers.Conv2D(128, 3, padding="same", activation="relu")(x)
13
14    # Decoder
15    x = layers.Conv2DTranspose(64, 3, strides=2, padding="same", activation="relu")(x)
16    x = layers.Conv2DTranspose(32, 3, strides=2, padding="same", activation="relu")(x)
17
18    # Per-pixel class probabilities
19    outputs = layers.Conv2D(num_classes, 1, activation="softmax")(x)
20
21    return Model(inputs, outputs)
22
23model = build_fcn()
24model.compile(
25    optimizer="adam",
26    loss="sparse_categorical_crossentropy",
27    metrics=["accuracy"]
28)
29model.summary()

This model downsamples to learn higher-level features and then upsamples back to the original spatial resolution.

Understand the Output Shape

If your input is 128 x 128 x 3 and num_classes=3, the output is:

128 x 128 x 3

That does not mean the model returns three separate images. It means each pixel location has three class scores, one for each segmentation class.

During training with sparse_categorical_crossentropy, the mask is usually shaped like:

128 x 128

with integer class IDs at each pixel.

Skip Connections and Better FCNs

The original FCN family and later architectures often improve segmentation quality with skip connections that combine low-level spatial detail from earlier encoder layers with high-level semantic features from deeper layers.

A simple idea is:

• encoder extracts semantic meaning
• decoder restores resolution
• skip connections recover fine detail

If you need stronger segmentation quality, that is usually the next improvement after a basic encoder-decoder FCN.

Train with the Right Masks

Your masks must align with the network output:

• same spatial resolution as the final output
• integer labels for sparse loss, or one-hot labels for categorical loss
• identical geometric preprocessing as the images

A mismatch here is one of the fastest ways to get a model that trains badly or appears to learn nothing.

Common Pitfalls

The most common mistake is using image-classification thinking for a segmentation problem. An FCN must output a dense spatial map, not a single class vector.

Another issue is choosing the wrong loss for the mask encoding. sparse_categorical_crossentropy expects integer class labels, while categorical_crossentropy expects one-hot labels.

Developers also often forget to check output shape against the mask shape. If upsampling does not restore the expected resolution, training will fail or silently misbehave.

Finally, plain pixel accuracy is often not enough for evaluating segmentation. Metrics such as IoU or Dice are usually more informative.

Summary

• An FCN is designed for dense prediction tasks such as semantic segmentation.
• In TensorFlow, you can build one with convolution, pooling, and upsampling layers.
• The output is a per-pixel class map, not a single image label.
• Mask shape and loss function must match the output format.
• Skip connections and stronger segmentation metrics are common next steps after a basic FCN.