Introduction

XOR is the standard example of a pattern that a single linear layer cannot represent. The four input points are not linearly separable, so a perceptron fails no matter how long you train it. A small multilayer perceptron with a hidden layer solves it easily, which is why XOR is still useful for checking whether a neural-network setup is structurally correct.

Why a Single Layer Fails

The XOR truth table is:

• '0, 0 -> 0'
• '0, 1 -> 1'
• '1, 0 -> 1'
• '1, 1 -> 0'

No single straight decision boundary can separate the positive and negative examples. That means you need at least one hidden layer with a nonlinear activation.

A Minimal TensorFlow Model

A tiny Keras model is enough. The key ingredients are:

• input size 2
• a hidden layer with a nonlinear activation such as tanh
• an output layer with sigmoid
• enough training epochs for the tiny dataset

1import numpy as np
2import tensorflow as tf
3
4tf.random.set_seed(42)
5np.random.seed(42)
6
7X = np.array([
8    [0.0, 0.0],
9    [0.0, 1.0],
10    [1.0, 0.0],
11    [1.0, 1.0],
12], dtype=np.float32)
13
14y = np.array([[0.0], [1.0], [1.0], [0.0]], dtype=np.float32)
15
16model = tf.keras.Sequential([
17    tf.keras.layers.Input(shape=(2,)),
18    tf.keras.layers.Dense(4, activation="tanh"),
19    tf.keras.layers.Dense(1, activation="sigmoid"),
20])
21
22model.compile(
23    optimizer=tf.keras.optimizers.Adam(learning_rate=0.05),
24    loss="binary_crossentropy",
25    metrics=["accuracy"],
26)
27
28model.fit(X, y, epochs=500, verbose=0)
29
30predictions = model.predict(X, verbose=0)
31print(np.round(predictions, 3))

This should produce outputs close to 0, 1, 1, 0.

Why This Architecture Works

The hidden layer lets the network build an intermediate representation where XOR becomes separable. You can think of the hidden units as learning multiple regions in the input space, then the output layer combines those regions into the final classification.

The exact number of hidden units is not very important here. Even a very small hidden layer can solve XOR. What matters is that you include a nonlinear activation.

Good Defaults for XOR

For a toy problem like XOR, overly complex architecture is usually a sign of confusion. A good baseline is:

• 'Dense(4, activation="tanh")'
• 'Dense(1, activation="sigmoid")'
• 'binary_crossentropy'
• 'Adam'

You can also use relu, but tanh often works smoothly for this tiny symmetric dataset.

How to Verify Learning

Do not stop at training accuracy. Inspect the raw predictions.

for features, pred in zip(X, predictions):
    print(features, float(pred))

This helps you see whether the network truly separated the four cases or just hovered around ambiguous values.

For XOR, a correct model should predict values near:

• '0 for [0, 0]'
• '1 for [0, 1]'
• '1 for [1, 0]'
• '0 for [1, 1]'

Common Reasons It Does Not Learn

If the network fails, the usual causes are:

• using no hidden layer
• using a linear activation everywhere
• training too briefly
• using an unstable learning rate
• shape mismatches between labels and outputs

A single dense output layer with sigmoid still cannot solve XOR because the core issue is representational, not just probabilistic output.

A Slightly More Explicit Functional Model

If you prefer the functional API:

1inputs = tf.keras.Input(shape=(2,))
2x = tf.keras.layers.Dense(4, activation="tanh")(inputs)
3outputs = tf.keras.layers.Dense(1, activation="sigmoid")(x)
4model = tf.keras.Model(inputs, outputs)

This behaves the same but makes it easier to inspect or extend the architecture later.

Why XOR Still Matters

XOR is small, but it tests the basics correctly. If a framework setup cannot learn XOR, the issue is often with:

• model structure
• activation choice
• label shape
• optimizer or training loop configuration

That makes it a useful sanity check when experimenting with custom training code.

Common Pitfalls

The biggest pitfall is assuming "MLP" automatically means the model is nonlinear. If you omit the hidden layer or use only linear activations, XOR still fails.

Another issue is undertraining. Four samples do not mean one epoch is enough. Toy problems often need enough iterations for the network to settle.

Be careful with output thresholds too. A prediction of 0.49 is not a correct confident zero just because it rounds down.

Finally, do not make the example harder than necessary. XOR is a structural demonstration, not a benchmark.

Summary

• XOR is not linearly separable, so a single-layer model cannot solve it.
• A simple MLP with one hidden layer and a nonlinear activation is enough.
• 'tanh in the hidden layer and sigmoid in the output layer is a solid baseline.'
• Train long enough and inspect the actual predictions, not only accuracy.
• If XOR does not learn, the model setup is usually structurally wrong.
• XOR remains a useful sanity check for TensorFlow and neural-network basics.