Introduction

The perceptron update rule is one of the clearest examples of learning by correction. When the model makes a mistake, it changes the weights so that the mistaken example would receive a better score next time.

The Decision Rule Comes First

A perceptron computes a linear score from an input vector x, a weight vector w, and a bias b.

score = w · x + b

The predicted class depends on the sign of the score:

• positive score means one class
• negative score means the other class

So the perceptron is learning a line or hyperplane that splits the feature space.

The Update Rule

When an example is classified incorrectly, a standard perceptron update is:

w = w + eta * y * x
b = b + eta * y

where:

• 'eta is the learning rate'
• 'y is the true label, usually +1 or -1'
• the update happens only on mistakes

This rule looks simple because it is simple. It says: move the model in the direction that would have helped with the example it just got wrong.

Geometric Intuition

The weight vector is perpendicular to the decision boundary. Changing w rotates or shifts that boundary.

If a positive example is mistakenly predicted as negative, adding x to w tends to increase the score for that example in the future.

If a negative example is mistakenly predicted as positive, the sign of y causes the update to push the boundary in the opposite direction.

So the perceptron is not searching everywhere at once. It is doing local mistake-driven nudges.

A Small Numerical Example

Suppose:

• 'w = [0, 0]'
• 'b = 0'
• 'eta = 1'
• 'x = [2, 1]'
• 'y = +1'

The current score is zero, so treat that as a mistake. The update becomes:

w = [0, 0] + 1 * 1 * [2, 1] = [2, 1]
b = 0 + 1 * 1 = 1

Now the same input gets score:

2*2 + 1*1 + 1 = 6

which is clearly on the positive side. One update already moved the boundary toward correctness.

Runnable Example

1import numpy as np
2
3
4def perceptron_train(X, y, epochs=20, lr=1.0):
5    w = np.zeros(X.shape[1], dtype=float)
6    b = 0.0
7
8    for _ in range(epochs):
9        errors = 0
10        for xi, yi in zip(X, y):
11            score = np.dot(w, xi) + b
12            pred = 1 if score >= 0 else -1
13            if pred != yi:
14                w += lr * yi * xi
15                b += lr * yi
16                errors += 1
17        if errors == 0:
18            break
19
20    return w, b
21
22
23X = np.array([
24    [2, 1],
25    [1, 1],
26    [-1, -1],
27    [-2, -1],
28], dtype=float)
29
30y = np.array([1, 1, -1, -1], dtype=int)
31
32w, b = perceptron_train(X, y)
33print("weights:", w)
34print("bias:", b)

This works well because the dataset is linearly separable.

Why It Converges on Separable Data

The perceptron convergence result says that if a linear separator exists, repeated mistake-driven updates eventually find one. It does not promise the maximum-margin separator, only a separating boundary.

If the data is not linearly separable, the perceptron can keep changing forever. That is not a bug in the code. It is a limit of the model class.

Common Pitfalls

The most common mistake is using labels 0 and 1 with an update rule written for -1 and +1.

Another mistake is forgetting the bias update. Without b, the boundary is forced through the origin.

It is also easy to expect convergence on data that cannot be separated by a linear boundary at all.

Summary

• The perceptron updates weights only when it makes a mistake.
• Each update moves the decision boundary in a direction that helps the misclassified example.
• Positive and negative mistakes push the boundary in opposite directions.
• The algorithm converges when a linear separator exists.
• Perceptron intuition is a useful foundation for later gradient-based models.