Introduction

Normalisation and regularisation sound similar, but they solve different problems in a machine learning pipeline. Normalisation changes the scale of data, while regularisation changes how strongly a model is allowed to fit that data.

Normalisation Changes the Input Scale

Normalisation is a preprocessing step applied to features before or during training. The goal is to put values on comparable scales so that optimization behaves well and no single feature dominates because of its unit of measurement.

Common examples include min-max scaling and standardization. In practice, standardization is often used for linear models, logistic regression, and neural networks because it centers features and gives them similar spread.

1import numpy as np
2from sklearn.preprocessing import StandardScaler
3
4X = np.array(
5    [
6        [18, 25000],
7        [25, 52000],
8        [42, 87000],
9        [57, 120000],
10    ],
11    dtype=float,
12)
13
14scaler = StandardScaler()
15X_scaled = scaler.fit_transform(X)
16
17print(X_scaled.mean(axis=0).round(6))
18print(X_scaled.std(axis=0).round(6))

Without scaling, the income column would dominate the age column simply because its numeric range is much larger. Normalisation does not change the meaning of the target variable, and it does not directly prevent overfitting. Its job is to make the input representation easier for the model to work with.

Regularisation Controls Model Complexity

Regularisation is a modeling technique that discourages overly complex solutions. It reduces overfitting by adding a penalty or constraint that pushes the model away from extreme parameter values.

For linear models, this often means adding an L1 or L2 penalty to the loss function. L2 regularisation, used by ridge regression, shrinks weights toward zero. L1 regularisation, used by lasso, can shrink some weights all the way to zero and effectively perform feature selection.

1import numpy as np
2from sklearn.linear_model import Ridge
3from sklearn.pipeline import make_pipeline
4from sklearn.preprocessing import StandardScaler
5
6X = np.array(
7    [
8        [1.0, 10.0],
9        [2.0, 20.0],
10        [3.0, 31.0],
11        [4.0, 39.0],
12        [5.0, 52.0],
13    ]
14)
15y = np.array([3.0, 5.0, 7.5, 8.5, 11.0])
16
17model = make_pipeline(StandardScaler(), Ridge(alpha=1.0))
18model.fit(X, y)
19
20ridge = model.named_steps["ridge"]
21print(ridge.coef_)

Here the alpha value controls the strength of the penalty. Higher regularisation usually lowers variance and increases bias, so it must be tuned rather than guessed.

They Are Often Used Together

These ideas complement each other. A typical pipeline may normalize the features first and then train a regularized model on the normalized data.

That sequence is common for a reason:

• scaling makes optimization numerically stable
• scaling makes regularisation penalties more comparable across features
• regularisation improves generalization by discouraging oversized weights

If one feature ranges from 0 to 1 and another ranges from 0 to 1,000,000, the regularisation term does not act fairly until the inputs are scaled. So even though the concepts are different, they interact in practice.

Regularisation Is Broader Than Weight Penalties

In neural networks, the word regularisation covers more than L1 and L2 penalties. Dropout, data augmentation, early stopping, and label smoothing are also used to reduce overfitting.

Normalisation also has a broader meaning in deep learning. You might hear about batch normalization or layer normalization, but those are architectural techniques inside the network, not the same thing as scaling raw input features with a preprocessing tool. The shared word can cause confusion, so it helps to ask whether the discussion is about input scaling or model behavior during training.

Common Pitfalls

• Treating normalisation and regularisation as interchangeable. One changes the data representation, while the other changes the fitting behavior.
• Fitting a scaler on the full dataset before the train and test split. That leaks information from evaluation data into training.
• Adding heavy regularisation to fix optimization issues that actually come from poorly scaled inputs.
• Forgetting that regularisation strength depends on the scale of the features. Unscaled data can make penalties behave unpredictably.
• Assuming normalization always means min-max scaling. Standardization, batch normalization, and layer normalization are related ideas but not the same tool.

Summary

• Normalisation rescales features so training and distance calculations behave better.
• Regularisation limits model complexity so the model generalizes better to unseen data.
• They solve different problems and are often used together in the same pipeline.
• Scale features using training data only, then apply the same transform to validation and test data.
• Tune regularisation strength deliberately, because too little leads to overfitting and too much leads to underfitting.