Introduction

Approximating x^2 with a neural network is a small problem, but it is a useful one because it isolates the core idea of supervised learning: fit a nonlinear mapping from examples. Since the target function is known exactly, you can focus on data scaling, model capacity, and training behavior instead of application noise.

Why a Network Can Learn x^2

A feed-forward network with nonlinear activations can approximate smooth functions over a bounded interval. For x^2, the task is regression: input one number, output one number.

The important detail is the interval. A network trained only on [-1, 1] may perform well there and fail badly on [-100, 100]. Function approximation is local to the data distribution unless you intentionally train for broader coverage.

For this problem, a small multilayer perceptron is enough:

• one scalar input
• one or two hidden layers
• nonlinear activation such as Tanh or ReLU
• one scalar output

Because x^2 is smooth and symmetric, the network does not need to be deep. The training setup matters more than model size.

A Minimal PyTorch Example

The following script generates training data, fits a small network, and prints a few predictions:

1import torch
2from torch import nn
3
4torch.manual_seed(7)
5
6# Training data on a bounded interval.
7x = torch.linspace(-2.0, 2.0, 200).unsqueeze(1)
8y = x ** 2
9
10model = nn.Sequential(
11    nn.Linear(1, 16),
12    nn.Tanh(),
13    nn.Linear(16, 16),
14    nn.Tanh(),
15    nn.Linear(16, 1),
16)
17
18loss_fn = nn.MSELoss()
19optimizer = torch.optim.Adam(model.parameters(), lr=0.01)
20
21for epoch in range(2000):
22    pred = model(x)
23    loss = loss_fn(pred, y)
24
25    optimizer.zero_grad()
26    loss.backward()
27    optimizer.step()
28
29    if epoch % 400 == 0:
30        print(f"epoch={epoch} loss={loss.item():.6f}")
31
32test = torch.tensor([[-2.0], [-0.5], [0.0], [1.5], [2.0]])
33with torch.no_grad():
34    predicted = model(test)
35
36for value, estimate in zip(test.squeeze(), predicted.squeeze()):
37    print(f"x={value.item():.1f}, y_hat={estimate.item():.4f}")

This is runnable as-is if PyTorch is installed. The predicted values should be close to 4.0, 0.25, 0.0, 2.25, and 4.0.

Why Normalization Helps

Even simple functions benefit from scaled inputs. Large raw values produce larger targets, larger gradients, and harder optimization. When the goal is to demonstrate learning rather than stress-test numeric range, normalize first.

For example, if your real inputs live in [0, 1000], you can divide by 1000 during training and rescale later:

1def normalize(x):
2    return x / 1000.0
3
4def denormalize_square(y_scaled):
5    return y_scaled * (1000.0 ** 2)

This keeps the optimization problem well-behaved. It also makes the network spend capacity learning shape rather than raw magnitude.

Interpreting the Result

A trained network does not "discover the formula" in symbolic form. It learns weights that approximate the relationship over the range it saw during training. That distinction matters.

If you ask for:

• exact arithmetic
• guaranteed extrapolation
• a closed-form rule

then a neural network is the wrong tool. For x^2, direct computation is simpler, faster, and exact. The value of this example is educational: it shows how neural networks fit continuous functions and where that process can fail.

Choosing Activations and Loss

MSELoss is the natural choice because the target is continuous. Activation choice is flexible:

• 'Tanh works well on small normalized intervals'
• 'ReLU can work too, but may need slightly more width to fit curvature smoothly'
• no activation should be used in the final layer for plain regression

You also do not need a huge dataset. Since the target function is deterministic, a few hundred evenly spaced samples are enough to demonstrate the idea clearly.

Common Pitfalls

One mistake is expecting good extrapolation. A network trained on [-2, 2] may output nonsense at 10, even if the in-range fit looks excellent.

Another mistake is using a classification-style output activation such as Sigmoid in the final layer. That compresses predictions into a limited range and makes it impossible to represent larger squared values.

Overbuilding the model is another frequent issue. A deep network can fit x^2, but it adds noise to the learning problem and makes the example harder to reason about. Start small.

Finally, do not judge the model only by the training loss. Print several predicted values across the interval and compare them with the real targets. For function approximation, spot checks reveal shape errors quickly.

Summary

• Approximating x^2 is a simple regression task that illustrates core neural-network behavior.
• A small feed-forward network is enough when training data covers a bounded interval.
• Input scaling improves optimization and makes the fit more stable.
• The network learns an approximation over the observed range, not an exact symbolic rule.
• For real computation of squares, direct arithmetic is better; the neural-network version is mainly a learning exercise.