Introduction

Approximating the function x^2 with a neural network is a simple regression problem and a good way to understand function approximation. The task is not hard mathematically, but it is useful because it forces you to think about training data range, output activation, and loss selection. A small feedforward network is usually enough.

Why This Is a Regression Problem

The square function maps one real number to another real number. That means the network should be trained like a regression model, not like a classifier.

So the setup is usually:

• one input feature, which is x
• one output value, which is x^2
• a regression loss such as mean squared error
• a linear output layer

A minimal TensorFlow example looks like this:

1import numpy as np
2import tensorflow as tf
3
4x = np.linspace(-2.0, 2.0, 200).reshape(-1, 1).astype("float32")
5y = (x ** 2).astype("float32")
6
7model = tf.keras.Sequential([
8    tf.keras.layers.Input(shape=(1,)),
9    tf.keras.layers.Dense(16, activation="tanh"),
10    tf.keras.layers.Dense(16, activation="tanh"),
11    tf.keras.layers.Dense(1),
12])
13
14model.compile(optimizer="adam", loss="mse")
15model.fit(x, y, epochs=200, verbose=0)
16
17print(model.predict(np.array([[1.5]], dtype="float32"), verbose=0))

The final Dense(1) layer has no activation specified, so it behaves linearly. That is the right default for unrestricted numeric output.

Why a Small Network Is Enough

x^2 is smooth, deterministic, and one-dimensional. You do not need a deep architecture or huge dataset to learn it over a bounded interval.

In fact, the main challenge is not model size but data coverage. If you train only on x values between -1 and 1, you should not expect reliable behavior far outside that range.

That is an important lesson in approximation: the network learns the function on the domain you show it, not on all real numbers magically.

Data Range Matters

A network trained on [-2, 2] is really learning a good approximation on that interval. If you evaluate it at x = 20, the output may be poor because that is extrapolation, not interpolation.

A practical experiment makes this clear:

1import numpy as np
2
3x_test = np.array([[-2.0], [-1.0], [0.0], [1.0], [2.0], [5.0]], dtype="float32")
4preds = model.predict(x_test, verbose=0)
5for value, pred in zip(x_test.flatten(), preds.flatten()):
6    print(f"x={value:>4}, predicted={pred:>8.4f}, true={value**2:>8.4f}")

Inside the training range, the fit is usually good. Outside it, the model may deviate significantly.

Activation Choice and Symmetry

Since x^2 is an even function, symmetric training data helps. If you sample both negative and positive inputs, the model has a better chance of learning the true shape rather than a biased approximation.

Hidden-layer activations such as tanh or relu both work. tanh is a nice fit for small smooth toy problems because it is bounded and handles positive and negative inputs naturally. relu also works, but may need a bit more care in small approximators when the data range is narrow.

Loss Function and Evaluation

Mean squared error is the natural training loss here:

model.compile(optimizer="adam", loss="mse", metrics=["mae"])

For evaluation, mean absolute error can be easier to interpret because it tells you how far predictions are from the true squared values on average.

If the target values span a much larger range, scaling inputs and outputs can make training more stable.

Comparing With the Exact Function

It is worth remembering that a neural network is overkill for x^2. The exact formula is trivial. The value of this problem is educational, not practical.

You use it to learn:

• how regression networks behave
• how approximation depends on data range
• why output activations and losses matter
• how interpolation differs from extrapolation

That makes x^2 a good teaching function even though you would never deploy a neural network just to square a number.

Common Pitfalls

The biggest mistake is treating the problem like classification. A softmax output or cross-entropy loss is the wrong tool for continuous numeric targets.

Another mistake is training on too narrow an interval and then judging the model harshly for poor extrapolation outside that interval. That is a data-domain problem, not necessarily a network bug.

People also forget that a linear output layer is usually correct for regression. Adding a bounded final activation can distort the function unnecessarily.

Finally, do not confuse a good fit on sampled training points with exact mathematical understanding. The network approximates; it does not "discover" the symbolic formula for squaring.

Summary

• Approximating x^2 with a neural network is a simple regression task.
• Use one input, one output, a regression loss, and a linear output layer.
• A small feedforward network is usually enough on a bounded input range.
• Data coverage matters more than model size for this toy problem.
• The exercise is useful for learning approximation behavior, not because a neural network is the practical way to compute x^2.