Introduction

Nonlinear regression is about fitting a curve or surface when the relationship between inputs and outputs is not well described by a straight line. TensorFlow is a good fit for this because it lets you build flexible function approximators, optimize them with automatic differentiation, and train on arbitrary nonlinear patterns.

Create a Simple Nonlinear Dataset

A clear way to understand nonlinear regression is to generate data with a curved relationship and train a small neural network to approximate it.

1import numpy as np
2
3rng = np.random.default_rng(42)
4x = rng.uniform(-3.0, 3.0, size=(1000, 1)).astype("float32")
5y = (0.5 * x[:, 0] ** 2 + np.sin(3 * x[:, 0]) + rng.normal(0, 0.2, size=1000)).astype("float32")
6y = y.reshape(-1, 1)
7
8print(x.shape, y.shape)

This target is clearly nonlinear, so a plain linear model would struggle to match the curve.

Build a Small Keras Model

In TensorFlow, a multilayer perceptron is often the simplest nonlinear regressor. The hidden layers introduce nonlinear activation functions, which allow the model to represent curved relationships.

1import tensorflow as tf
2
3model = tf.keras.Sequential([
4    tf.keras.layers.Input(shape=(1,)),
5    tf.keras.layers.Dense(32, activation="relu"),
6    tf.keras.layers.Dense(32, activation="relu"),
7    tf.keras.layers.Dense(1)
8])
9
10model.compile(
11    optimizer=tf.keras.optimizers.Adam(learning_rate=0.01),
12    loss="mse",
13    metrics=["mae"]
14)

The final dense layer has no activation because regression usually predicts an unrestricted real-valued output.

Train and Evaluate the Regressor

Now fit the model and inspect the training behavior.

1history = model.fit(
2    x,
3    y,
4    epochs=100,
5    batch_size=32,
6    validation_split=0.2,
7    verbose=0
8)
9
10print("final training loss:", history.history["loss"][-1])
11print("final validation loss:", history.history["val_loss"][-1])

After training, run predictions for a few sample inputs:

test_x = tf.constant([[-2.0], [0.0], [2.0]], dtype=tf.float32)
pred = model.predict(test_x, verbose=0)
print(pred)

That gives you a working nonlinear regression pipeline with only a few lines of TensorFlow code.

Why Hidden Layers Make It Nonlinear

A network with only one dense layer and no nonlinear activation is still just a linear model, no matter how you train it. The nonlinear part comes from activation functions such as relu, tanh, or sigmoid.

That is why this model can learn curves while a simple linear regressor cannot. The hidden layers transform the input into a richer representation before the output layer predicts the final numeric value.

For smooth scientific curves, tanh can sometimes work well. For many practical problems, relu is a solid default because it trains efficiently and is easy to scale.

Input Scaling Helps a Lot

Nonlinear regression models often train better if inputs are scaled. For small one-dimensional demos this may not look critical, but in real applications it can significantly affect optimizer stability and convergence speed.

1import numpy as np
2
3x_mean = x.mean(axis=0)
4x_std = x.std(axis=0)
5x_scaled = (x - x_mean) / x_std

If you scale during training, remember to apply the same transformation at inference time.

When a Neural Network Is Not the Best Tool

TensorFlow can solve nonlinear regression, but that does not mean a neural network is always the right first choice. If you know the functional form already, curve fitting with a domain-specific model may be easier to interpret.

Neural networks are most attractive when:

• the curve shape is unknown
• there are multiple interacting features
• you want one flexible model family for many problems

If interpretability is the main priority, a simpler nonlinear model may still be a better fit.

Common Pitfalls

One common mistake is building a regression model with no nonlinear activation in the hidden layers. That leaves you with a linear model even if the target relationship is curved.

Another is overfitting a tiny dataset with a network that is far too large. Validation loss is the signal to watch, not just the training loss.

Developers also sometimes forget to scale inputs or inspect the target range, which can make training unstable and predictions hard to interpret.

Finally, remember that regression targets are continuous. Using classification metrics or a classification-style output layer is the wrong setup for this problem.

Summary

• Nonlinear regression fits curved relationships rather than straight lines.
• In TensorFlow, a small neural network with nonlinear activations is a practical baseline.
• Hidden layers with activations are what make the model nonlinear.
• Input scaling and validation monitoring improve training stability.
• Use neural networks when you need flexibility, but do not ignore simpler nonlinear models when they fit the problem better.