Introduction

Polynomial regression in TensorFlow is still linear regression at the parameter level. The non-linearity comes from expanding the input into polynomial features such as x, x², and x³, then fitting a linear model on that transformed input.

Build Polynomial Features Explicitly

Suppose you start with one input feature x and want a cubic fit. The first step is to turn each scalar input into several polynomial terms.

1import tensorflow as tf
2
3
4def poly_features(x_tensor, degree):
5    cols = [tf.pow(x_tensor, i) for i in range(1, degree + 1)]
6    return tf.stack(cols, axis=1)
7
8
9x = tf.linspace(-2.0, 2.0, 300)
10noise = tf.random.normal([300], stddev=0.15)
11y = 1.2 * x * x - 0.7 * x + 0.3 + noise
12
13X = poly_features(x, degree=3)
14print(X.shape)

This is what makes the problem “polynomial.” The model itself is still just learning weights for a fixed feature vector.

Fit the Expanded Features with a Dense Layer

A single dense layer is enough for polynomial regression once the features are expanded.

1model = tf.keras.Sequential([
2    tf.keras.layers.Input(shape=(3,)),
3    tf.keras.layers.Dense(1)
4])
5
6model.compile(
7    optimizer=tf.keras.optimizers.Adam(1e-2),
8    loss="mse"
9)
10
11model.fit(X, y, epochs=300, verbose=0)
12pred = tf.squeeze(model(X), axis=1)
13print(tf.reduce_mean(tf.square(pred - y)).numpy())

This is just linear regression on the transformed feature matrix.

Normalize Features for Stability

Polynomial terms grow quickly, especially for higher degrees. That can make optimization unstable unless you normalize the inputs.

1norm = tf.keras.layers.Normalization()
2norm.adapt(X)
3
4stable_model = tf.keras.Sequential([
5    tf.keras.layers.Input(shape=(3,)),
6    norm,
7    tf.keras.layers.Dense(1)
8])
9
10stable_model.compile(optimizer="adam", loss="mse")
11stable_model.fit(X, y, epochs=300, verbose=0)

Normalization often matters more as the degree increases, because x⁵ and x⁶ can have very different scales from x itself.

Choose Polynomial Degree with Validation

The main modeling decision is not “Can TensorFlow fit this curve?” It is “How much polynomial flexibility should I allow before overfitting?”

A simple validation split helps answer that.

1idx = tf.range(tf.shape(x)[0])
2train_idx = idx[:240]
3val_idx = idx[240:]
4
5x_train = tf.gather(x, train_idx)
6y_train = tf.gather(y, train_idx)
7x_val = tf.gather(x, val_idx)
8y_val = tf.gather(y, val_idx)

Then try several degrees, fit them on the training portion, and compare validation loss. If training loss improves while validation loss worsens, the polynomial basis is probably too flexible.

Add Regularization When Degree Grows

Higher-degree polynomial fits can become unstable and sensitive to noise. A small L2 penalty can help control that.

1reg_model = tf.keras.Sequential([
2    tf.keras.layers.Input(shape=(5,)),
3    tf.keras.layers.Dense(1, kernel_regularizer=tf.keras.regularizers.l2(1e-3))
4])

Regularization does not replace validation, but it can make high-degree fits less brittle.

Plot the Fit, Not Just the Loss

A quick visual check is often worth more than one scalar loss value.

1import matplotlib.pyplot as plt
2
3plt.scatter(x.numpy(), y.numpy(), s=8, alpha=0.5, label="data")
4plt.plot(x.numpy(), pred.numpy(), color="red", label="fit")
5plt.legend()
6plt.show()

If the curve oscillates strangely between points, you may have chosen too high a degree even if the training loss looks excellent.

Polynomial Regression Is Useful, But Narrow

This approach is great when:

• input dimensionality is low
• the relationship is smooth
• interpretability matters
• you want a simple baseline

It is not usually the right model for complex high-dimensional patterns where tree-based models or deeper networks fit the structure better.

Common Pitfalls

A common mistake is increasing the degree until training loss is tiny without checking validation behavior.

Another mistake is skipping feature normalization and then blaming TensorFlow when optimization becomes unstable.

Developers also often forget that polynomial regression is still linear in the learned parameters. The non-linearity is in the feature engineering.

Finally, if deployment matters, save the feature-construction degree and normalization settings together with the model. Inference must use the same transformation as training.

Summary

• Polynomial regression in TensorFlow is linear regression on polynomially expanded features.
• A single dense layer is enough once those features are built.
• Normalize expanded features for more stable optimization.
• Choose the polynomial degree using validation, not training loss alone.
• Plot the fitted curve so overfitting is visible, not just numerical.