Introduction

In TensorFlow linear regression, the attribute matrix is the input feature matrix and the target matrix is the set of values the model should learn to predict. The most important part is shape discipline: rows represent samples, columns represent features, and the target must line up with the same sample order.

If the data shapes are wrong, the model will either fail immediately or train against mismatched labels. So before thinking about optimization, make sure the matrices are organized correctly.

Attribute Matrix Versus Target Matrix

For a dataset with n samples and m features:

• the attribute matrix X typically has shape [n, m]
• the target matrix y usually has shape [n, 1] for a single regression output

Example with two features per row:

1import numpy as np
2
3X = np.array([
4    [1500, 3],
5    [2500, 4],
6    [1200, 2],
7    [3000, 5],
8], dtype=np.float32)
9
10y = np.array([
11    [300000],
12    [450000],
13    [200000],
14    [500000],
15], dtype=np.float32)
16
17print(X.shape)  # (4, 2)
18print(y.shape)  # (4, 1)

Each row in X must correspond to the target value in the same row of y.

A Simple TensorFlow Linear Regression Model

In modern TensorFlow, the easiest linear regression model is a one-unit dense layer with no activation:

1import numpy as np
2import tensorflow as tf
3
4X = np.array([
5    [1.0, 2.0],
6    [2.0, 1.0],
7    [3.0, 4.0],
8    [4.0, 3.0],
9], dtype=np.float32)
10
11y = np.array([
12    [5.0],
13    [5.0],
14    [11.0],
15    [11.0],
16], dtype=np.float32)
17
18model = tf.keras.Sequential([
19    tf.keras.layers.Input(shape=(2,)),
20    tf.keras.layers.Dense(1),
21])
22
23model.compile(optimizer="adam", loss="mse")
24model.fit(X, y, epochs=200, verbose=0)
25
26prediction = model.predict(np.array([[5.0, 6.0]], dtype=np.float32), verbose=0)
27print(prediction)

That single dense unit computes a linear combination of the input features plus a bias term, which is exactly what linear regression needs.

Why Matrix Shape Matters

The feature matrix shape tells TensorFlow how many coefficients to learn. If X has shape [n, 2], then the model learns two weights and one bias for a single-output regression.

A common mistake is using a one-dimensional target array when later code expects a two-dimensional matrix. TensorFlow often handles both, but keeping y as shape [n, 1] avoids ambiguity and is easier to reason about.

Good:

y = np.array([[1.0], [2.0], [3.0]], dtype=np.float32)

Riskier for beginners:

y = np.array([1.0, 2.0, 3.0], dtype=np.float32)

Both may work in some cases, but the first shape matches the mental model of one target column.

Feature Scaling Helps Training

If one feature is in the thousands and another is a small count, optimization can be slower or less stable. Scaling inputs often helps:

1X = np.array([
2    [1500, 3],
3    [2500, 4],
4    [1200, 2],
5    [3000, 5],
6], dtype=np.float32)
7
8X_mean = X.mean(axis=0)
9X_std = X.std(axis=0)
10X_scaled = (X - X_mean) / X_std

Linear regression still works without scaling, but the training process is often easier when feature ranges are comparable.

Manual TensorFlow Version

If you want to see the matrix perspective directly, you can also implement linear regression with raw variables:

1import tensorflow as tf
2import numpy as np
3
4X = tf.constant([[1.0, 2.0], [2.0, 3.0], [3.0, 4.0]], dtype=tf.float32)
5y = tf.constant([[5.0], [8.0], [11.0]], dtype=tf.float32)
6
7w = tf.Variable(tf.random.normal([2, 1]))
8b = tf.Variable(tf.zeros([1]))
9optimizer = tf.keras.optimizers.SGD(learning_rate=0.01)
10
11for _ in range(1000):
12    with tf.GradientTape() as tape:
13        predictions = tf.matmul(X, w) + b
14        loss = tf.reduce_mean(tf.square(predictions - y))
15
16    gradients = tape.gradient(loss, [w, b])
17    optimizer.apply_gradients(zip(gradients, [w, b]))
18
19print("weights:", w.numpy())
20print("bias:", b.numpy())

This makes the matrix equation visible: X multiplied by w, plus b.

Common Pitfalls

The biggest pitfall is misaligning rows between X and y. If you shuffle one without the other, the model trains on the wrong targets.

Another issue is using the wrong input shape in the Keras model. If you have two features, the input layer should expect shape (2,), not (1,).

Developers also sometimes confuse a single sample with a single feature. A shape of [1, m] means one sample with m features, while [n, 1] means n samples with one feature each.

Finally, remember that linear regression is only linear in the features you provide. If the relationship is strongly nonlinear, the matrix setup may be correct but the model class may still be too simple.

Summary

• The attribute matrix X stores samples by rows and features by columns.
• The target matrix y must align row-for-row with X.
• In TensorFlow, a one-unit dense layer is the simplest linear regression model.
• Keep shapes explicit, especially for y, to avoid silent confusion.
• Good matrix organization is the foundation of correct linear regression training.