Introduction

Linear regression models a numeric target as a weighted sum of one or more input features. With Pandas and NumPy, you can prepare tabular data, build the design matrix, solve for coefficients, and generate predictions without pulling in a full machine learning framework.

That makes it a good exercise for understanding what regression is actually doing under the hood. Pandas handles the table-shaped data, and NumPy handles the matrix algebra.

Build a Small Regression Dataset

Start with a DataFrame so the feature columns are named and easy to inspect.

1import pandas as pd
2
3data = pd.DataFrame({
4    "hours_studied": [1, 2, 3, 4, 5, 6],
5    "practice_tests": [0, 1, 1, 2, 2, 3],
6    "score": [52, 57, 63, 68, 72, 78],
7})
8
9print(data)

We want to predict score from the two feature columns.

Create the Design Matrix With NumPy

Linear regression needs an intercept term. The usual trick is to add a column of ones to the feature matrix.

1import numpy as np
2
3X = data[["hours_studied", "practice_tests"]].to_numpy(dtype=float)
4y = data["score"].to_numpy(dtype=float)
5
6intercept = np.ones((X.shape[0], 1))
7X_design = np.hstack([intercept, X])
8
9print(X_design)
10print(y)

Now the first column represents the intercept coefficient, and the remaining columns represent feature coefficients.

Fit the Regression Model

A common way to solve ordinary least squares is np.linalg.lstsq. It is more stable than manually inverting matrices.

1coefficients, residuals, rank, singular_values = np.linalg.lstsq(
2    X_design, y, rcond=None
3)
4
5intercept_value = coefficients[0]
6feature_weights = coefficients[1:]
7
8print("Intercept:", intercept_value)
9print("Weights:", feature_weights)

If the output were something like:

• Intercept around 47
• Weight for hours_studied around 4
• Weight for practice_tests around 2

then the model would be estimating score roughly as base score plus four points per study hour plus two points per practice test.

Make Predictions

Once the coefficients are known, predictions are just matrix multiplication.

1predictions = X_design @ coefficients
2
3result = data.copy()
4result["predicted_score"] = predictions
5result["residual"] = result["score"] - result["predicted_score"]
6
7print(result)

To predict on new data, build new rows with the same feature order and the same intercept column.

1new_students = np.array([
2    [1.0, 2.5, 1.0],
3    [1.0, 5.0, 2.0],
4])
5
6new_predictions = new_students @ coefficients
7print(new_predictions)

Evaluate the Fit

Even a simple regression should be evaluated. One quick metric is the coefficient of determination, or R^2.

1ss_res = np.sum((y - predictions) ** 2)
2ss_tot = np.sum((y - np.mean(y)) ** 2)
3r_squared = 1 - (ss_res / ss_tot)
4
5print("R^2:", r_squared)

An R^2 value closer to 1 means the model explains more of the variation in the target. That does not prove the model is good in every practical sense, but it is a useful first check.

Why Use Pandas and NumPy Together

Pandas is convenient for cleaning and selecting columns. NumPy is better for the matrix math itself. That pairing keeps the workflow lightweight and transparent:

• Use Pandas to inspect, filter, and transform the table.
• Convert to NumPy arrays for fitting and prediction.
• Write results back to a DataFrame for analysis.

This is often enough for small projects, teaching, prototyping, or sanity-checking a model before moving to scikit-learn.

Common Pitfalls

One common mistake is forgetting the intercept column. If you omit it, the model is forced through zero, which can distort the coefficients badly.

Another issue is non-numeric data. Pandas columns that look numeric may still contain strings or missing values. Convert types and handle nulls before calling NumPy.

Multicollinearity is another practical problem. If two features contain nearly the same information, the fitted weights may become unstable even if predictions look reasonable.

Finally, do not judge a regression only on the training data. A model can appear accurate on a tiny hand-built dataset and still generalize poorly on new observations.

Summary

• Use Pandas for data preparation and NumPy for the regression math.
• Build a design matrix with an intercept column before fitting.
• Prefer np.linalg.lstsq over manual matrix inversion.
• Predict with matrix multiplication once the coefficients are known.
• Check fit quality and data assumptions before trusting the model.