Introduction

Weighted linear regression is ordinary least squares with unequal importance assigned to different observations. In practice, that means points with larger weights influence the fitted line more strongly, which is useful when some samples are more reliable, represent more cases, or should count more heavily in the objective.

What the Weights Mean

In an unweighted regression, every residual contributes equally to the loss. In weighted regression, the model minimizes a weighted squared error instead. A large weight tells the solver that errors on that observation matter more.

Common reasons to use weights include:

• heteroscedastic data where some observations are noisier than others
• aggregated data where one row represents many identical cases
• business rules where certain samples should have more impact

The idea is not to make the model "better" automatically. It is to make the fitted objective match the meaning of the data.

Use sample_weight With LinearRegression

Scikit-learn already supports weighted fitting through the sample_weight argument on several estimators, including LinearRegression.

1import numpy as np
2from sklearn.linear_model import LinearRegression
3
4X = np.array([[1], [2], [3], [4], [5]], dtype=float)
5y = np.array([1.2, 1.9, 3.1, 4.2, 10.0], dtype=float)
6weights = np.array([1, 1, 1, 1, 0.1], dtype=float)
7
8model = LinearRegression()
9model.fit(X, y, sample_weight=weights)
10
11print(model.coef_)
12print(model.intercept_)
13print(model.predict([[6.0]]))

In this example, the last point has a much smaller weight, so it has less influence on the line than it would in an ordinary unweighted fit.

That is the main Scikit-learn API you need. There is no separate special estimator just for basic weighted least squares.

Compare Weighted and Unweighted Fits

A quick comparison helps make the behavior concrete.

1import numpy as np
2from sklearn.linear_model import LinearRegression
3
4X = np.array([[1], [2], [3], [4], [5]], dtype=float)
5y = np.array([1.2, 1.9, 3.1, 4.2, 10.0], dtype=float)
6weights = np.array([1, 1, 1, 1, 0.1], dtype=float)
7
8plain = LinearRegression().fit(X, y)
9weighted = LinearRegression().fit(X, y, sample_weight=weights)
10
11print("plain coef", plain.coef_[0], "intercept", plain.intercept_)
12print("weighted coef", weighted.coef_[0], "intercept", weighted.intercept_)

The unweighted model gets pulled more strongly toward the high outlying point. The weighted model treats that point as less trustworthy and fits a line closer to the first four observations.

That is not always the right decision, but it shows clearly what weighting does.

Choose Weights for a Statistical Reason

The hardest part is not the API. It is deciding what the weights should mean.

If weights represent inverse noise variance, then larger weights belong to more precise observations. If weights represent row frequency, then a row with weight 10 stands in for ten repeated samples. Those are both valid interpretations, but they are not the same interpretation.

You should not choose weights arbitrarily just to force the model toward a preferred outcome. That turns weighting into hidden manual bias rather than a justified modeling assumption.

Weighted Pipelines Still Work Normally

If your model sits inside preprocessing steps, you can still pass sample weights through the estimator at fit time.

1from sklearn.pipeline import make_pipeline
2from sklearn.preprocessing import StandardScaler
3from sklearn.linear_model import LinearRegression
4
5pipe = make_pipeline(StandardScaler(), LinearRegression())
6pipe.fit(X, y, linearregression__sample_weight=weights)

The key detail is that the weight parameter must be forwarded to the named estimator inside the pipeline.

Common Pitfalls

A common mistake is assuming Scikit-learn lacks weighted regression entirely. For ordinary linear regression, sample_weight is already supported.

Another mistake is treating weights as if they were arbitrary tuning knobs. Weights need a defensible meaning tied to the data-generation process or business interpretation.

People also often compare weighted and unweighted models using the same intuition but forget that the loss functions are different. A better fit under one objective is not automatically a better fit under the other.

Finally, be careful when combining weights with train-test evaluation. If weighting matters during training, think carefully about whether evaluation should also reflect weighted or unweighted error depending on the question you are asking.

Summary

• Weighted linear regression gives different observations different influence in the loss.
• In Scikit-learn, use sample_weight when calling fit.
• Larger weights make a point matter more; smaller weights reduce its impact.
• The real challenge is choosing weights that have a clear statistical or domain meaning.
• Weighted and unweighted regression solve different optimization problems, so interpret the results accordingly.