Introduction

scikit-learn is designed primarily for prediction, model selection, and production-style workflows. Its LinearRegression estimator gives you coefficients and predictions, but it does not calculate p-values or the t-tests needed for classical statistical inference.

If you need significance tests, you usually have two practical choices: fit the same model in statsmodels, or compute the standard errors and p-values yourself after fitting the regression. The right option depends on whether you care more about convenience or keeping a pure scikit-learn pipeline.

Why LinearRegression Does Not Expose P-Values

P-values are not just another model attribute. They come from assumptions about the residual distribution, the design matrix, and the sampling process. A prediction library can fit a least-squares line without promising that those assumptions hold.

That is why LinearRegression stops at:

• estimated coefficients
• intercept
• predictions
• residual fit quality metrics that you compute separately

If your task is hypothesis testing, confidence intervals, or coefficient significance, a statistics-oriented library is often the better fit.

Use statsmodels When You Need Inference

For classical ordinary least squares, statsmodels is the simplest answer. It reports coefficients, standard errors, t-statistics, and p-values directly.

1import numpy as np
2import statsmodels.api as sm
3
4X = np.array([1.0, 2.0, 3.0, 4.0, 5.0, 6.0])
5y = np.array([1.9, 3.0, 4.2, 4.8, 6.1, 7.4])
6
7X_with_const = sm.add_constant(X)
8model = sm.OLS(y, X_with_const).fit()
9
10print("coefficients:", model.params)
11print("p-values:", model.pvalues)
12print(model.summary())

Two details matter here:

• 'sm.add_constant adds the intercept column explicitly.'
• 'model.pvalues contains one p-value for the intercept and one for each feature.'

If your analysis is mostly inferential, this is the cleanest approach because the library already handles the regression diagnostics that scikit-learn intentionally leaves out.

Compute P-Values After Fitting scikit-learn

If you still want to fit with scikit-learn, you can derive p-values from the residual variance and the covariance matrix of the estimated coefficients. The formula is standard ordinary least squares:

• fit the model
• compute residuals
• estimate residual variance
• invert X^T X
• turn standard errors into t-statistics
• convert those t-statistics into two-sided p-values

1import numpy as np
2from scipy import stats
3from sklearn.linear_model import LinearRegression
4
5X = np.array([[1.0], [2.0], [3.0], [4.0], [5.0], [6.0]])
6y = np.array([1.9, 3.0, 4.2, 4.8, 6.1, 7.4])
7
8# Build the design matrix manually so the intercept is part of the math.
9X_design = np.column_stack([np.ones(len(X)), X[:, 0]])
10
11model = LinearRegression(fit_intercept=False)
12model.fit(X_design, y)
13
14y_hat = model.predict(X_design)
15residuals = y - y_hat
16
17n, p = X_design.shape
18degrees_of_freedom = n - p
19residual_variance = (residuals @ residuals) / degrees_of_freedom
20
21covariance_matrix = residual_variance * np.linalg.inv(X_design.T @ X_design)
22standard_errors = np.sqrt(np.diag(covariance_matrix))
23t_statistics = model.coef_ / standard_errors
24p_values = 2 * (1 - stats.t.cdf(np.abs(t_statistics), df=degrees_of_freedom))
25
26for name, coef, p_value in zip(["intercept", "x"], model.coef_, p_values):
27    print(f"{name}: coefficient={coef:.3f}, p-value={p_value:.4f}")

This works because LinearRegression is ordinary least squares. The important limitation is that the p-values are only as meaningful as the assumptions behind the model.

Interpret the Result Carefully

A small p-value suggests that the coefficient is unlikely to be zero under the null hypothesis, given the model assumptions. It does not mean the variable is important in every practical sense, and it does not prove causation.

Keep the usual conditions in mind:

• residuals should be roughly independent
• the relationship should be modeled correctly
• severe multicollinearity can destabilize standard errors
• heteroskedasticity can make the default p-values misleading

If your data violates those assumptions, consider robust standard errors or a different modeling strategy instead of trusting the first p-value you compute.

Common Pitfalls

• Forgetting the intercept term. If you compute p-values manually, your design matrix must match the model you actually fit.
• Using highly collinear features. The model may fit well while individual coefficient p-values become unstable and hard to interpret.
• Treating p-values as feature-selection truth. They are conditional on the chosen model and on the assumptions that justify the test.
• Mixing predictive and inferential goals. Cross-validation tells you how well the model predicts; p-values tell you something different.
• Applying the formula to regularized models such as Ridge or Lasso. The ordinary least squares standard error formula does not transfer directly.

Summary

• 'scikit-learn LinearRegression does not provide p-values by design.'
• Use statsmodels.OLS when statistical inference is the main goal.
• You can compute p-values manually from residual variance and the covariance matrix after fitting ordinary least squares.
• Include the intercept correctly, or your calculations will be wrong.
• Interpret significance only in the context of model assumptions, not as a standalone measure of importance.