Introduction

In machine learning, model evaluation is a crucial step to ensure the robustness and effectiveness of predictive models. The Scikit-learn library offers numerous tools that simplify model evaluation. Specifically, GridSearchCV is a valuable technique for hyperparameter tuning, while also providing the ability to cross-validate the model. However, extracting multiple evaluation metrics such as Mean Squared Error (MSE) and R-squared (R²) from GridSearchCV can be tricky. This article will demonstrate how to achieve this, with technical explanations and examples.

Understanding Key Concepts

Mean Squared Error (MSE)

• Definition: MSE is a measure of the average squared difference between the estimated values (y_hat ) and the actual value (y ). • Formula: $$\text{MSE} = \frac{1}{n} \sum\_{i=1}^{n} (y\_i - \hat{y}\_i)^2$$

• Used For: Measuring the quality of an estimator; the lower the MSE, the better the estimation quality.

R-squared (R²)

• Definition: R² provides the proportion of the variance in the dependent variable that is predictable from the independent variables. • Formula: $$R^2 = 1 - \frac{\sum (y\_i - \hat{y}\_i)^2}{\sum (y\_i - \bar{y})^2}$$

• Used For: Determining the goodness-of-fit; R-squared values closer to 1 indicate a better fit.

Using GridSearchCV to Obtain MSE and R²

Step-by-step Guide

1. Importing Libraries

• Hyperparameter Tuning: GridSearchCV iterates over parameter values specified in param_grid to find the optimal combination. • Custom Scorers: make_scorer is used to create custom scoring metrics to calculate MSE and R². • Parallel Evaluation: By setting the scoring parameter as a dictionary and specifying refit='R2' , each fit is judged by both metrics but optimized for R². • Accessing Results: The cv_results_ attribute provides access to scores from cross-validation.