GridSearchCV
scikit-learn
hyperparameter tuning
mean_test_score
Python

what is Gridsearch.cv_results_ , could any explain all the things in that i.e mean_test_score etc .?

Master System Design with Codemia

Enhance your system design skills with over 120 practice problems, detailed solutions, and hands-on exercises.

Introduction

GridSearchCV.cv_results_ is the full score table produced by a grid search. It stores one row of results for every parameter combination that was tried, along with per-fold scores, averages, rankings, and timing information.

If best_params_ tells you which candidate won, cv_results_ tells you why it won and what happened to every other candidate. Once you read it as a table instead of a mysterious dictionary, the field names make sense.

What cv_results_ Looks Like

cv_results_ is a dictionary of arrays. Each position across the arrays refers to one candidate parameter setting.

Here is a complete example:

python
1import pandas as pd
2from sklearn import datasets
3from sklearn.model_selection import GridSearchCV
4from sklearn.svm import SVC
5
6iris = datasets.load_iris()
7
8search = GridSearchCV(
9    estimator=SVC(),
10    param_grid={
11        "kernel": ["linear", "rbf"],
12        "C": [0.1, 1, 10],
13    },
14    cv=3,
15    return_train_score=True,
16)
17
18search.fit(iris.data, iris.target)
19
20results = pd.DataFrame(search.cv_results_)
21print(results[[
22    "params",
23    "mean_test_score",
24    "std_test_score",
25    "rank_test_score",
26    "mean_fit_time",
27]])

Turning the dictionary into a DataFrame is often the easiest way to inspect it.

Meaning of the Most Important Fields

These keys are the ones you will use most often:

  • 'params: the actual parameter dictionary for that candidate'
  • 'param_C, param_kernel, and other param_* keys: the same values split into separate columns'
  • 'split0_test_score, split1_test_score, and so on: score for each individual validation fold'
  • 'mean_test_score: average validation score across all folds'
  • 'std_test_score: variability of the validation score across folds'
  • 'rank_test_score: ranking of candidates, where 1 is best'
  • 'mean_fit_time: average training time in seconds'
  • 'mean_score_time: average scoring time in seconds'

If you set return_train_score=True, you also get:

  • 'split0_train_score, split1_train_score, and related fields'
  • 'mean_train_score'
  • 'std_train_score'

These training fields are useful when you want to compare training and validation performance for signs of overfitting.

How to Read mean_test_score

mean_test_score is usually the most important column. It is the average score on held-out validation folds, not on the training data.

For example, with cv=3, each candidate is trained three times. Each training run uses a different split of the dataset, and each run produces one test score. mean_test_score is just the average of those three values.

If you use accuracy as the scorer, then mean_test_score is mean validation accuracy. If you use scoring="neg_mean_squared_error", then mean_test_score is the mean of that metric instead. The field name stays the same; only the scoring function changes.

Connecting cv_results_ to the Best Model

The index of the winning row is stored in best_index_. That lets you connect the summary attributes back to the table:

python
1best_row = search.cv_results_["params"][search.best_index_]
2best_score = search.cv_results_["mean_test_score"][search.best_index_]
3
4print(best_row)
5print(best_score)
6print(search.best_params_)
7print(search.best_score_)

This is helpful because best_params_ alone does not show the spread of the rest of the search space. Looking at nearby candidates often reveals that several settings performed almost the same, which can matter when you prefer a simpler or faster model.

Special Cases Worth Knowing

With multi-metric scoring, the names change slightly. Instead of mean_test_score, you get keys such as mean_test_precision or mean_test_f1, depending on the scorer names you provided.

Also note that some param_* columns may be masked arrays when a parameter only applies to some candidates. That is normal. For example, degree may be meaningful for a polynomial kernel but irrelevant for an RBF kernel.

Common Pitfalls

  • Reading mean_test_score as training performance. It is validation performance.
  • Ignoring std_test_score. A slightly lower mean with lower variance can be the safer choice.
  • Looking only at best_params_ and never inspecting the rest of the grid.
  • Forgetting that the meaning of "score" depends entirely on the scoring argument.
  • Using training scores to choose the model. Model selection should be based on validation scores, not mean_train_score.

Summary

  • 'cv_results_ is the complete result table for every parameter candidate tried by GridSearchCV.'
  • Each row corresponds to one parameter combination.
  • 'mean_test_score is the average validation score across folds.'
  • 'rank_test_score tells you the ordering, and best_index_ points to the winning row.'
  • Convert cv_results_ to a DataFrame when you want to analyze results instead of only reading the best settings.

Course illustration
Course illustration

All Rights Reserved.