Practice Now
sklearn
GridSearchCV
machine learning
model tuning
python

Is there anyway to know the progress in sklearn GridSearch

ML System Design practice on Codemia

Design recommenders, ranking systems and training pipelines the way ML interviews actually ask for them, with worked solutions.

Practice ML system design
Machine LearningPython

Introduction

Yes. Set verbose=2 or verbose=3 in GridSearchCV to print progress during the search. For more control, use tqdm with a custom scorer or callback. Since scikit-learn 0.24, you can also use joblib callbacks to track progress of parallel jobs. The simplest approach is GridSearchCV(..., verbose=2), which prints the parameter combination, score, and timing for each fit.

Built-in verbose Parameter

python
1from sklearn.model_selection import GridSearchCV
2from sklearn.svm import SVC
3from sklearn.datasets import load_iris
4
5X, y = load_iris(return_X_y=True)
6
7param_grid = {
8    'C': [0.1, 1, 10, 100],
9    'kernel': ['rbf', 'linear'],
10    'gamma': ['scale', 'auto']
11}
12
13# verbose=1: shows number of fits
14# verbose=2: shows each fit with score and timing
15# verbose=3: shows fold details
16grid = GridSearchCV(SVC(), param_grid, cv=5, verbose=2, n_jobs=-1)
17grid.fit(X, y)
18
19# Output:
20# Fitting 5 folds for each of 16 candidates, totalling 80 fits
21# [CV] END C=0.1, gamma=scale, kernel=rbf; total time=   0.0s
22# [CV] END C=0.1, gamma=scale, kernel=linear; total time=   0.0s
23# ...

Using tqdm for a Progress Bar

python
1from sklearn.model_selection import GridSearchCV
2from sklearn.svm import SVC
3from sklearn.datasets import make_classification
4from tqdm import tqdm
5import numpy as np
6
7X, y = make_classification(n_samples=1000, n_features=20, random_state=42)
8
9param_grid = {
10    'C': [0.01, 0.1, 1, 10, 100],
11    'kernel': ['rbf', 'linear', 'poly'],
12    'gamma': ['scale', 'auto', 0.01, 0.1]
13}
14
15# Calculate total fits
16n_candidates = np.prod([len(v) for v in param_grid.values()])
17n_folds = 5
18total_fits = n_candidates * n_folds
19print(f"Total fits: {total_fits}")  # 300
20
21# Custom scorer with tqdm progress bar
22from sklearn.metrics import accuracy_score, make_scorer
23
24pbar = tqdm(total=total_fits, desc="GridSearchCV")
25
26def scoring_with_progress(estimator, X, y):
27    pbar.update(1)
28    return accuracy_score(y, estimator.predict(X))
29
30grid = GridSearchCV(
31    SVC(), param_grid, cv=5,
32    scoring=scoring_with_progress,
33    n_jobs=1  # Must be 1 for tqdm to work correctly
34)
35grid.fit(X, y)
36pbar.close()
37
38print(f"Best: {grid.best_score_:.4f} with {grid.best_params_}")

Using joblib Callbacks (scikit-learn 0.24+)

python
1from sklearn.model_selection import GridSearchCV
2from sklearn.ensemble import RandomForestClassifier
3import joblib
4from tqdm.auto import tqdm
5
6class TqdmCallback(joblib.parallel.BatchCompletionCallBack):
7    def __init__(self, *args, **kwargs):
8        super().__init__(*args, **kwargs)
9
10    def __call__(self, *args, **kwargs):
11        self.tqdm_bar.update(self.batch_size)
12        return super().__call__(*args, **kwargs)
13
14param_grid = {
15    'n_estimators': [50, 100, 200],
16    'max_depth': [5, 10, 20, None],
17    'min_samples_split': [2, 5, 10]
18}
19
20n_candidates = 3 * 4 * 3  # 36
21n_folds = 5
22total = n_candidates * n_folds
23
24grid = GridSearchCV(
25    RandomForestClassifier(random_state=42),
26    param_grid, cv=5, n_jobs=-1
27)
28
29with tqdm(total=total, desc="GridSearchCV") as pbar:
30    old_callback = joblib.parallel.BatchCompletionCallBack
31    TqdmCallback.tqdm_bar = pbar
32    joblib.parallel.BatchCompletionCallBack = TqdmCallback
33    try:
34        grid.fit(X, y)
35    finally:
36        joblib.parallel.BatchCompletionCallBack = old_callback
37
38print(f"Best: {grid.best_score_:.4f}")

Estimating Time Remaining

python
1import time
2from sklearn.model_selection import ParameterGrid
3
4param_grid = {
5    'C': [0.1, 1, 10],
6    'kernel': ['rbf', 'linear']
7}
8
9# Calculate search space size
10grid_size = len(list(ParameterGrid(param_grid)))
11print(f"Parameter combinations: {grid_size}")  # 6
12print(f"Total fits (5 folds): {grid_size * 5}")  # 30
13
14# Time a single fit to estimate total time
15from sklearn.svm import SVC
16from sklearn.model_selection import cross_val_score
17
18start = time.time()
19scores = cross_val_score(SVC(C=1, kernel='rbf'), X, y, cv=5)
20single_fit_time = (time.time() - start) / 5  # Average per fit
21
22estimated_total = single_fit_time * grid_size * 5
23print(f"Estimated total time: {estimated_total:.1f}s")
python
1# After fit, inspect cv_results_ for detailed timing
2import pandas as pd
3
4grid.fit(X, y)
5
6results = pd.DataFrame(grid.cv_results_)
7print(results[['params', 'mean_test_score', 'std_test_score', 'mean_fit_time']].to_string())
8
9# Sort by score
10top_results = results.nsmallest(5, 'rank_test_score')[
11    ['params', 'mean_test_score', 'mean_fit_time']
12]
13print(top_results)

Alternative: Optuna for Built-in Progress

python
1# Optuna has built-in progress tracking
2import optuna
3from sklearn.svm import SVC
4from sklearn.model_selection import cross_val_score
5
6def objective(trial):
7    C = trial.suggest_float('C', 0.01, 100, log=True)
8    kernel = trial.suggest_categorical('kernel', ['rbf', 'linear'])
9    gamma = trial.suggest_categorical('gamma', ['scale', 'auto'])
10
11    clf = SVC(C=C, kernel=kernel, gamma=gamma)
12    score = cross_val_score(clf, X, y, cv=5).mean()
13    return score
14
15# Built-in progress bar and logging
16study = optuna.create_study(direction='maximize')
17study.optimize(objective, n_trials=50, show_progress_bar=True)
18
19print(f"Best: {study.best_value:.4f} with {study.best_params}")

Common Pitfalls

  • Using n_jobs=-1 with tqdm custom scorer: When n_jobs > 1, multiple workers call the scorer in parallel. tqdm is not thread-safe by default, causing garbled output or inaccurate counts. Either set n_jobs=1 or use tqdm's thread-safe variant with locks.
  • Underestimating search space size: The total number of fits is n_candidates * n_folds. With 5 parameters each having 5 values and 5-fold CV, that is 5^5 * 5 = 15,625 fits. Use len(ParameterGrid(param_grid)) to check before running, and consider RandomizedSearchCV for large spaces.
  • verbose output overwhelming the terminal: verbose=3 on a search with thousands of fits produces massive output. Use verbose=1 for a summary or redirect output to a file. Better yet, use tqdm for a clean single-line progress bar.
  • Not using refit=True (default) for the best model: After GridSearchCV completes, grid.best_estimator_ is available only if refit=True. If you set refit=False for speed during exploration, you must manually retrain the best model.
  • Ignoring mean_fit_time in results: cv_results_['mean_fit_time'] shows how long each parameter combination takes. Checking this helps identify expensive combinations and decide whether to narrow the search space or increase parallelism.

Summary

  • Use verbose=2 for basic progress output showing each fit with timing and score
  • Use tqdm with a custom scorer for a clean progress bar (set n_jobs=1)
  • Check len(ParameterGrid(param_grid)) * n_folds to know total fits before starting
  • Use cv_results_ after fitting to analyze scores, timing, and parameter performance
  • Consider Optuna or RandomizedSearchCV for large search spaces with built-in progress tracking
Related reading
Free course
Beginner
7 lessons
2 hours
Tackling System Design Interview Problems

A short course that equips you with the skills to approach system design interviews methodically.

Start the free course
Track what you have practised

A free account saves your progress, solutions and study plan across every problem on Codemia.

ML System Design practice on Codemia

Design recommenders, ranking systems and training pipelines the way ML interviews actually ask for them, with worked solutions.

Practice ML system design
Codemia
© 2026 Codemia
Resources
Legal
Social
All Rights Reserved.