Introduction

In scikit-learn, a Pipeline is for transformations followed by a final estimator. A metric is not a transformation step and it does not belong “after” the classifier inside the pipeline itself. Instead, metrics are applied during evaluation through score, cross_val_score, GridSearchCV, or a custom scorer.

Why Metrics Are Not Pipeline Steps

A scikit-learn pipeline expects each intermediate step to implement fit and transform, and the final step to implement fit plus prediction behavior. Metrics do not transform data for later steps. They compare predictions with ground truth after the model has been fit.

So this is conceptually wrong:

• scaler
• vectorizer
• classifier
• metric

The metric should operate on model outputs, not as part of the training data flow.

Build the Pipeline First

A normal classifier pipeline looks like this:

1from sklearn.pipeline import Pipeline
2from sklearn.preprocessing import StandardScaler
3from sklearn.linear_model import LogisticRegression
4
5pipe = Pipeline([
6    ("scale", StandardScaler()),
7    ("clf", LogisticRegression(max_iter=1000)),
8])

This is the training and prediction pipeline. Evaluation comes afterward.

Evaluate with a Metric Function

The simplest pattern is:

1. fit the pipeline
2. predict
3. apply the metric

1from sklearn.datasets import load_iris
2from sklearn.model_selection import train_test_split
3from sklearn.metrics import accuracy_score
4
5X, y = load_iris(return_X_y=True)
6X_train, X_test, y_train, y_test = train_test_split(
7    X, y, test_size=0.2, random_state=42
8)
9
10pipe.fit(X_train, y_train)
11y_pred = pipe.predict(X_test)
12
13print(accuracy_score(y_test, y_pred))

That is often all you need for a direct train-test evaluation.

Use Cross-Validation with a Scoring Metric

For more reliable evaluation, pass a metric name or scorer into cross-validation.

1from sklearn.model_selection import cross_val_score
2
3scores = cross_val_score(pipe, X, y, cv=5, scoring="accuracy")
4print(scores)
5print(scores.mean())

This is the standard way to apply metrics “with” a pipeline in scikit-learn.

You can swap in other metrics such as:

• 'f1_macro'
• 'precision_macro'
• 'recall_macro'
• 'roc_auc, when appropriate'

The pipeline stays unchanged. Only the evaluation call changes.

Use a Custom Metric with make_scorer

If you need a custom metric function, convert it into a scorer.

1from sklearn.metrics import fbeta_score, make_scorer
2from sklearn.model_selection import cross_val_score
3
4fbeta = make_scorer(fbeta_score, beta=2, average="macro")
5scores = cross_val_score(pipe, X, y, cv=5, scoring=fbeta)
6
7print(scores.mean())

This is how you integrate non-default metric logic into model selection without changing the pipeline structure.

Use Metrics in Grid Search

Metrics are especially important during hyperparameter tuning.

1from sklearn.model_selection import GridSearchCV
2
3param_grid = {
4    "clf__C": [0.1, 1.0, 10.0]
5}
6
7search = GridSearchCV(
8    pipe,
9    param_grid=param_grid,
10    scoring="f1_macro",
11    cv=5
12)
13
14search.fit(X, y)
15print(search.best_params_)
16print(search.best_score_)

The metric tells grid search how to compare candidate models. It still is not part of the pipeline graph itself.

When score Is Enough

Many estimators implement a default score method. For classifiers, that is often accuracy.

pipe.fit(X_train, y_train)
print(pipe.score(X_test, y_test))

This is convenient, but it can be too implicit. If the problem is imbalanced or business-critical, choose the metric explicitly rather than relying on default accuracy.

Keep Evaluation Separate from Training Logic

A good mental model is:

• pipeline handles preprocessing plus estimator
• scorer handles evaluation criteria

That separation keeps experiments easier to reason about and avoids trying to force metrics into the wrong abstraction.

Common Pitfalls

The biggest mistake is trying to add a metric as a literal step after the classifier in the pipeline. Scikit-learn pipelines are not designed for that.

Another issue is relying on default score when the real problem needs recall, F1, or ROC-AUC instead of raw accuracy.

Developers also sometimes create custom metrics but forget to wrap them with make_scorer, which means model-selection tools cannot use them correctly.

Summary

• Metrics are not pipeline steps in scikit-learn.
• Fit the pipeline, then evaluate predictions with metrics.
• Use cross_val_score or GridSearchCV with scoring for model comparison.
• Use make_scorer for custom metric functions.
• Keep preprocessing and evaluation as separate concerns.