Introduction

Measuring the accuracy of a KNN classifier in Python means evaluating predictions on data the model did not train on. The core metric is classification accuracy, but a sound evaluation usually also includes feature scaling, a held-out test set, and often cross-validation so the score is not just an accident of one split.

KNN is especially sensitive to preprocessing because it relies on distance. If you skip scaling or test on the training set, you may think you are measuring KNN accuracy when you are really measuring a flawed experiment.

Start with a Proper Train-Test Split

Accuracy is the fraction of correct predictions over the total number of predictions. In scikit-learn, the basic evaluation flow is:

1. split the dataset into training and test data
2. fit the KNN model on the training set
3. predict labels for the test set
4. compare predictions against the true labels

1from sklearn.datasets import load_iris
2from sklearn.metrics import accuracy_score
3from sklearn.model_selection import train_test_split
4from sklearn.neighbors import KNeighborsClassifier
5from sklearn.pipeline import make_pipeline
6from sklearn.preprocessing import StandardScaler
7
8X, y = load_iris(return_X_y=True)
9
10X_train, X_test, y_train, y_test = train_test_split(
11    X,
12    y,
13    test_size=0.2,
14    random_state=42,
15    stratify=y,
16)
17
18model = make_pipeline(
19    StandardScaler(),
20    KNeighborsClassifier(n_neighbors=5),
21)
22
23model.fit(X_train, y_train)
24y_pred = model.predict(X_test)
25
26print("Accuracy:", accuracy_score(y_test, y_pred))

This example uses StandardScaler inside a pipeline so the scaler is fit only on training data. That detail matters because fitting preprocessing on the full dataset leaks information from the test set into the model.

Look Beyond One Accuracy Number

Accuracy is easy to report, but it does not tell you which classes are being confused. A confusion matrix and classification report make the evaluation much more informative.

1from sklearn.metrics import classification_report, confusion_matrix
2
3print(confusion_matrix(y_test, y_pred))
4print(classification_report(y_test, y_pred))

If one class is underperforming, overall accuracy can still look fine. That is why accuracy alone is often too thin for any decision that matters.

This becomes even more important when classes are imbalanced. A model can achieve deceptively high accuracy by predicting the majority class too often, while doing poorly on the minority class you may care about most.

Use Cross-Validation for a More Stable Estimate

A single train-test split can be noisy. Cross-validation produces a more stable estimate by repeating the train-and-evaluate cycle across several folds.

1from sklearn.model_selection import cross_val_score
2
3scores = cross_val_score(
4    make_pipeline(
5        StandardScaler(),
6        KNeighborsClassifier(n_neighbors=5),
7    ),
8    X,
9    y,
10    cv=5,
11    scoring="accuracy",
12)
13
14print("Fold scores:", scores)
15print("Mean accuracy:", scores.mean())

When reporting model quality, a mean cross-validation score is usually more defensible than a single lucky split. It also helps you compare different KNN settings more honestly.

Tune the Number of Neighbors

There is nothing magical about k=5. Smaller values tend to fit local structure more aggressively and can become noisy. Larger values smooth the decision boundary and can underfit. The best k depends on the dataset.

1from sklearn.model_selection import cross_val_score
2
3for k in range(1, 16):
4    scores = cross_val_score(
5        make_pipeline(
6            StandardScaler(),
7            KNeighborsClassifier(n_neighbors=k),
8        ),
9        X,
10        y,
11        cv=5,
12        scoring="accuracy",
13    )
14    print(k, round(scores.mean(), 4))

This simple loop is often enough to find a sensible baseline. If you later move to grid search, the logic stays the same: evaluate under the same preprocessing and validation strategy you plan to trust.

Common Pitfalls

The biggest mistake is measuring accuracy on the same data used to train the classifier. That usually produces an inflated score and says little about how the model will behave on new examples.

Another common mistake is skipping feature scaling. Because KNN depends on distance, a feature with a larger numeric range can dominate the calculation and distort the result.

It is also easy to focus on overall accuracy and ignore class-level behavior. A good confusion matrix often reveals problems that a single scalar score hides.

Finally, do not treat the value of k as fixed. KNN performance changes meaningfully with the neighbor count, so accuracy should be measured after at least basic tuning.

Summary

• Measure KNN on held-out data, not the training set.
• Use a pipeline so scaling happens correctly and reproducibly.
• Report accuracy_score for the baseline metric.
• Add confusion matrices or classification reports when class-level errors matter.
• Use cross-validation and tune k before trusting the final score.