Introduction

If you want to control how hard a binary classification problem is, the key variable is class overlap. The closer the two classes sit in feature space, or the more label noise you inject, the more ambiguous the decision boundary becomes.

What "Overlap" Really Means

Two classes overlap when points from class 0 and class 1 occupy the same region of feature space. In practice, overlap comes from a few knobs:

• the distance between class centers
• the spread of each class
• the number of clusters per class
• label noise, meaning some points are intentionally assigned the opposite label

If you only want a quick synthetic dataset, sklearn.datasets.make_classification already exposes several of these controls.

Using make_classification

The most direct parameter is class_sep. Larger values move the classes farther apart. Smaller values make the boundary fuzzier. You can also use flip_y to inject label noise.

This example generates two datasets with different overlap levels and plots them:

1import matplotlib.pyplot as plt
2from sklearn.datasets import make_classification
3
4
5def build_dataset(class_sep, flip_y):
6    X, y = make_classification(
7        n_samples=400,
8        n_features=2,
9        n_informative=2,
10        n_redundant=0,
11        n_repeated=0,
12        n_classes=2,
13        n_clusters_per_class=1,
14        class_sep=class_sep,
15        flip_y=flip_y,
16        random_state=42,
17    )
18    return X, y
19
20
21easy_X, easy_y = build_dataset(class_sep=2.0, flip_y=0.0)
22hard_X, hard_y = build_dataset(class_sep=0.6, flip_y=0.08)
23
24fig, axes = plt.subplots(1, 2, figsize=(10, 4))
25
26axes[0].scatter(easy_X[:, 0], easy_X[:, 1], c=easy_y, cmap="coolwarm", s=18)
27axes[0].set_title("Lower overlap")
28
29axes[1].scatter(hard_X[:, 0], hard_X[:, 1], c=hard_y, cmap="coolwarm", s=18)
30axes[1].set_title("Higher overlap")
31
32plt.tight_layout()
33plt.show()

That is usually enough when you want to benchmark models at several difficulty levels.

When You Need Finer Control

make_classification is convenient, but it does not give you an explicit geometric model of each class. If you want exact control over where each class lives, generate the classes yourself from probability distributions.

A common pattern is to sample each class from a Gaussian distribution with its own mean and covariance matrix:

1import numpy as np
2import matplotlib.pyplot as plt
3
4rng = np.random.default_rng(7)
5
6mean_a = np.array([0.0, 0.0])
7cov_a = np.array([[1.0, 0.4], [0.4, 1.0]])
8
9mean_b = np.array([1.2, 1.0])
10cov_b = np.array([[1.0, -0.2], [-0.2, 1.0]])
11
12class_a = rng.multivariate_normal(mean_a, cov_a, size=200)
13class_b = rng.multivariate_normal(mean_b, cov_b, size=200)
14
15X = np.vstack([class_a, class_b])
16y = np.array([0] * len(class_a) + [1] * len(class_b))
17
18plt.scatter(class_a[:, 0], class_a[:, 1], label="class 0", alpha=0.7)
19plt.scatter(class_b[:, 0], class_b[:, 1], label="class 1", alpha=0.7)
20plt.legend()
21plt.show()

To increase overlap, move mean_b closer to mean_a or increase the variances in the covariance matrices. To reduce overlap, move the means farther apart or tighten the covariance.

Choosing the Right Control Knob

Different knobs produce different kinds of difficulty.

• 'class_sep mostly changes how far apart the classes are.'
• 'flip_y adds mislabeled samples, which simulates annotation noise.'
• Larger covariance values create wider clouds and more geometric overlap.
• Multiple clusters per class create nontrivial boundaries even when overall separation looks large.

That matters because not all overlap is the same. A linear model may struggle with intertwined clusters for a different reason than it struggles with noisy labels.

Measure, Do Not Guess

A useful habit is to generate several datasets and compare both a plot and a baseline model score. For example, if logistic regression still gets near-perfect accuracy, your overlap is probably too low for the experiment you want.

Synthetic data is most valuable when the difficulty setting is intentional instead of aesthetic.

Common Pitfalls

• Assuming overlap is controlled only by class means. Variance and label noise matter just as much.
• Using high-dimensional data and then judging overlap only from a two-dimensional projection.
• Forgetting that imbalanced classes and overlapping classes are different problems.
• Making the data unrealistically clean, which can produce misleadingly strong benchmark results.
• Using flip_y to simulate geometric overlap. It simulates noisy labels, which is related but not identical.

Summary

• Class overlap controls how difficult a binary classification task is.
• 'make_classification is the quickest way to adjust overlap with class_sep and flip_y.'
• Custom Gaussian sampling gives finer control over means, covariance, and cluster shape.
• Different overlap mechanisms stress models in different ways.
• Plot the data and test a baseline model so your synthetic dataset matches the difficulty you intend.