Introduction

DecisionTreeClassifier can work on imbalanced data, but it will often lean toward the majority class unless you help it. The usual fixes are class weighting, better evaluation metrics, and sometimes resampling the training data before fitting the tree.

Start with the Real Problem

Imbalance means accuracy becomes a misleading metric. If 95 percent of examples belong to one class, a model can look "accurate" while failing on the minority class almost every time.

So balancing classification is not only about changing the model. It is also about measuring the right things:

• precision
• recall
• F1 score
• confusion matrix
• ROC AUC, when appropriate

If you tune only for accuracy, you can convince yourself the tree is good when it is actually useless for the rare class.

Use class_weight

The simplest built-in tool in scikit-learn is the class_weight parameter. Setting it to "balanced" adjusts class weights inversely to class frequencies, which makes minority-class mistakes more expensive during training.

1from sklearn.datasets import make_classification
2from sklearn.metrics import classification_report
3from sklearn.model_selection import train_test_split
4from sklearn.tree import DecisionTreeClassifier
5
6X, y = make_classification(
7    n_samples=2000,
8    n_features=12,
9    n_informative=6,
10    n_redundant=2,
11    weights=[0.92, 0.08],
12    random_state=42,
13)
14
15X_train, X_test, y_train, y_test = train_test_split(
16    X, y, test_size=0.25, stratify=y, random_state=42
17)
18
19model = DecisionTreeClassifier(
20    class_weight="balanced",
21    max_depth=5,
22    min_samples_leaf=10,
23    random_state=42,
24)
25
26model.fit(X_train, y_train)
27predictions = model.predict(X_test)
28
29print(classification_report(y_test, predictions))

This is usually the first thing to try because it is simple, reproducible, and does not require changing the dataset itself.

Why Stratified Splits Matter

Always split imbalanced data with stratification unless you have a strong reason not to. Without stratification, one split can accidentally contain very few minority examples, which makes training and evaluation noisy.

That is why the previous example used:

train_test_split(..., stratify=y)

Balancing the classifier starts with balancing the experimental setup.

Resampling Can Help

Sometimes class weights are not enough, especially when the minority class is extremely rare. In those cases, resampling the training set can help:

• oversampling the minority class
• undersampling the majority class
• synthetic methods such as SMOTE

Here is a simple oversampling example without extra libraries:

1import numpy as np
2from sklearn.utils import resample
3
4minority_X = X_train[y_train == 1]
5minority_y = y_train[y_train == 1]
6majority_X = X_train[y_train == 0]
7majority_y = y_train[y_train == 0]
8
9upsampled_X, upsampled_y = resample(
10    minority_X,
11    minority_y,
12    replace=True,
13    n_samples=len(majority_y),
14    random_state=42,
15)
16
17X_balanced = np.vstack([majority_X, upsampled_X])
18y_balanced = np.concatenate([majority_y, upsampled_y])
19
20tree = DecisionTreeClassifier(max_depth=5, random_state=42)
21tree.fit(X_balanced, y_balanced)

Only resample the training set. Do not rebalance the test set, or your evaluation will stop reflecting reality.

Trees Still Need Regularization

A tree trained on oversampled or weighted data can overfit badly if you let it grow without constraints. Parameters such as these are often more important on imbalanced problems than people expect:

• 'max_depth'
• 'min_samples_leaf'
• 'min_samples_split'
• 'ccp_alpha'

Balancing does not remove the need for model control. In fact, it often increases it.

A Reasonable Workflow

For a plain DecisionTreeClassifier, a pragmatic workflow is:

1. use a stratified split
2. establish a baseline without balancing
3. try class_weight="balanced"
4. compare recall, precision, and F1 for the minority class
5. if results are still poor, try resampling on the training set
6. regularize the tree to avoid memorizing duplicated minority points

This gives you a path from simplest intervention to more invasive ones.

Common Pitfalls

Using raw accuracy as the main metric hides minority-class failure and makes the model look better than it is.

Resampling before the train-test split leaks information and invalidates the evaluation.

Balancing the training data but leaving the tree unconstrained often produces an overfit model with unstable predictions.

Assuming class_weight="balanced" is always enough can be optimistic when the class skew is severe or the minority class is noisy.

Summary

• Start by measuring the right metrics, not just accuracy.
• Use class_weight="balanced" as the simplest built-in fix for DecisionTreeClassifier.
• Keep train-test splits stratified.
• If needed, resample only the training set and then regularize the tree.
• Evaluate improvements on minority-class precision, recall, and F1 instead of trusting headline accuracy.