Introduction

RandomForestClassifier can work on imbalanced data, but its default behavior often favors the majority class because that is where most training examples live. To use it well on unbalanced classification problems, you usually need to adjust class weighting, evaluation metrics, and sometimes resampling strategy rather than relying on raw accuracy.

Why Imbalance Causes Trouble

Imagine a fraud dataset where 99 percent of examples are legitimate and 1 percent are fraud. A model that predicts "not fraud" for everything gets 99 percent accuracy and is still useless.

That is why imbalance changes both training and evaluation. The model sees fewer minority-class examples, and naive metrics can hide the failure.

Start with Class Weights

In scikit-learn, the simplest first step is usually class_weight. For random forests, useful built-in values include balanced and balanced_subsample.

1from sklearn.datasets import make_classification
2from sklearn.ensemble import RandomForestClassifier
3from sklearn.model_selection import train_test_split
4from sklearn.metrics import classification_report
5
6X, y = make_classification(
7    n_samples=5000,
8    n_features=20,
9    n_informative=8,
10    n_redundant=4,
11    weights=[0.95, 0.05],
12    random_state=42,
13)
14
15X_train, X_test, y_train, y_test = train_test_split(
16    X, y, test_size=0.2, stratify=y, random_state=42
17)
18
19model = RandomForestClassifier(
20    n_estimators=300,
21    class_weight="balanced",
22    random_state=42,
23    n_jobs=-1,
24)
25
26model.fit(X_train, y_train)
27pred = model.predict(X_test)
28print(classification_report(y_test, pred))

balanced computes weights inversely proportional to class frequency across the full training set. balanced_subsample recomputes that weighting inside each bootstrap sample, which can sometimes work better with bagged trees.

Use Better Metrics Than Accuracy

For imbalanced problems, metrics such as these are usually more informative:

• recall for the minority class
• precision for the minority class
• F1 score
• ROC AUC
• precision-recall AUC

Here is a small evaluation example:

1from sklearn.metrics import roc_auc_score, average_precision_score
2
3proba = model.predict_proba(X_test)[:, 1]
4
5print("ROC AUC:", roc_auc_score(y_test, proba))
6print("PR AUC:", average_precision_score(y_test, proba))

Precision-recall metrics are especially useful when the positive class is rare.

Threshold Tuning Often Matters More Than the Forest Itself

By default, predict() uses a probability threshold of 0.5. For imbalanced data, that may not match your business objective.

If the goal is to catch more minority-class examples, you may want a lower threshold.

1import numpy as np
2from sklearn.metrics import precision_score, recall_score
3
4threshold = 0.25
5custom_pred = (proba >= threshold).astype(int)
6
7print("precision:", precision_score(y_test, custom_pred))
8print("recall:", recall_score(y_test, custom_pred))

This is often where real model behavior changes most. The forest may already rank examples reasonably well, but the default threshold may be too conservative.

Resampling Can Help, but Use It Carefully

Class weighting is not the only option. You can also try:

• undersampling the majority class
• oversampling the minority class
• synthetic methods such as SMOTE

If you do resampling, it must happen only on the training data inside the validation loop, never before the train-test split. Otherwise you leak information and overestimate performance.

A simple training-only undersampling example:

1from sklearn.utils import resample
2import numpy as np
3
4minority = X_train[y_train == 1]
5majority = X_train[y_train == 0]
6
7majority_downsampled = resample(
8    majority,
9    replace=False,
10    n_samples=len(minority) * 3,
11    random_state=42,
12)
13
14X_balanced = np.vstack([majority_downsampled, minority])
15y_balanced = np.array([0] * len(majority_downsampled) + [1] * len(minority))

This is only one option, but it shows the principle.

Hyperparameters Still Matter

Even with class weighting, forest configuration still affects minority-class behavior. Parameters worth watching include:

• 'n_estimators'
• 'max_depth'
• 'min_samples_leaf'
• 'max_features'

For imbalance, a slightly larger min_samples_leaf can sometimes reduce unstable minority predictions caused by tiny terminal leaves. There is no universal rule, but tuning should include imbalance-aware metrics.

A Practical Workflow

A solid workflow is usually:

1. stratify the split
2. train a weighted baseline forest
3. evaluate with recall, precision, F1, and PR AUC
4. tune the decision threshold
5. only then compare resampling strategies

This is better than jumping directly to aggressive oversampling before you know how far the baseline already gets you.

Common Pitfalls

A common mistake is reporting accuracy as the main metric on a heavily imbalanced dataset. That often hides a model that ignores the minority class.

Another issue is resampling before the train-test split, which leaks information into evaluation.

Developers also sometimes use class_weight and assume the problem is solved automatically. It helps, but it does not eliminate the need for threshold tuning and appropriate metrics.

Finally, be careful with probability interpretation. Random-forest probabilities are often useful for ranking, but they are not always perfectly calibrated.

Summary

• 'RandomForestClassifier can handle imbalance better with class_weight.'
• Use recall, precision, F1, and PR AUC instead of raw accuracy.
• Tune the prediction threshold for your actual business objective.
• Apply resampling only inside the training workflow, never before the split.
• Start with a weighted baseline before adding more complex imbalance strategies.