Introduction

In scikit-learn, a bagging ensemble does not magically create feature importances unless the base estimator itself knows how to measure them. If you use BaggingClassifier or BaggingRegressor with decision trees, you can usually derive ensemble-level importances by averaging the feature importances reported by the individual fitted trees.

That distinction matters because the bagging meta-estimator is just orchestrating many base models trained on bootstrap samples. The importance signal still comes from the base estimator, not from bagging as an abstract concept.

Start with a Tree-Based Base Estimator

A common setup is bagging over decision trees:

1import numpy as np
2from sklearn.datasets import make_classification
3from sklearn.ensemble import BaggingClassifier
4from sklearn.tree import DecisionTreeClassifier
5
6X, y = make_classification(n_samples=500, n_features=6, random_state=42)
7
8model = BaggingClassifier(
9    estimator=DecisionTreeClassifier(random_state=42),
10    n_estimators=50,
11    random_state=42
12)
13model.fit(X, y)

Each fitted tree inside the ensemble has its own feature_importances_ attribute because decision trees compute impurity-based importance.

Average Importances Across Estimators

Once the bagging model is trained, you can aggregate the importances from the fitted base estimators:

1import numpy as np
2
3importances = np.array([
4    estimator.feature_importances_
5    for estimator in model.estimators_
6])
7
8mean_importance = importances.mean(axis=0)
9print(mean_importance)

This produces one importance value per input feature by averaging across the ensemble.

You can also inspect variability across the trees:

std_importance = importances.std(axis=0)
print(std_importance)

That is useful because one feature may look important on average but unstable across bootstrap samples.

Why There Is No Universal Bagging Importance

Bagging itself does not define one universal feature-importance formula. The ensemble simply resamples data and trains many estimators. So the right question is really:

• does the base estimator expose feature importance
• if so, how should I aggregate those per-estimator importances

If the base estimator is a decision tree, averaging is a reasonable and common approach. If the base estimator is something like KNeighborsClassifier, there may be no built-in feature-importance signal to aggregate at all.

Consider Permutation Importance Too

Impurity-based tree importances are convenient, but they are not the only option. Model-agnostic permutation importance can be computed for the entire fitted bagging model:

1from sklearn.inspection import permutation_importance
2
3result = permutation_importance(model, X, y, n_repeats=10, random_state=42)
4print(result.importances_mean)

This has two advantages:

• it works at the full-model level
• it does not depend on tree impurity formulas

Permutation importance is often more directly tied to predictive performance, although it can be slower to compute.

Be Careful Interpreting the Numbers

Feature importance is not the same as causality. Correlated features, data leakage, and bootstrap variation can all distort interpretation.

Bagging can reduce variance in model predictions, but it does not automatically make importance estimates easy to explain. If two features carry similar information, importance may be split between them or shift from one tree to another.

That is why many practitioners report both the mean importance and some variability measure rather than only a single ranking.

Common Pitfalls

• Expecting BaggingClassifier or BaggingRegressor to provide meaningful feature importances when the base estimator does not support them.
• Averaging per-tree importances without checking how stable they are across estimators.
• Treating impurity-based importance as a perfect measure of real-world explanatory power.
• Forgetting that permutation importance can be a better full-model alternative in some cases.

Summary

• In bagging, feature importance comes from the base estimators, not from bagging by itself.
• With tree-based estimators, a common ensemble importance is the mean of each estimator's feature_importances_.
• Permutation importance is a useful model-agnostic alternative for the full fitted ensemble.
• Interpret importance values carefully, especially with correlated features.
• Always check whether the chosen base estimator actually supports importance extraction.