Does SimpleImputer remove features?
ML System Design practice on Codemia
Design recommenders, ranking systems and training pipelines the way ML interviews actually ask for them, with worked solutions.
Introduction
SimpleImputer is mainly an imputation tool, not a feature-selection tool. Its usual job is to fill in missing values, but there is one important exception: columns that are entirely missing at fit time may be dropped unless you configure the transformer to keep them.
What SimpleImputer Normally Does
In the common case, SimpleImputer keeps the same features and replaces missing entries with a statistic such as the mean, median, most frequent value, or a constant. For a partially missing column, the column remains in the output after transformation.
In that example, the second column is not removed. The missing value is filled with the column mean, and the output shape still matches the original two-feature input.
When Features Can Be Removed
The edge case is a feature that contains only missing values at fit time. According to the scikit-learn documentation, those fully empty columns are dropped by default for non-constant strategies because there is no usable statistic to compute.
The first column disappears because it is entirely missing during fit. That surprises many people because they expected imputation to preserve the original feature count automatically.
How to Keep the Same Number of Features
If your pipeline depends on a fixed shape, set keep_empty_features=True. Then SimpleImputer keeps the empty columns and fills them with a fallback value. With strategy="constant", it uses your fill_value. Otherwise, empty features are typically filled with zero.
This is often the right choice when SimpleImputer sits inside a larger pipeline and downstream code expects the same feature positions every time.
SimpleImputer Is Not Feature Selection
It is important to separate these ideas:
- Imputation fills missing values.
- Feature selection intentionally removes columns based on usefulness.
If you want to remove low-value features, use tools such as VarianceThreshold, model-based feature selection, or manual column filtering. SimpleImputer only drops features as a side effect when it cannot compute a statistic for a column, not because it judged the feature unimportant.
add_indicator Can Increase Features
There is another shape-related detail worth knowing. If you set add_indicator=True, scikit-learn appends missing-value indicator columns to the transformed output. That means the output can have more features than the input.
So SimpleImputer can either keep the same shape, drop fully empty columns, or add indicator features depending on configuration. The exact result depends on the data and parameters you choose.
Common Pitfalls
The biggest pitfall is assuming that imputation always preserves column count. That is usually true, but not when a column is completely empty during fitting and keep_empty_features is left at its default value.
Another mistake is fitting the imputer outside a pipeline and then being surprised when train and test transformations do not align. Putting the imputer inside a scikit-learn Pipeline makes the behavior easier to manage.
Be careful with strategy choice as well. mean and median only work for numeric data, while text columns generally need most_frequent or constant.
Finally, do not mistake dropped empty columns for intentional feature selection. If feature removal is part of your modeling plan, handle it explicitly with the right transformer.
Summary
- '
SimpleImputernormally fills missing values and keeps existing features.' - Fully empty columns can be dropped at transform time if
keep_empty_features=False. - Use
keep_empty_features=Truewhen your pipeline requires a fixed feature count. - '
add_indicator=Truecan increase the number of output features by appending missingness flags.' - Use dedicated feature-selection tools when you actually want to remove columns on purpose.
Related reading
- Does TensorBoard TensorFlow have the features to add labels for axes and legends on plots? If so, how?
- Does the dataset size influence a machine learning algorithm?
- Download link for Ta Feng Grocery dataset
- Drawing decision boundaries in R
- Drop all duplicate rows across multiple columns in Python Pandas
- Drop columns whose name contains a specific string from pandas DataFrame
- Dropping infinite values from dataframes in pandas?
- Dummy variables when not all categories are present
.png&w=3840&q=75)
Tackling System Design Interview Problems
A short course that equips you with the skills to approach system design interviews methodically.
Start the free courseTrack what you have practised
A free account saves your progress, solutions and study plan across every problem on Codemia.
ML System Design practice on Codemia
Design recommenders, ranking systems and training pipelines the way ML interviews actually ask for them, with worked solutions.