How do I find numeric columns in Pandas?
ML System Design practice on Codemia
Design recommenders, ranking systems and training pipelines the way ML interviews actually ask for them, with worked solutions.
Introduction
Finding numeric columns in a pandas DataFrame is essential for data preprocessing, statistical analysis, and machine learning pipelines. The primary method is df.select_dtypes(include='number'), which returns a DataFrame containing only numeric columns (int, float, complex). For just the column names, use df.select_dtypes(include='number').columns. Other approaches include pd.api.types.is_numeric_dtype() for checking individual columns and df.describe() which only summarizes numeric columns by default.
Method 1: select_dtypes (Recommended)
Include and Exclude Options
Method 2: pd.api.types.is_numeric_dtype
Check individual columns:
Note: is_numeric_dtype considers boolean columns as numeric. Use select_dtypes(include='number') if you want to exclude booleans.
Method 3: df.dtypes Inspection
dtype kind codes:
i— signed integeru— unsigned integerf— floating pointc— complex floating pointb— booleanO— object (usually strings)S— byte stringU— unicode stringM— datetime
Boolean Handling
Booleans are a common source of confusion:
Practical Use Cases
Correlation Matrix
Scaling/Normalization
Fill Missing Values
Summary Statistics
Handling Mixed-Type Columns
Common Pitfalls
- Assuming boolean columns are excluded by
is_numeric_dtype:is_numeric_dtype(bool_series)returns True. Useselect_dtypes(include='number')which excludes booleans, or explicitly checkdtype.kind != 'b'. - Not converting string-encoded numbers: Columns read from CSV files may contain numbers stored as strings (object dtype). Use
pd.to_numeric(col, errors='coerce')to convert them before selecting numeric columns. - Using
df.dtypes == 'int64'for exact matching: This missesint32,float64, and other numeric types. Useselect_dtypes(include='number')to catch all numeric types regardless of bit width. - Modifying a view instead of a copy:
df.select_dtypes(include='number')returns a new DataFrame. Modifying it does not change the original. Usedf[numeric_cols]to modify specific columns in the original DataFrame. - Ignoring nullable integer types: Pandas nullable types (
Int64,Float64with capital letters) may not be selected byinclude='number'in older pandas versions. Useinclude=[np.number, 'Int64', 'Float64']or upgrade to pandas 2.0+ where they are handled correctly.
Summary
- Use
df.select_dtypes(include='number')to get a DataFrame of all numeric columns - Use
.columns.tolist()to get just the column names as a list select_dtypes(include='number')excludes booleans;is_numeric_dtypeincludes them- Use
pd.to_numeric(col, errors='coerce')to convert string columns to numeric before analysis - Practical applications include correlation matrices, scaling, missing value imputation, and train/test splits
Related reading
- How do I find which attributes my tree splits on, when using scikit-learn?
- How do I generate a random vector in TensorFlow and maintain it for further use?
- How do I get a list of all the duplicate items using pandas in python?
- How do I get indices of N maximum values in a NumPy array?
- How do I find out my PYTHONPATH using Python?
- How do I find the duplicates in a list and create another list with them?
- How do I get indices of N maximum values in a NumPy array?
- How do I get the current IPython / Jupyter Notebook name
.png&w=3840&q=75)
Tackling System Design Interview Problems
A short course that equips you with the skills to approach system design interviews methodically.
Start the free courseTrack what you have practised
A free account saves your progress, solutions and study plan across every problem on Codemia.
ML System Design practice on Codemia
Design recommenders, ranking systems and training pipelines the way ML interviews actually ask for them, with worked solutions.