Convert dataframe columns of object type to float
ML System Design practice on Codemia
Design recommenders, ranking systems and training pipelines the way ML interviews actually ask for them, with worked solutions.
Introduction
In pandas, columns with dtype object often contain numbers stored as strings. That is common after reading CSV files, scraping data, or merging inconsistent sources. Converting those columns to float is usually simple, but only if you first decide how to handle bad values such as empty strings, commas, or mixed text.
Start with One Clean Column
If a column contains numeric-looking strings, the most direct conversion is:
That works only when every value is already convertible. If any row contains "N/A" or "1,200", astype(float) raises an exception.
Use pd.to_numeric for Real Data
For messy data, pd.to_numeric is safer because it lets you choose what to do with invalid values.
With errors="coerce", invalid values become NaN instead of crashing the conversion. That is often the right default for analysis pipelines because it makes the problem visible while still letting the column become numeric.
Clean Formatting Before Conversion
Real object columns often fail conversion for formatting reasons rather than truly non-numeric content. Common examples include:
- commas in thousands separators
- percent signs
- leading or trailing whitespace
- currency symbols
Clean the text first, then convert:
The order matters. Cleaning first usually keeps more valid rows.
Convert Several Object Columns at Once
If multiple columns should be numeric, select them explicitly instead of converting every object column blindly. Some object columns are really categorical text and should stay that way.
This keeps the conversion targeted and easier to review.
Know the Difference Between Conversion and Inference
Pandas can sometimes infer numeric dtype during file import, but inference is not guaranteed. If a column contains one bad string among many numeric rows, pandas may load the whole column as object.
That is why conversion is often a cleanup step after reading the file:
If bad values are unexpected, you may prefer errors="raise" while debugging so the pipeline fails loudly.
Float Is Not Always the Best Target
People often say "convert to float" when they really mean "convert to numeric." If a column contains integers only, Int64 or int64 may be a better type. If precision matters, such as currency calculations, float may not be ideal at all.
Still, float is a common target because it can represent NaN, which makes mixed-quality data easier to handle during cleanup.
Common Pitfalls
- Using
astype(float)on dirty data and being surprised when one bad value crashes the whole conversion. - Converting every object column blindly, including text columns that should remain strings.
- Forgetting to strip commas, currency symbols, or whitespace before conversion.
- Assuming
objectmeans text only; it is really a catch-all dtype. - Treating float conversion as semantically correct for every numeric field, even when integer or decimal precision matters more.
Summary
- '
objectcolumns often contain numeric strings that need explicit conversion.' - Use
astype(float)only when the data is already clean. - Use
pd.to_numeric(..., errors="coerce")for safer real-world conversions. - Clean formatting artifacts such as commas and currency symbols before conversion.
- Convert only the columns that are supposed to become numeric, not every object column automatically.
Related reading
- Convert dataframe index to datetime
- Convert floats to ints in Pandas?
- Convert list of dictionaries to a pandas DataFrame
- Convert list of dictionaries to a pandas DataFrame
- Convert date to datetime in Python
- Convert date to datetime in Python
- Convert NumPy array to Python list
- Convert own image to MNIST's image
.png&w=3840&q=75)
Tackling System Design Interview Problems
A short course that equips you with the skills to approach system design interviews methodically.
Start the free courseNo course covers this one yet
Basis Lab writes one for you from a sentence about what you want to be able to do, then teaches it and checks you understood. Your first course is free.
Build my courseTrack what you have practised
A free account saves your progress, solutions and study plan across every problem on Codemia.
ML System Design practice on Codemia
Design recommenders, ranking systems and training pipelines the way ML interviews actually ask for them, with worked solutions.