pandas
read_csv
dtype
python
data-processing

Pandas read_csv dtype read all columns but few as string

Master System Design with Codemia

Enhance your system design skills with over 120 practice problems, detailed solutions, and hands-on exercises.

Introduction

If you want pandas.read_csv to read almost every column as string except a small set, the practical approach is usually to read everything as string first and then convert the few exception columns. That is simpler than trying to build a large dtype map for every column you want to keep as text.

Read Everything as String First

This pattern preserves values exactly, including leading zeros and mixed-format identifiers.

python
1import pandas as pd
2
3df = pd.read_csv("data.csv", dtype=str)
4print(df.dtypes)

At this point every column is an object-backed string column. That is often the safest starting point when the file contains IDs, codes, or messy mixed data.

Convert Only the Columns That Should Be Numeric

Once the file is loaded, convert the exceptions explicitly:

python
1import pandas as pd
2
3df = pd.read_csv("data.csv", dtype=str)
4
5df["age"] = pd.to_numeric(df["age"], errors="coerce")
6df["price"] = pd.to_numeric(df["price"], errors="coerce")
7df["created_at"] = pd.to_datetime(df["created_at"], errors="coerce")
8
9print(df.dtypes)

This makes the intent obvious:

  • default everything to string
  • convert only the fields that need real numeric or datetime types

It is also easy to audit later when the input schema changes.

When a dtype Dictionary Makes Sense

If the CSV schema is fixed and known in advance, you can still build a dtype dictionary. That is useful when only a few columns should be strings and most should use specific numeric types.

python
1import pandas as pd
2
3dtypes = {
4    "id": str,
5    "zip_code": str,
6    "quantity": "Int64",
7}
8
9df = pd.read_csv("data.csv", dtype=dtypes)

The limitation is that dtype does not support a direct "all columns except these" rule. You either specify selected columns or use a global default such as dtype=str.

Why This Pattern Is Useful

CSV files often contain columns that look numeric but are really identifiers:

  • postal codes
  • account numbers
  • product codes
  • reference numbers with leading zeros

If pandas infers types automatically, those values can be altered in ways you do not want. Reading everything as string first gives you control over which conversions are intentional.

Scaling the Pattern to Many Exception Columns

If there are several exception columns, keep the conversion logic in one loop:

python
numeric_columns = ["age", "price", "quantity"]
for column in numeric_columns:
    df[column] = pd.to_numeric(df[column], errors="coerce")

This keeps the schema rules readable without relying on fragile automatic inference.

Why Post-Load Conversion Is Often Cleaner

Reading all columns as string first also gives you a chance to inspect the raw input before coercion. That is valuable when CSV files are inconsistent across days or sources. You can validate suspicious columns, log unexpected values, and only then decide which conversions should happen.

This extra control is usually worth more than a small amount of additional code, especially when ingestion bugs are expensive to diagnose later.

Common Pitfalls

  • Expecting dtype to support an "all except these columns" rule directly.
  • Letting pandas infer types for identifier columns that must retain leading zeros.
  • Converting all columns to numeric too early and losing the original text representation.
  • Ignoring parse errors instead of deciding whether invalid values should become NaN, raise, or stay as text.
  • Mixing schema control between dtype, converters, and post-load casting without a clear strategy.

Summary

  • The easiest way to read almost everything as string is pd.read_csv(..., dtype=str).
  • Convert only the few exception columns after loading the file.
  • This pattern is safer for IDs and codes that only look numeric.
  • Use a dtype dictionary only when you truly want to control specific columns at read time.
  • Clear, explicit post-load conversion is often easier to maintain than relying on inference.

Course illustration
Course illustration

All Rights Reserved.