Replacing column values in a pandas DataFrame

pandas

DataFrame

data manipulation

Python

column replacement

Replacing column values in a pandas DataFrame

ML System Design practice on Codemia

Design recommenders, ranking systems and training pipelines the way ML interviews actually ask for them, with worked solutions.

Practice ML system design

Data Science Python

Introduction

Pandas provides several methods to replace values in a DataFrame column, each suited to different scenarios. replace() handles exact value mapping, .loc[] with conditions handles rule-based replacement, np.where() handles binary conditions, map() handles complete column remapping, and apply() handles complex transformations. The right choice depends on whether you are replacing specific values, applying conditions, or transforming the entire column.

replace() — Exact Value Mapping

python

1import pandas as pd
2
3df = pd.DataFrame({
4    "status": ["P", "D", "P", "C", "D"],
5    "priority": ["H", "L", "M", "H", "L"]
6})
7
8# Replace specific values with a dictionary
9df["status"] = df["status"].replace({
10    "P": "pending",
11    "D": "done",
12    "C": "cancelled"
13})
14print(df["status"].values)
15# ['pending' 'done' 'pending' 'cancelled' 'done']
16
17# Replace a single value
18df["priority"] = df["priority"].replace("H", "high")
19
20# Replace with regex
21df["priority"] = df["priority"].replace(r"^L$", "low", regex=True)

replace() is the go-to for mapping old values to new values. It leaves values not in the mapping unchanged.

.loc[] — Conditional Replacement

python

1df = pd.DataFrame({
2    "score": [85, 42, 73, 91, 55],
3    "grade": ["", "", "", "", ""]
4})
5
6# Replace based on conditions
7df.loc[df["score"] >= 90, "grade"] = "A"
8df.loc[(df["score"] >= 80) & (df["score"] < 90), "grade"] = "B"
9df.loc[(df["score"] >= 70) & (df["score"] < 80), "grade"] = "C"
10df.loc[(df["score"] >= 60) & (df["score"] < 70), "grade"] = "D"
11df.loc[df["score"] < 60, "grade"] = "F"
12
13print(df)
14#    score grade
15# 0     85     B
16# 1     42     F
17# 2     73     C
18# 3     91     A
19# 4     55     F

.loc[] modifies the DataFrame in place for rows matching the condition. It is clear and readable for multi-condition replacements.

np.where() — Binary Conditions

python

1import numpy as np
2
3df = pd.DataFrame({
4    "amount": [150, 2500, 75, 10000, 500],
5})
6
7# Two-way conditional
8df["category"] = np.where(df["amount"] > 1000, "high", "normal")
9print(df)
10#    amount category
11# 0     150   normal
12# 1    2500     high
13# 2      75   normal
14# 3   10000     high
15# 4     500   normal

np.where(condition, value_if_true, value_if_false) is the most concise way to handle binary choices.

Nested np.where for Multiple Conditions

python

1df["tier"] = np.where(
2    df["amount"] > 5000, "premium",
3    np.where(df["amount"] > 1000, "standard", "basic")
4)
5print(df["tier"].values)
6# ['basic' 'standard' 'basic' 'premium' 'basic']

For more than 2-3 tiers, np.select() is cleaner.

np.select() — Multiple Conditions

python

1conditions = [
2    df["amount"] > 5000,
3    df["amount"] > 1000,
4    df["amount"] > 100,
5]
6choices = ["premium", "standard", "basic"]
7
8df["tier"] = np.select(conditions, choices, default="free")
9print(df["tier"].values)
10# ['basic' 'standard' 'basic' 'premium' 'basic']

np.select() evaluates conditions in order and assigns the first matching choice. The default parameter handles rows that match none of the conditions.

map() — Complete Column Remapping

python

1df = pd.DataFrame({
2    "day_num": [1, 2, 3, 4, 5]
3})
4
5day_names = {1: "Mon", 2: "Tue", 3: "Wed", 4: "Thu", 5: "Fri"}
6df["day_name"] = df["day_num"].map(day_names)
7print(df)
8#    day_num day_name
9# 0        1      Mon
10# 1        2      Tue
11# 2        3      Wed
12# 3        4      Thu
13# 4        5      Fri

map() replaces every value using the mapping. Values not in the mapping become NaN — unlike replace() which leaves them unchanged.

python

# map with a function
df["day_upper"] = df["day_name"].map(str.upper)
# ['MON', 'TUE', 'WED', 'THU', 'FRI']

apply() — Complex Transformations

python

1def categorize_amount(x):
2    if x > 5000:
3        return "premium"
4    elif x > 1000:
5        return "standard"
6    elif x > 100:
7        return "basic"
8    else:
9        return "free"
10
11df["tier"] = df["amount"].apply(categorize_amount)

apply() runs a Python function on each value. It is the most flexible but slowest option — prefer vectorized operations (np.where, np.select, .loc[]) when possible.

String Replacement

python

1df = pd.DataFrame({
2    "email": ["[email protected]", "[email protected]", "[email protected]"]
3})
4
5# Replace substring in string column
6df["email"] = df["email"].str.replace("@old.com", "@new.com", regex=False)
7print(df["email"].values)
8# ['[email protected]' '[email protected]' '[email protected]']
9
10# Regex replacement
11df["clean"] = df["email"].str.replace(r"@.*", "", regex=True)
12print(df["clean"].values)
13# ['alice' 'bob' 'carol']

.str.replace() operates on string columns. Set regex=False for literal string replacement (faster and safer).

Replacing NaN and Missing Values

python

1df = pd.DataFrame({
2    "city": ["NYC", None, "LA", None, "Chicago"],
3    "score": [85, float("nan"), 73, 91, float("nan")]
4})
5
6# Fill NaN with a default
7df["city"] = df["city"].fillna("unknown")
8df["score"] = df["score"].fillna(df["score"].mean())
9
10# Replace specific values with NaN
11df["city"] = df["city"].replace("unknown", pd.NA)

Replacing Across Multiple Columns

python

1df = pd.DataFrame({
2    "a": ["yes", "no", "yes"],
3    "b": ["no", "yes", "no"]
4})
5
6# Replace in all columns at once
7df = df.replace({"yes": 1, "no": 0})
8print(df)
9#    a  b
10# 0  1  0
11# 1  0  1
12# 2  1  0
13
14# Replace in specific columns
15df[["a", "b"]] = df[["a", "b"]].replace({1: True, 0: False})

Performance Comparison

Method	Best For	Speed
`replace()`	Exact value mapping	Fast (vectorized)
`.loc[]`	Conditional replacement	Fast (vectorized)
`np.where()`	Binary conditions	Fastest
`np.select()`	Multiple conditions	Fast
`map()`	Complete remapping	Fast
`apply()`	Complex logic	Slow (row-by-row)
`.str.replace()`	String patterns	Medium

Common Pitfalls

Chained assignment warning: df[df["x"] > 0]["y"] = 1 does not modify df — it modifies a copy. Use df.loc[df["x"] > 0, "y"] = 1 instead.
map() turns unmapped values to NaN: Unlike replace(), map() sets values not in the mapping to NaN. Use replace() if you want to keep unmapped values unchanged.
Forgetting regex=False in str.replace: By default, str.replace() treats the pattern as a regex. Characters like ., (, $ have special meaning. Use regex=False for literal replacements.
Modifying during iteration: Never replace values while iterating with iterrows(). Use vectorized operations or apply().
Type changes after replacement: Replacing numeric values with strings changes the column dtype. Check with df.dtypes after replacement.

Summary

Use replace() for mapping specific old values to new values
Use .loc[] for conditional replacement based on column values
Use np.where() for binary (if/else) replacement
Use np.select() for multiple conditions with multiple choices
Use map() for complete column remapping (unmapped values become NaN)
Use apply() only when vectorized alternatives cannot express the logic
Always assign back to the column (df["col"] = ...) to avoid chained assignment issues

Tackling System Design Interview Problems

A short course that equips you with the skills to approach system design interviews methodically.

Start the free course

Track what you have practised

A free account saves your progress, solutions and study plan across every problem on Codemia.

ML System Design practice on Codemia

Design recommenders, ranking systems and training pipelines the way ML interviews actually ask for them, with worked solutions.

Practice ML system design

Replacing column values in a pandas DataFrame

ML System Design practice on Codemia

Introduction

replace() — Exact Value Mapping

.loc[] — Conditional Replacement

np.where() — Binary Conditions

Nested np.where for Multiple Conditions

np.select() — Multiple Conditions

map() — Complete Column Remapping

apply() — Complex Transformations

String Replacement

Replacing NaN and Missing Values

Replacing Across Multiple Columns

Performance Comparison

Common Pitfalls

Summary

Related reading

Tackling System Design Interview Problems

Track what you have practised

ML System Design practice on Codemia