Introduction

Remapping a pandas column with a dictionary is easy until missing values enter the picture. The subtle part is not preserving existing NaN values, because pandas already does that well. The real subtlety is deciding what should happen to non-null values that are not present in the mapping dictionary.

map Preserves Existing NaN Values

The simplest remapping tool is Series.map:

1import pandas as pd
2import numpy as np
3
4df = pd.DataFrame(
5    {
6        "status": ["new", "done", np.nan, "new", "hold"]
7    }
8)
9
10mapping = {
11    "new": "open",
12    "done": "closed",
13}
14
15result = df["status"].map(mapping)
16print(result)

Output:

10      open
21    closed
32       NaN
43      open
54       NaN
6Name: status, dtype: object

Notice the two different reasons for NaN in the result:

• the original third row was already missing
• the value "hold" was not found in the dictionary

That is the key behavior of map. It preserves existing NaN values, but it also turns unmapped non-null values into NaN.

Keep Unmapped Values Instead of Replacing Them with NaN

If you want to remap only known values and leave everything else unchanged, combine map with fillna:

remapped = df["status"].map(mapping).fillna(df["status"])
print(remapped)

Output:

10      open
21    closed
32       NaN
43      open
54      hold
6Name: status, dtype: object

This pattern is often the most useful answer in real data-cleaning work because it preserves both original missing values and original unmapped categories.

replace Is Sometimes the Better Tool

For straightforward value substitution, replace is even more direct:

replaced = df["status"].replace(mapping)
print(replaced)

Output:

10      open
21    closed
32       NaN
43      open
54      hold
6Name: status, dtype: object

Unlike map, replace changes only matching values and leaves everything else alone. That makes it a strong choice when your mapping is partial by design.

Another practical difference is intent. When another developer reads replace(mapping), the code clearly says "swap known labels." When they read map(mapping), it often implies a stricter lookup where missing keys may be meaningful. That small readability difference is useful in data-cleaning pipelines.

Apply the Remap Back to the DataFrame

Once the logic is correct, write it back explicitly:

df["status"] = df["status"].replace(mapping)
print(df)

If the column uses pandas' nullable dtypes or categorical data, it is still worth checking the final dtype after transformation. Some remapping operations can widen the dtype to object when the new values do not fit the original representation cleanly.

Choose the Method Based on Intent

Use:

• 'map when you want a strict mapping and are comfortable with unknown values becoming NaN'
• 'map(...).fillna(original_series) when you want mapped values plus untouched unknowns'
• 'replace when you want partial substitution and existing unknowns should remain unchanged'

That choice matters more than memorizing one function name.

Common Pitfalls

• Assuming map will leave unmapped non-null values unchanged.
• Forgetting that NaN in the result may come from either original missing values or missing dictionary keys.
• Using apply with a custom lambda for a simple dictionary substitution that map or replace already handles efficiently.
• Overwriting the original column before checking how unmapped categories behaved.
• Ignoring dtype changes after remapping, especially with categorical or nullable columns.

Summary

• pandas map already preserves existing NaN values.
• Unmapped non-null values become NaN when you use map.
• Use map(...).fillna(original_series) if unknown values should remain unchanged.
• Use replace when you want partial substitution without turning unknown values into missing values.
• The right tool depends on whether unmapped values should disappear or survive.