pandas
dataframes
data cleaning
infinite values
Python

Dropping infinite values from dataframes in pandas?

ML System Design practice on Codemia

Design recommenders, ranking systems and training pipelines the way ML interviews actually ask for them, with worked solutions.

Practice ML system design

Introduction

Infinite values (np.inf and -np.inf) in pandas DataFrames arise from operations like division by zero, logarithm of zero, or overflow in floating-point arithmetic. These values break statistical computations (mean(), std()), machine learning model training, and plotting. The standard approach is df.replace([np.inf, -np.inf], np.nan).dropna() — replace infinities with NaN, then use pandas' built-in NaN handling. Alternatively, use pd.options.mode.use_inf_as_na = True to treat infinities as NaN globally.

Identifying Infinite Values

python
1import pandas as pd
2import numpy as np
3
4df = pd.DataFrame({
5    "A": [1.0, 2.0, np.inf, 4.0],
6    "B": [-np.inf, 5.0, 6.0, 7.0],
7    "C": [8.0, 9.0, 10.0, np.inf]
8})
9
10# Detect infinite values
11print(np.isinf(df))
12#        A      B      C
13# 0  False   True  False
14# 1  False  False  False
15# 2   True  False  False
16# 3  False  False   True
17
18# Count infinities per column
19print(np.isinf(df).sum())
20# A    1
21# B    1
22# C    1
23
24# Count total infinities
25print(np.isinf(df).sum().sum())  # 3
26
27# Check which rows have any infinity
28rows_with_inf = df[np.isinf(df).any(axis=1)]
29print(rows_with_inf)

np.isinf() detects both positive and negative infinity. Use np.isposinf() and np.isneginf() to check for each separately.

Method 1: Replace and Drop

python
1# Replace inf with NaN, then drop rows with NaN
2cleaned = df.replace([np.inf, -np.inf], np.nan).dropna()
3print(cleaned)
4#      A    B     C
5# 1  2.0  5.0   9.0
6
7# Drop only in specific columns
8cleaned = df.replace([np.inf, -np.inf], np.nan).dropna(subset=["A", "B"])
9print(cleaned)
10#      A    B      C
11# 1  2.0  5.0    9.0
12# 3  4.0  7.0  inf    # C still has inf because we only checked A and B

This is the most common approach. replace() converts infinities to NaN, and dropna() removes rows containing NaN.

Method 2: Replace with a Specific Value

python
1# Replace inf with 0
2df_zero = df.replace([np.inf, -np.inf], 0)
3print(df_zero)
4#      A    B     C
5# 0  1.0  0.0   8.0
6# 1  2.0  5.0   9.0
7# 2  0.0  6.0  10.0
8# 3  4.0  7.0   0.0
9
10# Replace with column maximum (finite values only)
11for col in df.columns:
12    finite_max = df.loc[np.isfinite(df[col]), col].max()
13    df[col] = df[col].replace([np.inf, -np.inf], finite_max)
14
15# Replace positive inf with max, negative inf with min
16df_capped = df.copy()
17for col in df_capped.select_dtypes(include=[np.number]).columns:
18    finite = df_capped[col][np.isfinite(df_capped[col])]
19    df_capped[col] = df_capped[col].replace(np.inf, finite.max())
20    df_capped[col] = df_capped[col].replace(-np.inf, finite.min())

Method 3: Global Setting

python
1# Treat inf as NaN globally (affects all pandas operations)
2pd.options.mode.use_inf_as_na = True
3
4df = pd.DataFrame({"A": [1.0, np.inf, 3.0], "B": [4.0, 5.0, -np.inf]})
5
6# Now dropna() catches inf values automatically
7cleaned = df.dropna()
8print(cleaned)
9#      A    B
10# 0  1.0  4.0
11
12# mean() also ignores inf (treated as NaN)
13print(df.mean())
14# A    2.0
15# B    4.5
16
17# Reset to default behavior
18pd.options.mode.use_inf_as_na = False

Setting use_inf_as_na = True makes all pandas NaN-handling functions (like dropna(), fillna(), isna()) automatically treat infinities as missing values.

Method 4: Boolean Masking

python
1# Keep only rows where all values are finite
2df_clean = df[np.isfinite(df).all(axis=1)]
3print(df_clean)
4
5# Keep only rows where specific columns are finite
6df_clean = df[np.isfinite(df["A"]) & np.isfinite(df["B"])]
7
8# Using query with a helper column
9df["is_finite"] = np.isfinite(df).all(axis=1)
10df_clean = df[df["is_finite"]].drop(columns="is_finite")

Method 5: clip() to Cap Extreme Values

python
1# Cap values to a reasonable range instead of dropping
2df_clipped = df.clip(lower=-1e10, upper=1e10)
3print(df_clipped)
4
5# This replaces inf with 1e10 and -inf with -1e10
6# Useful when you want to preserve row count

clip() replaces values outside the specified range, effectively capping infinities to a large but finite number.

Preventing Infinite Values

python
1# Safe division that avoids inf
2def safe_divide(a, b, fill=0):
3    return np.where(b != 0, a / b, fill)
4
5df["ratio"] = safe_divide(df["A"], df["B"])
6
7# Safe log that avoids -inf
8df["log_A"] = np.log(df["A"].clip(lower=1e-10))
9
10# Detect inf before it enters the pipeline
11def validate_dataframe(df):
12    inf_count = np.isinf(df.select_dtypes(include=[np.number])).sum().sum()
13    if inf_count > 0:
14        print(f"Warning: {inf_count} infinite values detected")
15    return inf_count == 0

Common Pitfalls

  • isna() does not detect infinity by default: pd.isna(np.inf) returns False. Infinities are not NaN. You must use np.isinf() or set pd.options.mode.use_inf_as_na = True to catch them.
  • dropna() alone does not remove infinities: df.dropna() only drops NaN values. You need df.replace([np.inf, -np.inf], np.nan).dropna() to also remove infinite values.
  • Silent propagation in calculations: np.mean([1, 2, np.inf]) returns inf, not an error. Infinite values silently corrupt aggregate statistics without warning.
  • Mixed types hide infinities: Columns with mixed types (object dtype) are not checked by np.isinf(). Ensure numeric columns are typed correctly with df.astype(float) before checking.
  • use_inf_as_na is deprecated in recent pandas: In pandas 2.1+, use_inf_as_na is deprecated. Use explicit replace([np.inf, -np.inf], np.nan) instead for future compatibility.

Summary

  • Use df.replace([np.inf, -np.inf], np.nan).dropna() to remove rows with infinities
  • Use np.isinf(df) to detect infinite values (not pd.isna())
  • Replace infinities with specific values using replace() or clip()
  • pd.options.mode.use_inf_as_na = True treats infinities as NaN globally (deprecated in pandas 2.1+)
  • Prevent infinities with safe division (np.where(b != 0, a/b, fill)) and clamped log (np.log(x.clip(lower=epsilon)))
  • Always check for infinities as part of data validation before analysis or model training

Related reading
Free course
Beginner
7 lessons
2 hours
Tackling System Design Interview Problems

A short course that equips you with the skills to approach system design interviews methodically.

Start the free course
Track what you have practised

A free account saves your progress, solutions and study plan across every problem on Codemia.

ML System Design practice on Codemia

Design recommenders, ranking systems and training pipelines the way ML interviews actually ask for them, with worked solutions.

Practice ML system design

All Rights Reserved.