pandas
DataFrame
null values
data selection
Python

How to select rows with one or more nulls from a pandas DataFrame without listing columns explicitly?

Master System Design with Codemia

Enhance your system design skills with over 120 practice problems, detailed solutions, and hands-on exercises.

Introduction

In pandas, the clean way to select rows that contain at least one missing value is to build a boolean mask with isna() and reduce it across columns with any(axis=1). This works for the whole DataFrame, so you do not need to name every column manually.

The pattern is concise, fast, and readable once you understand what each piece does. It is also easy to adapt if later you want rows with all nulls, rows with no nulls, or rows with nulls only in a subset of columns.

Because the check is column-agnostic, it also survives schema changes better than hand-written column lists. That makes it a good default in notebooks, ETL jobs, and exploratory analysis.

Use isna().any(axis=1)

Here is the standard solution:

python
1import pandas as pd
2import numpy as np
3
4df = pd.DataFrame({
5    "name": ["Ava", "Ben", "Cara", "Drew"],
6    "age": [28, np.nan, 31, 25],
7    "city": ["Toronto", "Ottawa", None, "Montreal"],
8})
9
10rows_with_nulls = df[df.isna().any(axis=1)]
11print(rows_with_nulls)

Output:

python
   name   age      city
1   Ben   NaN    Ottawa
2  Cara  31.0      None

This works because:

  • 'df.isna() produces a boolean DataFrame'
  • 'any(axis=1) checks whether any column in each row is True'
  • the resulting boolean Series is used to filter df

Understand the Intermediate Mask

It is often helpful to inspect the mask directly:

python
mask = df.isna().any(axis=1)
print(mask)

That prints something like:

python
10    False
21     True
32     True
43    False
5dtype: bool

Seeing the mask makes it obvious why the final filtering works. This is the same general pattern used for many pandas row-selection tasks.

If you want rows where all columns are null, use all(axis=1) instead:

python
rows_all_null = df[df.isna().all(axis=1)]

If you want rows with no nulls, invert the mask:

python
rows_without_nulls = df[~df.isna().any(axis=1)]

If later you decide only some columns matter, select those columns first:

python
rows_null_in_subset = df[df[["age", "city"]].isna().any(axis=1)]

That still avoids spelling out every column in the full DataFrame.

Count Missing Values Per Row

Sometimes you need more than a yes or no answer. You can count nulls per row and filter on that count:

python
null_count = df.isna().sum(axis=1)
rows_with_two_or_more_nulls = df[null_count >= 2]

This is useful for data-quality checks and threshold-based cleanup.

Why Not Loop Through Columns Manually

You could write Python code that inspects each row and column yourself, but pandas is built for vectorized operations. The isna().any(axis=1) idiom is shorter, faster, and communicates intent clearly to other pandas users.

Manual loops also make it easy to accidentally miss a column when the schema changes. The whole point of this pattern is that it automatically handles whatever columns the DataFrame currently has.

Common Pitfalls

  • Using axis=0 by mistake, which checks columns instead of rows.
  • Confusing None, NaN, and empty strings. Empty strings are not considered null unless you convert them.
  • Calling dropna() when you only wanted to inspect matching rows rather than remove them.
  • Forgetting that boolean masks must align with the DataFrame index.
  • Writing a manual loop when pandas already provides a vectorized solution.

Summary

  • Use df[df.isna().any(axis=1)] to select rows with at least one null value.
  • 'isna() marks missing entries and any(axis=1) reduces the check across each row.'
  • Switch to all(axis=1) when you need rows that are entirely null.
  • Invert the mask with ~ to keep only complete rows.
  • This approach automatically adapts to all current columns without naming them one by one.

Course illustration
Course illustration

All Rights Reserved.