Remove rows with all or some NAs (missing values) in data.frame
ML System Design practice on Codemia
Design recommenders, ranking systems and training pipelines the way ML interviews actually ask for them, with worked solutions.
Introduction
In R, removing rows with missing values is simple once you separate two different questions: do you want to drop rows that contain any NA, or only rows that are entirely missing? Those are different cleaning rules, and choosing the wrong one can quietly remove much more data than you intended.
Remove Rows That Contain Any NA
If your rule is "drop the row if any column is missing," the usual tools are na.omit() or complete.cases().
Example:
Equivalent with complete.cases():
Both keep only rows where every column is present.
This is the strictest rule and is useful when the downstream analysis requires complete records.
Remove Rows Where All Values Are NA
Sometimes you only want to remove rows that are completely empty, while keeping partially filled rows.
A common pattern is:
Here is what it does:
- '
is.na(df)creates a logical matrix' - '
rowSums(...)counts missing values per row' - '
ncol(df)is the total number of columns' - rows with missing-count equal to all columns are removed
This is the right pattern when you want to keep rows that still contain at least one real value.
Remove Rows with a Threshold of Missingness
You may want something in between those two extremes. For example, remove rows only if they have two or more missing values.
This keeps rows with zero or one missing value and drops rows with two or more.
That approach is often more realistic in data-cleaning workflows because many real datasets contain a small amount of missingness that you may be willing to tolerate.
Restrict the Check to Specific Columns
Sometimes only certain fields matter for the decision.
Suppose you want to keep rows only when A and B are both present, regardless of C:
Or remove rows where both A and B are missing:
This is safer than applying a global missing-value rule to columns that do not matter for the current analysis.
subset() and dplyr Alternatives
Base R is often enough, but some teams prefer dplyr for readability.
Example with dplyr to remove rows where all selected columns are missing:
For a small number of columns, that is easy to read. For broader patterns, rowSums(is.na(...)) is usually more scalable.
Why the Choice Matters Analytically
Dropping every row with any NA can shrink the dataset dramatically, especially when many columns are present. A row may be usable for one model or report even if one unrelated column is missing.
That is why a good cleaning rule is usually tied to the actual analysis goal:
- strict completeness for some statistical models
- partial tolerance for reporting or exploratory work
- column-specific rules for business logic
The right rule depends on what the missing data means, not just on what code is shortest.
Common Pitfalls
The most common mistake is using na.omit() without realizing it removes rows containing any NA anywhere in the data frame. That is sometimes much more aggressive than intended.
Another issue is forgetting to limit the missing-value test to relevant columns. If one optional metadata column is incomplete, it should not necessarily disqualify the whole row.
People also confuse "remove rows with all NA" and "remove rows with some NA" because the code patterns look similar. Always state the rule in plain language before writing the expression.
Finally, do not assume dropping rows is always the right solution. In some analyses, imputing missing values or modeling the missingness directly is more appropriate.
Summary
- Use
na.omit()orcomplete.cases()to remove rows with anyNA. - Use
rowSums(is.na(df)) < ncol(df)to remove only rows that are entirely missing. - Threshold-based rules are easy with
rowSums(is.na(df)). - Restrict missing-value checks to relevant columns when necessary.
- Choose the removal rule based on the analysis goal, not just on convenience.
Related reading
- Remove unwanted parts from strings in a column
- Removing Conda environment
- Removing index column in pandas when reading a csv
- Rename Pandas DataFrame Index
- Rename specific columns in pandas
- Renaming column names in Pandas
- Replace all elements of NumPy array that are greater than some value
- Replace nan values in tensorflow tensor
.png&w=3840&q=75)
Tackling System Design Interview Problems
A short course that equips you with the skills to approach system design interviews methodically.
Start the free courseTrack what you have practised
A free account saves your progress, solutions and study plan across every problem on Codemia.
ML System Design practice on Codemia
Design recommenders, ranking systems and training pipelines the way ML interviews actually ask for them, with worked solutions.