How to pass another entire column as argument to pandas fillna
ML System Design practice on Codemia
Design recommenders, ranking systems and training pipelines the way ML interviews actually ask for them, with worked solutions.
Introduction
In Pandas, fillna does not need a single scalar value. You can pass an entire Series, which lets Pandas fill missing values in one column with row-aligned values from another column.
The direct pattern
If column a has missing values and column b contains the fallback values, the simplest solution is:
The important point is that df["b"] is a full Series, not one number. Pandas aligns by index, then fills only the rows where a is missing.
This makes the operation expressive and efficient. You do not need a Python loop or an apply for something that is fundamentally vectorized.
Why alignment matters
Pandas works by index alignment, not just by physical position. If the fallback series has the same values but a different index, the fill can produce unexpected results because Pandas matches labels first.
That behavior is usually a feature, not a bug. It protects you when rows were filtered, sorted, or merged earlier in the pipeline. But it also means you should be deliberate about which Series you pass into fillna.
If the source column comes from the same DataFrame, you usually get the intended alignment automatically.
Related alternatives
fillna is not the only way to express this idea. combine_first is often equally readable when you want to say "take values from the left series, otherwise use the right series."
For many readers, fillna is clearer when you are thinking about a column with holes. combine_first is clearer when you are thinking about one preferred series backed by a fallback series.
You can also express the same logic with where:
That is useful when the rule grows beyond simple missing-value replacement.
Data types and practical behavior
The fill only happens where the target column is missing. Existing non-null values stay untouched. The resulting dtype depends on the combination of the two columns. For example, if one column is float because it contains missing values and the other is integer, the result may stay float unless you convert it afterward.
That is normal Pandas behavior. Missing numeric data often pushes columns into floating-point or nullable dtypes. If the final business meaning is integer, cast the result explicitly once the missing values are resolved.
When this pattern is the right choice
Use this when two columns represent equivalent or fallback information, such as preferred_name and legal_name, price_override and base_price, or city and default_city. The pattern is especially strong when the relationship is row-by-row and local.
If the replacement depends on group-level rules, such as filling from another row in the same department, use groupby plus transform before calling fillna.
Common Pitfalls
- Passing a series with a mismatched index and expecting purely positional filling.
- Using
applyor Python loops for a problem thatfillnaalready solves vectorially. - Forgetting that only missing values are replaced; existing values stay as they are.
- Ignoring dtype changes after the fill and then being surprised by floats or nullable types.
- Using another column as a fallback when the columns do not actually represent compatible data.
Summary
- '
fillnacan take an entireSeries, not just a scalar.' - '
df["a"].fillna(df["b"])fills missing values inafrom row-aligned values inb.' - Pandas aligns by index, so matching labels matter.
- '
combine_firstandwhereare useful alternatives for related cases.' - After filling, check the resulting dtype if the column has a strict business meaning.
Related reading
- How to pass in multidimensional data to xgboost model
- How to pass pandas dataframe to airflow tasks
- How to perform 10 fold cross validation with LibSVM in R?
- How to perform mean subtraction and normalization with Tensorflow
- How to pass arguments to a Button command in Tkinter?
- How to pass body into aiohttp get request?
- How to pick color palette for a pie-chart?
- How to plot a high resolution graph
.png&w=3840&q=75)
Tackling System Design Interview Problems
A short course that equips you with the skills to approach system design interviews methodically.
Start the free courseTrack what you have practised
A free account saves your progress, solutions and study plan across every problem on Codemia.
ML System Design practice on Codemia
Design recommenders, ranking systems and training pipelines the way ML interviews actually ask for them, with worked solutions.