Pandas sum DataFrame rows for given columns
ML System Design practice on Codemia
Design recommenders, ranking systems and training pipelines the way ML interviews actually ask for them, with worked solutions.
Introduction
Summing selected columns row by row is a very common pandas operation. It appears in scoring models, reporting tables, feature engineering, and data-cleaning pipelines. The mechanics are simple, but correct results depend on choosing the right columns, using the right axis, and deciding how missing values should behave.
Sum Specific Columns Row-Wise
The basic pattern is to select the columns you want and call sum(axis=1). The axis=1 part is essential because it tells pandas to sum across columns for each row.
This keeps the original columns and adds a derived total. It is the right approach when the set of columns is known and stable.
Selecting Columns Dynamically
Hardcoding every column is not always practical. In wide tables, it is often better to derive the list from a naming rule.
This is useful when upstream schemas change or when the column group is defined by metadata rather than by a fixed hand-written list.
Handling Missing Values Intentionally
By default, pandas skips missing values when summing. That is often convenient, but it is not always correct for the business rule.
The skip_missing column still produces totals when one input is missing. The min_count=3 version returns a missing result unless all three values are present. Choose the rule explicitly. Otherwise your totals may look valid while hiding incomplete records.
Fixing Data Types Before Summing
Another common problem is mixed types. Numeric-looking values sometimes arrive as strings, especially after CSV imports or poorly typed JSON ingestion. Convert them before summing.
Using errors="coerce" turns invalid numeric values into missing values instead of throwing immediately. That can be useful in cleaning pipelines, but it should be paired with validation so bad source data does not pass unnoticed.
Weighted or Conditional Row Sums
Sometimes a row total is not a plain arithmetic sum. You may need weights or conditions.
Or you may want to sum only positive values:
These patterns are common in scoring systems and business rules where not every column contributes equally.
Common Pitfalls
The most common mistake is forgetting axis=1. Without it, sum aggregates down each column instead of across each row.
Another issue is accidentally including non-numeric columns in the selection set. That may fail outright or produce inconsistent behavior depending on dtypes.
Missing-value behavior is another source of confusion. The default skip behavior is convenient, but it may hide incomplete input. If all values are required, use min_count or validate before summing.
Finally, avoid writing row-wise Python loops for this task. pandas column operations are clearer and usually much faster than iterating through rows manually.
Summary
- Use
df[columns].sum(axis=1)to sum selected columns per row. - Build the column list dynamically when schema rules are pattern-based.
- Decide whether missing values should be skipped or should invalidate the result.
- Convert string-like numeric data before aggregation.
- Prefer vectorized
pandasoperations over manual row loops.
Related reading
- pandas three-way joining multiple dataframes on columns
- pandas unique values multiple columns
- pandas.factorize on an entire data frame
- pandas.parser.CParserError Error tokenizing data
- Parallel Import a python file from sibling folder
- Parse a .py file, read the AST, modify it, then write back the modified source code
- Parallel processes in distributed tensorflow
- Parsing one terabyte of text and efficiently counting the number of occurrences of each word
.png&w=3840&q=75)
Tackling System Design Interview Problems
A short course that equips you with the skills to approach system design interviews methodically.
Start the free courseTrack what you have practised
A free account saves your progress, solutions and study plan across every problem on Codemia.
ML System Design practice on Codemia
Design recommenders, ranking systems and training pipelines the way ML interviews actually ask for them, with worked solutions.