Introduction

DataFrame.apply can reference multiple columns, but it fails when axis is wrong, row indexing is incorrect, or return shape does not match assignment target. Most issues come from mixing row wise and column wise mental models. A small setup change usually fixes it.

The Most Common Cause: Wrong axis

By default, apply works column wise with axis=0. To process each row using multiple columns, use axis=1.

1import pandas as pd
2
3df = pd.DataFrame({
4    "price": [100, 200, 150],
5    "qty": [2, 1, 3]
6})
7
8def total(row):
9    return row["price"] * row["qty"]
10
11df["total"] = df.apply(total, axis=1)
12print(df)

Without axis=1, row is actually a column series and key lookups fail.

Accessing Columns Correctly Inside Function

When row input is a Series, use labels that exactly match DataFrame column names.

1def shipping_band(row):
2    if row["price"] > 180 and row["qty"] >= 1:
3        return "premium"
4    return "standard"
5
6df["band"] = df.apply(shipping_band, axis=1)

Typos or trailing spaces in column names are a frequent source of KeyError.

Returning Multiple Values

If your function returns more than one field, expand it into multiple columns.

1def compute_fields(row):
2    subtotal = row["price"] * row["qty"]
3    tax = subtotal * 0.13
4    return pd.Series({"subtotal": subtotal, "tax": tax})
5
6df[["subtotal", "tax"]] = df.apply(compute_fields, axis=1)
7print(df)

If return object shape is inconsistent, assignment errors appear.

Prefer Vectorization When Possible

apply(axis=1) is convenient but slower than vectorized operations for large frames.

df["total"] = df["price"] * df["qty"]
df["tax"] = df["total"] * 0.13

Use apply for complex branching logic that is hard to express vectorially. Otherwise prefer direct column operations.

Safer Conditional Logic with np.where

For many row conditions, numpy.where is faster and clearer.

1import numpy as np
2
3df["band"] = np.where(
4    (df["price"] > 180) & (df["qty"] >= 1),
5    "premium",
6    "standard"
7)

This avoids Python level row loops and scales better.

Debugging a Broken apply

When apply fails, inspect one row sample and function output directly.

sample = df.iloc[0]
print(sample)
print(total(sample))

Also check dtypes:

print(df.dtypes)

String values in numeric columns can cause unexpected arithmetic behavior or conversion errors.

Passing Extra Arguments

If function needs constants, pass via args or closure.

1def discounted_total(row, rate):
2    return row["price"] * row["qty"] * (1 - rate)
3
4df["discounted"] = df.apply(discounted_total, axis=1, args=(0.1,))

This is cleaner than hardcoding constants inside function body.

Unit Testing apply Logic

A practical way to prevent regressions is extracting row logic into a pure function and testing it with representative dictionaries. Include tests for nulls, string numbers, and unexpected categories so conversion or branching bugs are caught before full DataFrame execution. This also makes refactoring from apply to vectorized logic easier because expected outputs are already documented by tests.

For pipelines with strict schemas, add astype conversions before apply so row functions receive normalized types. This removes implicit coercion surprises and makes results reproducible across CSV, parquet, and database sourced data.

Profile runtime on representative data sizes before committing to row wise apply in production workflows.

Common Pitfalls

• Forgetting axis=1 when row wise logic references multiple columns.
• Returning variable length outputs and assigning to fixed column targets.
• Using apply for simple arithmetic that should be vectorized.
• Referencing wrong column labels due to typos or hidden whitespace.
• Ignoring dtype issues that make numeric operations behave like string concatenation.

Summary

• For multi column row logic, use apply(..., axis=1).
• Access row values by exact column labels.
• Return Series when expanding into multiple output columns.
• Prefer vectorized expressions for performance on large datasets.
• Debug by testing function on a single row and verifying input dtypes.