pandas
Python
data analysis
data manipulation
apply function

Applying function with multiple arguments to create a new pandas column

ML System Design practice on Codemia

Design recommenders, ranking systems and training pipelines the way ML interviews actually ask for them, with worked solutions.

Practice ML system design

Introduction

When working with Pandas DataFrames, you often need to compute a new column based on values from multiple existing columns. While simple arithmetic can be done directly with column operations, more complex logic requires passing multiple arguments into a function. Pandas provides several approaches for this, each with different trade-offs in readability, flexibility, and performance. This article walks through four techniques so you can pick the right one for your situation.

Using apply with a Lambda Function

The most straightforward approach is calling apply() on a single column and passing additional arguments through a lambda. This works well when your function takes one "main" column value plus some extra parameters.

python
1import pandas as pd
2
3df = pd.DataFrame({
4    'price': [100, 200, 150],
5    'tax_rate': [0.08, 0.10, 0.06]
6})
7
8def compute_total(price, tax_rate):
9    return price * (1 + tax_rate)
10
11df['total'] = df.apply(lambda row: compute_total(row['price'], row['tax_rate']), axis=1)
12print(df)

Output:

text
1   price  tax_rate  total
20    100      0.08  108.0
31    200      0.10  220.0
42    150      0.06  159.0

The axis=1 parameter tells Pandas to pass each row (not each column) to the lambda. Inside the lambda, row is a Series with the column names as its index, so you access values with row['column_name'].

You can also use the args parameter to pass extra fixed values:

python
1def compute_discounted(price, discount):
2    return price * (1 - discount)
3
4df['discounted'] = df['price'].apply(compute_discounted, args=(0.15,))

Here, apply calls compute_discounted(price_value, 0.15) for each element in the price column. The args tuple supplies the second argument.

Using apply with axis=1 and a Named Function

For more complex row-wise logic, define a standalone function that receives the entire row. This is cleaner than cramming everything into a lambda.

python
1def categorize_item(row):
2    if row['price'] > 180 and row['tax_rate'] > 0.08:
3        return 'premium'
4    elif row['price'] > 120:
5        return 'standard'
6    else:
7        return 'budget'
8
9df['category'] = df.apply(categorize_item, axis=1)
10print(df)

Output:

text
1   price  tax_rate  total  discounted  category
20    100      0.08  108.0        85.0    budget
31    200      0.10  220.0       170.0   premium
42    150      0.06  159.0       127.5  standard

This pattern is ideal when the logic involves conditionals, lookups, or anything that does not reduce to a simple arithmetic expression. The function signature takes a single row parameter, and you access columns by name inside the function body.

Using np.vectorize

When apply feels slow on large DataFrames, np.vectorize offers a middle ground. It wraps a Python function so you can call it with entire columns at once, even though it still loops internally.

python
1import numpy as np
2
3def compute_total(price, tax_rate):
4    return price * (1 + tax_rate)
5
6vectorized_total = np.vectorize(compute_total)
7df['total_v'] = vectorized_total(df['price'], df['tax_rate'])

The key advantage is readability: the call site looks like a vectorized operation even though the underlying execution is element-wise. Performance is modestly better than apply with axis=1 because it avoids constructing a Series for each row, but it is not as fast as true NumPy vectorization.

Use np.vectorize when your function is simple, takes multiple column inputs, and you want clean syntax without rewriting the logic as a pure NumPy expression.

Using df.assign with Vectorized Operations

The fastest approach is to skip apply entirely and use native Pandas or NumPy vectorized operations. The assign method lets you create new columns in a chainable, expressive way.

python
1df = df.assign(
2    total_fast=lambda d: d['price'] * (1 + d['tax_rate']),
3    savings=lambda d: d['price'] - d['price'] * (1 - 0.15)
4)
5print(df[['price', 'tax_rate', 'total_fast', 'savings']])

Output:

text
1   price  tax_rate  total_fast  savings
20    100      0.08       108.0     15.0
31    200      0.10       220.0     30.0
42    150      0.06       159.0     22.5

This is the recommended approach whenever the computation can be expressed as arithmetic on columns. It runs at C speed through NumPy's underlying implementation and can be orders of magnitude faster than apply on DataFrames with hundreds of thousands of rows.

You can also use np.where for conditional logic:

python
df['label'] = np.where(df['price'] > 150, 'expensive', 'affordable')

Common Pitfalls

  • Forgetting axis=1 when using apply row-wise. The default axis=0 applies the function column-wise, which produces confusing errors or wrong results when you intended row-wise operation.
  • Using apply for simple arithmetic that could be vectorized. Writing df.apply(lambda row: row['a'] + row['b'], axis=1) is roughly 100 times slower than df['a'] + df['b'] on large DataFrames. Always prefer vectorized operations when possible.
  • Passing mutable default arguments to the function. If your function has a default argument like def f(row, cache={}), the cache persists across calls and can cause subtle bugs.
  • Confusing args and kwargs in apply. The args parameter expects a tuple for positional arguments. A common mistake is writing args=(0.15) instead of args=(0.15,), which passes a float rather than a single-element tuple.
  • Assuming np.vectorize gives true vectorized performance. Despite the name, np.vectorize is a convenience wrapper, not a performance tool. For genuine speedups, rewrite the logic using NumPy array operations or use libraries like Numba.

Summary

  • Use df.apply(func, axis=1) with a lambda or named function when you need row-wise access to multiple columns and the logic is too complex for vectorization.
  • Use the args parameter on Series.apply() to pass additional fixed arguments to your function.
  • Use np.vectorize for cleaner syntax when applying a simple multi-argument function across columns, but do not expect major speed gains.
  • Use df.assign with vectorized column operations for the best performance on large DataFrames.
  • Always benchmark with %timeit before committing to an approach, especially when working with DataFrames that have more than 10,000 rows.

Free course
Beginner
7 lessons
2 hours
Tackling System Design Interview Problems

A short course that equips you with the skills to approach system design interviews methodically.

Start the free course
Basis Lab
No course covers this one yet

Basis Lab writes one for you from a sentence about what you want to be able to do, then teaches it and checks you understood. Your first course is free.

Build my course
ML System Design practice on Codemia

Design recommenders, ranking systems and training pipelines the way ML interviews actually ask for them, with worked solutions.

Practice ML system design