Introduction

One of the most frequent operations in data analysis is pulling values from one column based on a condition in another column. In pandas this is called conditional selection, and there are several ways to do it. Knowing which method to reach for and why it fits your situation keeps your code both readable and efficient.

Boolean Indexing: The Foundation

Boolean indexing is the most common and idiomatic way to filter a DataFrame. You create a boolean Series by applying a condition to one column, then use it to select rows.

1import pandas as pd
2
3df = pd.DataFrame({
4    'name': ['Alice', 'Bob', 'Charlie', 'Diana'],
5    'age': [30, 25, 35, 28],
6    'city': ['New York', 'Boston', 'Chicago', 'Boston']
7})
8
9# Extract names of people older than 28
10result = df[df['age'] > 28]['name']
11print(result)
12# 0      Alice
13# 2    Charlie
14# Name: name, dtype: object

The expression df['age'] > 28 produces a boolean Series (True, False, True, False), and df[...] keeps only the rows where the value is True. Chaining ['name'] at the end selects just that column from the filtered rows.

Using .loc[] for Label-Based Selection

The .loc[] accessor lets you filter rows and select columns in a single expression. This is generally preferred over chained indexing because it avoids potential SettingWithCopyWarning issues and is more explicit.

1# Single condition, single column
2names = df.loc[df['age'] > 28, 'name']
3
4# Single condition, multiple columns
5subset = df.loc[df['city'] == 'Boston', ['name', 'age']]
6print(subset)
7#    name  age
8# 1   Bob   25
9# 3  Diana   28

The first argument to .loc[] is the row selector (a boolean Series), and the second is the column selector (a label or list of labels).

Multiple Conditions with & and |

When filtering on more than one column, combine conditions using & (and) and | (or). Each individual condition must be wrapped in parentheses because of Python's operator precedence rules.

1# People in Boston who are older than 26
2result = df.loc[(df['city'] == 'Boston') & (df['age'] > 26), 'name']
3print(result)
4# 3    Diana
5# Name: name, dtype: object
6
7# People in New York OR younger than 27
8result = df.loc[(df['city'] == 'New York') | (df['age'] < 27), 'name']
9print(result)
10# 0    Alice
11# 1      Bob
12# Name: name, dtype: object

Do not use Python's and and or keywords here. They operate on scalar truth values and will raise a ValueError when applied to Series.

Using .query() for Readable Filters

The .query() method accepts a string expression, which can be more readable for complex conditions, especially when column names are simple.

1# Equivalent to df.loc[(df['city'] == 'Boston') & (df['age'] > 26)]
2result = df.query("city == 'Boston' and age > 26")['name']
3print(result)
4# 3    Diana
5# Name: name, dtype: object
6
7# Using variables with @
8min_age = 26
9result = df.query("age > @min_age")['name']

The @ prefix lets you reference Python variables inside the query string. This avoids awkward f-string concatenation.

Extracting a Single Scalar Value

When your filter returns exactly one row and you need the actual value rather than a Series, use .item(), .iloc[0], or .values[0].

1# Get Alice's age
2age = df.loc[df['name'] == 'Alice', 'age'].item()
3print(age)       # 30
4print(type(age)) # <class 'numpy.int64'>
5
6# Alternative approaches
7age = df.loc[df['name'] == 'Alice', 'age'].iloc[0]
8age = df.loc[df['name'] == 'Alice', 'age'].values[0]

The .item() method is the safest choice because it raises a ValueError if the result contains more or fewer than exactly one element, catching logic errors early. The .iloc[0] approach silently returns the first match when there are multiple rows.

Creating Conditional Columns with np.where

Sometimes you do not want to filter rows but instead create a new column whose values depend on another column. np.where works like a vectorized if-else statement.

1import numpy as np
2
3df['age_group'] = np.where(df['age'] >= 30, 'senior', 'junior')
4print(df)
5#       name  age      city age_group
6# 0    Alice   30  New York    senior
7# 1      Bob   25    Boston    junior
8# 2  Charlie   35   Chicago    senior
9# 3    Diana   28    Boston    junior

For more than two categories, use np.select:

1conditions = [
2    df['age'] < 26,
3    df['age'].between(26, 32),
4    df['age'] > 32
5]
6choices = ['young', 'mid', 'senior']
7
8df['category'] = np.select(conditions, choices, default='unknown')

Using .isin() for Multiple Value Matching

When you need to filter based on whether a column's value belongs to a set of values, .isin() is cleaner than chaining multiple | conditions.

1target_cities = ['Boston', 'Chicago']
2result = df.loc[df['city'].isin(target_cities), 'name']
3print(result)
4# 1        Bob
5# 2    Charlie
6# 3      Diana
7# Name: name, dtype: object

Common Pitfalls

• Using and/or instead of &/|: Python's logical operators do not work element-wise on Series. You will get a ValueError telling you the truth value of a Series is ambiguous.
• Forgetting parentheses around conditions: df[df['a'] > 1 & df['b'] < 5] is parsed as df[df['a'] > (1 & df['b']) < 5] due to operator precedence. Always wrap each condition in parentheses.
• Chained indexing triggering SettingWithCopyWarning: df[df['a'] > 1]['b'] = 10 may not modify df. Use df.loc[df['a'] > 1, 'b'] = 10 instead.
• Calling .item() on multi-row results: If your filter matches more than one row, .item() raises an error. Use .iloc[0] if you intentionally want just the first match, or check the result length first.
• Applying string methods without .str accessor: To filter on string patterns, use df['col'].str.contains('pattern'), not plain Python string operations on the Series.

Summary

• Boolean indexing (df[df['col'] > x]['target']) is the most fundamental filtering pattern in pandas.
• Prefer .loc[] over chained indexing to avoid SettingWithCopyWarning and to select rows and columns in one step.
• Combine multiple conditions with & and |, always wrapping each condition in parentheses.
• Use .query() for cleaner syntax when conditions are complex and column names are simple.
• Extract scalar values with .item() for safety, or .iloc[0] when you only need the first match.
• Use np.where or np.select to create new columns based on conditions in existing columns.