1import pandas as pd
2
3
4df = pd.DataFrame(
5    {
6        "name": ["Alice Blue", "Bob Redford", "Charlie Grey", None],
7        "city": ["Toronto", "Boston", "Calgary", "Berlin"],
8    }
9)
10
11filtered = df[df["name"].str.contains("Red", na=False)]
12print(filtered)

filtered = df[df["name"].str.contains("red", case=False, regex=False, na=False)]
print(filtered)

1starts = df[df["name"].str.startswith("Alice", na=False)]
2ends = df[df["name"].str.endswith("Grey", na=False)]
3
4print(starts)
5print(ends)

mask = (
    df["name"].str.contains("o", case=False, na=False)
| df["city"].str.contains("o", case=False, na=False) ) print(df[mask]) ``` Using masks explicitly keeps the logic readable when multiple text conditions are involved. It is also a good habit to normalize the column before matching when data quality is uneven. Converting values to string, trimming spaces, or standardizing case once up front is often cleaner than embedding those fixes into every filter expression. ## Regex Filtering When You Actually Need It Regular expressions are still useful when the matching rule is more complex than a plain substring. For example, filter names that start with `A`, `B`, or `C`: ```python regex_filtered = df[df["name"].str.contains(r"^[ABC]", na=False)] print(regex_filtered) ``` Use regex deliberately. Do not leave it on by default if the search text comes from users and should be treated literally. For larger data-cleaning pipelines, it is often worth storing the mask in a named variable instead of filtering inline. That makes debugging easier because you can inspect how many rows matched before applying the final selection. That same pattern also helps when multiple substring rules need to be combined with `&` and `|`, because each intermediate mask can be tested independently. ## Common Pitfalls The most common mistake is forgetting `na=False`. Missing values then create a mask with null entries, which often causes confusing behavior. Another mistake is forgetting that `str.contains` uses regex by default. A pattern such as `.` or `+` has special meaning unless `regex=False` is set. A third mistake is filtering non-string columns without converting them first. If the column contains mixed types, clean or cast the data before applying string methods. ## Summary - Use `str.contains` for general substring filtering in pandas. - Add `na=False` so missing values do not break the filter mask. - Use `case=False` for case-insensitive matching. - Set `regex=False` when the search term should be treated as a literal string. - Prefer `startswith` and `endswith` when the matching rule is positional rather than “contains anywhere.”