pandas
DataFrame
groupby
count
Python

Pandas DataFrame Groupby two columns and get counts

Master System Design with Codemia

Enhance your system design skills with over 120 practice problems, detailed solutions, and hands-on exercises.

Introduction

Counting rows by combinations of two columns is one of the most common pandas aggregations. The core operation is groupby([...]).size(), which returns frequency per key pair. This pattern powers analytics use cases such as event counts by country and device, order counts by region and channel, or error counts by service and status. While the one-liner is simple, production workflows often need sorted output, missing-category handling, and pivoted formats for reporting. This guide covers robust patterns for two-column group counts with practical code.

Basic Two-Column Count

Start with groupby and size:

python
1import pandas as pd
2
3df = pd.DataFrame({
4    "team": ["A", "A", "B", "B", "B", "C"],
5    "result": ["win", "loss", "win", "win", "loss", "win"],
6})
7
8counts = (
9    df.groupby(["team", "result"])
10      .size()
11      .reset_index(name="count")
12)
13
14print(counts)

reset_index(name="count") converts the grouped Series into a clean dataframe ready for joins and visualization.

Sort and Filter Counts

For dashboards, sort by frequency and optionally filter small groups:

python
1top_counts = (
2    df.groupby(["team", "result"])
3      .size()
4      .reset_index(name="count")
5      .sort_values("count", ascending=False)
6)
7
8filtered = top_counts[top_counts["count"] >= 2]

This keeps high-signal combinations visible while suppressing noisy long tails.

Include Missing Category Combinations

By default, only observed combinations appear. If you need all category combinations (including zero counts), define categorical dtypes and use observed=False (or reindex against a complete cartesian index).

python
1team_cat = pd.CategoricalDtype(categories=["A", "B", "C", "D"])
2res_cat = pd.CategoricalDtype(categories=["win", "loss"])
3
4df2 = df.copy()
5df2["team"] = df2["team"].astype(team_cat)
6df2["result"] = df2["result"].astype(res_cat)
7
8all_counts = (
9    df2.groupby(["team", "result"], observed=False)
10       .size()
11       .reset_index(name="count")
12)

Now you can report zero rows explicitly, which is often required in business reporting.

Pivot to Matrix Form

If you want a cross-tab table, pivot the result:

python
1matrix = (
2    df.groupby(["team", "result"])
3      .size()
4      .unstack(fill_value=0)
5)
6
7print(matrix)

Equivalent shortcut:

python
matrix2 = pd.crosstab(df["team"], df["result"])

crosstab is concise for pure frequency matrices; groupby is more flexible when you need custom pipelines.

Practical Verification Workflow

A reliable way to avoid regressions is to validate the solution in three passes: baseline, controlled change, and repeatability check. First, capture a baseline outcome before you apply fixes. This could be a failing command, a wrong output sample, a stack trace, or a screenshot of current behavior. Second, apply one focused change and rerun exactly the same checks so you can attribute improvements to a specific edit. Third, rerun the checks multiple times or with slightly different inputs to ensure the fix is not accidental or data-specific.

A lightweight template you can adapt for most projects looks like this:

bash
1# 1) reproduce current behavior
2./run_example.sh > before.txt
3
4# 2) apply your change
5# edit config/code based on this article
6
7# 3) verify behavior after change
8./run_example.sh > after.txt
9diff -u before.txt after.txt

If your environment involves tests, add at least one focused regression test that would fail before the fix and pass after it. This turns a one-time troubleshooting success into a durable maintenance improvement, which is especially important when teams rotate ownership or upgrade dependencies later.

Common Pitfalls

  • Forgetting reset_index(name="count"), leaving a Series that is awkward to merge later.
  • Assuming missing category combinations are counted automatically.
  • Using count() instead of size() and unintentionally depending on non-null columns.
  • Sorting before grouping and expecting grouped output order guarantees.
  • Ignoring dtype/category setup when report requirements include zero-count groups.

Summary

To count rows by two pandas columns, use groupby([col1, col2]).size() and convert to dataframe form with reset_index. Add sorting, category control, and pivoting based on reporting needs. With these patterns, two-key frequency analysis remains simple, accurate, and ready for downstream analytics.

For recurring reports, encapsulate this counting logic in a reusable helper so teams produce consistent grouping output across notebooks, jobs, and dashboards.


Course illustration
Course illustration

All Rights Reserved.