Pandas DataFrame Groupby two columns and get counts
ML System Design practice on Codemia
Design recommenders, ranking systems and training pipelines the way ML interviews actually ask for them, with worked solutions.
Introduction
Counting rows by combinations of two columns is one of the most common pandas aggregations. The core operation is groupby([...]).size(), which returns frequency per key pair. This pattern powers analytics use cases such as event counts by country and device, order counts by region and channel, or error counts by service and status. While the one-liner is simple, production workflows often need sorted output, missing-category handling, and pivoted formats for reporting. This guide covers robust patterns for two-column group counts with practical code.
Basic Two-Column Count
Start with groupby and size:
reset_index(name="count") converts the grouped Series into a clean dataframe ready for joins and visualization.
Sort and Filter Counts
For dashboards, sort by frequency and optionally filter small groups:
This keeps high-signal combinations visible while suppressing noisy long tails.
Include Missing Category Combinations
By default, only observed combinations appear. If you need all category combinations (including zero counts), define categorical dtypes and use observed=False (or reindex against a complete cartesian index).
Now you can report zero rows explicitly, which is often required in business reporting.
Pivot to Matrix Form
If you want a cross-tab table, pivot the result:
Equivalent shortcut:
crosstab is concise for pure frequency matrices; groupby is more flexible when you need custom pipelines.
Practical Verification Workflow
A reliable way to avoid regressions is to validate the solution in three passes: baseline, controlled change, and repeatability check. First, capture a baseline outcome before you apply fixes. This could be a failing command, a wrong output sample, a stack trace, or a screenshot of current behavior. Second, apply one focused change and rerun exactly the same checks so you can attribute improvements to a specific edit. Third, rerun the checks multiple times or with slightly different inputs to ensure the fix is not accidental or data-specific.
A lightweight template you can adapt for most projects looks like this:
If your environment involves tests, add at least one focused regression test that would fail before the fix and pass after it. This turns a one-time troubleshooting success into a durable maintenance improvement, which is especially important when teams rotate ownership or upgrade dependencies later.
Common Pitfalls
- Forgetting
reset_index(name="count"), leaving a Series that is awkward to merge later. - Assuming missing category combinations are counted automatically.
- Using
count()instead ofsize()and unintentionally depending on non-null columns. - Sorting before grouping and expecting grouped output order guarantees.
- Ignoring dtype/category setup when report requirements include zero-count groups.
Summary
To count rows by two pandas columns, use groupby([col1, col2]).size() and convert to dataframe form with reset_index. Add sorting, category control, and pivoting based on reporting needs. With these patterns, two-key frequency analysis remains simple, accurate, and ready for downstream analytics.
For recurring reports, encapsulate this counting logic in a reusable helper so teams produce consistent grouping output across notebooks, jobs, and dashboards.
Related reading
- Pandas DataFrame RangeIndex
- Pandas DataFrame replace all values in a column, based on condition
- pandas DataFrame replace nan values with average of columns
- pandas dataframe select columns in multiindex
- Pandas DataFrame to List of Dictionaries
- Pandas DataFrame to List of Lists
- Pandas drop a level from a multi-level column index?
- Pandas every nth row
.png&w=3840&q=75)
Tackling System Design Interview Problems
A short course that equips you with the skills to approach system design interviews methodically.
Start the free courseTrack what you have practised
A free account saves your progress, solutions and study plan across every problem on Codemia.
ML System Design practice on Codemia
Design recommenders, ranking systems and training pipelines the way ML interviews actually ask for them, with worked solutions.