How do I use itertools.groupby?
Interview Questions practice on Codemia
Over 8,000 real interview questions from top companies, searchable by company and role.
Introduction
itertools.groupby is useful when you want to group consecutive items that share the same key. The most important thing to remember is that it does not do SQL-style global grouping on arbitrary input. It groups runs of adjacent values, which makes ordering central to correct use.
Understand What groupby Returns
groupby(iterable, key=...) yields pairs of:
- the group key
- an iterator over the items in that consecutive group
Example:
This produces two separate books groups because the books rows are not adjacent in the original input.
Sort First When You Need Global Grouping
If you want one group per key across the whole dataset, sort by the same key first:
The sort key and the grouping key should match. Sorting by one field and grouping by another is a common cause of confusing output.
Consume Group Iterators Immediately
Each group iterator shares the underlying input stream. Once the outer loop advances, the previous group is effectively gone.
If you need to reuse the grouped items, materialize them immediately:
That is the right pattern when you need both the items and a derived statistic such as the count.
Aggregate While Grouping
groupby is great for sorted record streams where you want streaming-style aggregation:
If the groups are large, you can aggregate directly from the iterator instead of materializing the whole group into a list.
Know When a Dictionary Is Better
groupby shines when the input is already sorted or naturally ordered. If the data arrives in arbitrary order and you need global grouping without sorting, a dictionary-based accumulation approach may be simpler.
That does not make groupby wrong. It just means its strength is ordered grouping, not universal aggregation across random input.
Nested Grouping Works in Stages
You can group by multiple levels by sorting on a composite key and then grouping in stages:
This keeps the processing stream-oriented and avoids unnecessary intermediate structures.
Common Pitfalls
The biggest mistake is expecting groupby to combine matching keys across unsorted input.
Another issue is sorting by one key and grouping by another, which fragments the data in surprising ways.
People also try to reuse group iterators after the outer loop moves on, which does not work because the iterators are single-pass.
Summary
- '
itertools.groupbygroups consecutive items, not arbitrary matching keys across unsorted input.' - Sort first when you need one global group per key.
- Consume or materialize each group immediately because the iterators are single-pass.
- Use
groupbyfor ordered data and dictionary accumulation for unordered aggregation. - Keep the sort key and the group key aligned.
Related reading
- How do I use method overloading in Python?
- How do I use np.newaxis?
- How do I use Pandas group-by to get the sum?
- How do I use raw_input in Python 3?
- How do I use sklearn CountVectorizer with both 'word' and 'char' analyzer? - python
- How do I use the group_by_window function in TensorFlow
- How do I use threading in Python?
- How do I use threading in Python?
.png&w=3840&q=75)
Tackling System Design Interview Problems
A short course that equips you with the skills to approach system design interviews methodically.
Start the free courseTrack what you have practised
A free account saves your progress, solutions and study plan across every problem on Codemia.
Interview Questions practice on Codemia
Over 8,000 real interview questions from top companies, searchable by company and role.