Introduction

Finding duplicates in a Python list sounds simple, but there are two different questions hidden inside it. You may want a list of values that appear more than once, or you may want every extra occurrence beyond the first one. Choosing the right approach depends on which result you actually need.

Counting Values With Counter

If you only care about which values are duplicated, collections.Counter is usually the clearest tool. It counts each item once, then you filter counts greater than one.

1from collections import Counter
2
3items = ["apple", "banana", "apple", "orange", "banana", "banana"]
4counts = Counter(items)
5
6duplicates = [value for value, count in counts.items() if count > 1]
7print(duplicates)  # ['apple', 'banana']

This approach is concise and efficient. It runs in linear time for hashable items and communicates intent well. It is especially useful in reporting code, validation logic, and data cleaning jobs where the final output should list each duplicated value once.

One tradeoff is ordering. Counter preserves insertion order in modern Python because it is based on a dictionary, but the result still represents unique duplicated values rather than every repeated appearance.

Preserving Duplicate Discovery Order

Sometimes you want duplicates in the order they are first recognized. A seen set plus a duplicates list is a good pattern for that.

1items = ["a", "b", "a", "c", "b", "d", "b"]
2
3seen = set()
4reported = set()
5duplicates = []
6
7for item in items:
8    if item in seen and item not in reported:
9        duplicates.append(item)
10        reported.add(item)
11    else:
12        seen.add(item)
13
14print(duplicates)  # ['a', 'b']

Here, seen tracks values encountered before, while reported prevents the same duplicated value from being added multiple times. This is often the best answer when someone says "create another list with the duplicates" and expects each duplicated value only once, in a stable and readable order.

Capturing Every Extra Occurrence

In some workflows, the duplicates list should include every repeated occurrence after the first. That produces a different result:

1items = ["a", "b", "a", "c", "b", "d", "b"]
2
3seen = set()
4extra_occurrences = []
5
6for item in items:
7    if item in seen:
8        extra_occurrences.append(item)
9    else:
10        seen.add(item)
11
12print(extra_occurrences)  # ['a', 'b', 'b']

This version is useful when you want to inspect how often duplication happens in the original stream. For example, if duplicate user ids represent bad rows in an import file, listing every extra occurrence can help you locate each bad record.

Choosing the Right Data Structure

These examples assume the list contains hashable values such as strings, numbers, or tuples. Hashability matters because sets and dictionary-based tools rely on hashing.

If the list contains unhashable values such as nested lists or dictionaries, you need a different strategy. One option is to convert items into a stable hashable form before counting them.

1from collections import Counter
2
3rows = [[1, 2], [3, 4], [1, 2]]
4normalized = [tuple(row) for row in rows]
5
6counts = Counter(normalized)
7duplicates = [list(value) for value, count in counts.items() if count > 1]
8
9print(duplicates)  # [[1, 2]]

The normalization step keeps the logic simple while making the data compatible with Counter and sets.

Common Pitfalls

One common mistake is using list.count() inside a loop. That works for small lists, but it scans the list repeatedly and turns a linear task into a much slower quadratic one.

Another pitfall is failing to define what "duplicates" means before writing code. Do you want unique repeated values, every repeated occurrence, or duplicate indexes? Those are different outputs.

It is also easy to lose ordering accidentally by converting the whole result into a set too early. Sets are great for fast membership checks, but they are the wrong final structure if output order matters to a user or test.

Summary

• Decide whether you want unique duplicated values or every extra repeated occurrence.
• Use Counter when you want a clear count-based solution.
• Use a seen set when you want to preserve discovery order efficiently.
• Normalize unhashable values before counting them with sets or Counter.
• Avoid list.count() inside loops for large inputs because it is unnecessarily slow.