Introduction

itertools.permutations works on element positions, not element uniqueness. When input values repeat, different positional selections can produce visually identical tuples. That is why duplicates appear in the output even though the algorithm is behaving correctly.

Position-Based Permutation Logic

Consider input [1, 1, 2]. The two 1 values occupy different indices, so swapping them counts as a different positional permutation even if tuple values look the same.

1from itertools import permutations
2
3data = [1, 1, 2]
4all_perm = list(permutations(data))
5print(all_perm)
6print("count:", len(all_perm))

You get 3! = 6 positional permutations, including repeated tuples like (1, 1, 2).

Remove Duplicates with set

If unique value permutations are needed, wrap output with set.

1from itertools import permutations
2
3data = [1, 1, 2]
4unique_perm = sorted(set(permutations(data)))
5print(unique_perm)
6print("unique count:", len(unique_perm))

This is simple and often enough for small inputs.

Compute Unique Counts Without Enumerating All

You can compute theoretical unique count using multinomial logic.

For values with multiplicities n1, n2, ... in total length n, unique count is n! / (n1! * n2! * ...).

1import math
2from collections import Counter
3
4
5def unique_perm_count(seq):
6    n = len(seq)
7    counts = Counter(seq).values()
8    denom = 1
9    for c in counts:
10        denom *= math.factorial(c)
11    return math.factorial(n) // denom
12
13print(unique_perm_count([1, 1, 2]))

This avoids generating huge intermediate lists when only counts are required.

Memory-Aware Unique Generation

set(permutations(...)) can consume large memory. For bigger inputs, sort data and use backtracking that skips duplicate branches.

1
2def unique_permutations(nums):
3    nums = sorted(nums)
4    used = [False] * len(nums)
5    path = []
6
7    def backtrack():
8        if len(path) == len(nums):
9            yield tuple(path)
10            return
11
12        for i in range(len(nums)):
13            if used[i]:
14                continue
15            if i > 0 and nums[i] == nums[i - 1] and not used[i - 1]:
16                continue
17
18            used[i] = True
19            path.append(nums[i])
20            yield from backtrack()
21            path.pop()
22            used[i] = False
23
24    yield from backtrack()
25
26print(list(unique_permutations([1, 1, 2])))

This generates only unique permutations directly.

Practical Use Cases

In combinatorial search, distinct-value permutations are often what you want. In cryptographic or position-sensitive simulations, positional permutations may still be valid even with repeated values. Clarify requirement before choosing dedup strategy.

Also test both correctness and memory footprint because input size grows factorially.

Unique Permutations by Counter-Based Backtracking

For larger inputs, generating only unique permutations is more efficient than generating all and deduplicating.

1from collections import Counter
2
3def unique_perm_counter(seq):
4    counts = Counter(seq)
5    path = []
6    n = len(seq)
7
8    def backtrack():
9        if len(path) == n:
10            yield tuple(path)
11            return
12        for value in list(counts.keys()):
13            if counts[value] == 0:
14                continue
15            counts[value] -= 1
16            path.append(value)
17            yield from backtrack()
18            path.pop()
19            counts[value] += 1
20
21    yield from backtrack()
22
23print(list(unique_perm_counter([1, 1, 2])))

This technique scales better in memory than set(permutations(...)) for repeated values.

Choosing Representation for Downstream Tasks

If permutations feed another algorithm, think about representation early. Tuples are hashable and work with sets. Lists are mutable and friendlier for in-place edits. Representation choice affects both speed and API compatibility in later pipeline stages.

Benchmarking Dedup Strategies

For small lists, set(permutations(...)) is usually fine. For repeated runs or larger multisets, direct unique generation often wins in memory and runtime. Measure both approaches with representative input sizes before choosing a production implementation.

Common Pitfalls

• Assuming itertools.permutations deduplicates values automatically.
• Using set on very large permutation spaces and exhausting memory.
• Confusing positional uniqueness with value uniqueness.
• Computing full permutations when only count is needed.
• Forgetting factorial growth and running expensive brute-force loops.

Summary

• itertools.permutations is index-position based, so duplicates with repeated values are expected.
• Use set for simple deduplication on small inputs.
• Use multinomial formula for unique counts without full generation.
• Use skip-duplicate backtracking for memory-aware unique generation.
• Choose algorithm based on whether you need positional or value-level uniqueness.