Introduction

Generating permutations looks simple until duplicate values enter the input. At that point, a naive loop produces repeated arrangements, which wastes time and makes downstream logic harder to trust.

Why Duplicate Permutations Appear

If all input values are distinct, a standard permutation routine yields n! arrangements with no repeats. The trouble starts when the input contains repeated values such as [1, 1, 2]. A naive algorithm treats the first 1 and second 1 as different positions even though they produce the same final ordering.

One brute-force fix is to generate every permutation and store the results in a set. That works for tiny inputs, but it does extra work and consumes memory for permutations you never wanted in the first place. A better approach is to avoid creating duplicates during the search.

Backtracking for Unique Permutations

The common pattern is:

1. Sort the input so duplicates are adjacent.
2. Track which positions are already used.
3. Skip a value if it is the same as the previous value and the previous value has not been used in the current branch.

That skip rule prevents the algorithm from choosing equivalent values in interchangeable positions.

1def unique_permutations(values):
2    values = sorted(values)
3    used = [False] * len(values)
4    current = []
5
6    def backtrack():
7        if len(current) == len(values):
8            yield current.copy()
9            return
10
11        for i, value in enumerate(values):
12            if used[i]:
13                continue
14
15            if i > 0 and values[i] == values[i - 1] and not used[i - 1]:
16                continue
17
18            used[i] = True
19            current.append(value)
20            yield from backtrack()
21            current.pop()
22            used[i] = False
23
24    yield from backtrack()
25
26
27for item in unique_permutations([1, 1, 2]):
28    print(item)

Output:

[1, 1, 2]
[1, 2, 1]
[2, 1, 1]

This solution scales much better than generating duplicates first and filtering later.

Looping Through Several Input Sets

Many real problems are not "generate one set of permutations." They are "for each input set, generate the distinct orderings and process them." In that case, keep the permutation logic separate from the outer loop.

1datasets = [
2    ["A", "A", "B"],
3    [1, 2, 3],
4    ["x", "x", "y", "y"],
5]
6
7for index, dataset in enumerate(datasets, start=1):
8    print(f"Dataset {index}: {dataset}")
9    for perm in unique_permutations(dataset):
10        print("  ", perm)

Structuring the code this way makes it clear where one dataset ends and the next begins. It also lets you replace the inner processing step with validation, scoring, or persistence logic without rewriting the generator.

When itertools.permutations Is Not Enough

Python’s itertools.permutations is fast and convenient, but it does not remove duplicates for repeated input values. You can wrap it in a set, but then you lose streaming behavior and pay an avoidable cost for larger inputs.

1from itertools import permutations
2
3values = [1, 1, 2]
4unique = {tuple(p) for p in permutations(values)}
5print(sorted(unique))

This is acceptable for quick experiments. For production logic or larger search spaces, a duplicate-aware backtracking solution is usually the better design.

Choosing the Right Representation

If you only need to iterate once, yielding lists or tuples is enough. If you need to compare or store permutations, tuples are often a better output type because they are hashable and safe to use as dictionary keys. That choice depends on what the next stage of your program needs.

Another useful technique is to stop early. If you are searching for the first permutation that satisfies a condition, do not materialize the whole result set. Iterate lazily and break when you find a match.

Common Pitfalls

• Generating all permutations and then deduplicating with a set is easy to write but inefficient for repeated values.
• Forgetting to sort the input breaks the duplicate-skip rule because equal values are no longer adjacent.
• Reusing the same list object without copying it causes every saved permutation to end up identical.
• Materializing every permutation in memory can become expensive very quickly.
• Mixing the outer dataset loop with the inner backtracking logic makes the code harder to test and reuse.

Summary

• Duplicate values require more than a plain permutation loop.
• A sorted-input backtracking algorithm can avoid repeated arrangements before they are created.
• Keep dataset iteration separate from permutation generation for cleaner code.
• 'itertools.permutations is useful, but it does not solve uniqueness on its own.'
• Prefer lazy iteration when you do not need the entire result set at once.