Introduction

When you have a dictionary where each key maps to a list of possible values, a common task is generating every combination of those values -- one value chosen from each key. This is called the Cartesian product, and it appears in scenarios like hyperparameter tuning, configuration testing, and A/B experiment design. Python's itertools.product makes this straightforward.

Understanding the Problem

Consider a dictionary that represents configurable options:

1options = {
2    "color": ["red", "blue"],
3    "size": ["S", "M", "L"],
4    "material": ["cotton", "polyester"],
5}

You want every possible combination: red-S-cotton, red-S-polyester, red-M-cotton, and so on. With 2 colors, 3 sizes, and 2 materials, that gives you 2 x 3 x 2 = 12 combinations. Each combination picks exactly one value from each key.

Using itertools.product

The itertools.product function computes the Cartesian product of input iterables. To turn dictionary values into combinations and then map them back to their keys, combine it with zip:

1import itertools
2
3options = {
4    "color": ["red", "blue"],
5    "size": ["S", "M", "L"],
6    "material": ["cotton", "polyester"],
7}
8
9keys = options.keys()
10values = options.values()
11
12combinations = [
13    dict(zip(keys, combo))
14    for combo in itertools.product(*values)
15]
16
17for c in combinations:
18    print(c)

Output:

1{'color': 'red', 'size': 'S', 'material': 'cotton'}
2{'color': 'red', 'size': 'S', 'material': 'polyester'}
3{'color': 'red', 'size': 'M', 'material': 'cotton'}
4...
5{'color': 'blue', 'size': 'L', 'material': 'polyester'}

The *values unpacking passes each list as a separate argument to itertools.product. The zip(keys, combo) call pairs each key with the corresponding value from the current combination tuple, and dict() turns those pairs into a dictionary.

A Pure List Comprehension Approach

If you prefer to avoid importing itertools, you can build the Cartesian product with nested comprehensions, though it only works cleanly when you know the number of keys in advance:

1options = {"color": ["red", "blue"], "size": ["S", "M", "L"]}
2
3combinations = [
4    {"color": c, "size": s}
5    for c in options["color"]
6    for s in options["size"]
7]

This approach is readable for two or three keys but becomes unwieldy for larger dictionaries. The itertools.product method scales to any number of keys without changing the code structure.

Real-World Example: Hyperparameter Grid Search

One of the most common uses for this pattern is generating a parameter grid for machine learning experiments:

1import itertools
2
3param_grid = {
4    "learning_rate": [0.001, 0.01, 0.1],
5    "batch_size": [16, 32, 64],
6    "optimizer": ["adam", "sgd"],
7}
8
9configs = [
10    dict(zip(param_grid.keys(), combo))
11    for combo in itertools.product(*param_grid.values())
12]
13
14print(f"Total configurations to test: {len(configs)}")
15
16for config in configs:
17    # train_model(**config)
18    print(config)

This generates 3 x 3 x 2 = 18 configurations. Each one can be passed directly as keyword arguments to a training function.

Real-World Example: A/B Test Variant Matrix

Another practical application is generating all variants for a multivariate A/B test:

1import itertools
2
3test_variants = {
4    "button_color": ["green", "orange"],
5    "headline": ["Free Trial", "Get Started"],
6    "layout": ["single-column", "two-column"],
7}
8
9variants = [
10    dict(zip(test_variants.keys(), combo))
11    for combo in itertools.product(*test_variants.values())
12]
13
14for i, v in enumerate(variants):
15    print(f"Variant {i + 1}: {v}")

This produces 8 variants, each representing a unique combination of visual elements to test.

Filtering Combinations

In many cases you do not want every combination. You can filter results with a condition:

1import itertools
2
3options = {
4    "cpu": [2, 4, 8],
5    "memory_gb": [4, 8, 16, 32],
6    "storage_gb": [100, 500],
7}
8
9# Only keep combinations where memory is at least 2x the CPU count
10valid_configs = [
11    dict(zip(options.keys(), combo))
12    for combo in itertools.product(*options.values())
13    if combo[1] >= 2 * combo[0]  # memory_gb >= 2 * cpu
14]
15
16print(f"{len(valid_configs)} valid out of {3 * 4 * 2} total")

Filtering inside the list comprehension keeps the code concise and avoids building an intermediate list of all combinations.

Memory Considerations for Large Spaces

The Cartesian product grows multiplicatively. A dictionary with 10 keys each having 10 values produces 10 billion combinations. Building a list of all of them will exhaust your memory. Use the iterator form of itertools.product instead of materializing everything at once:

1import itertools
2
3large_options = {f"param_{i}": list(range(5)) for i in range(8)}
4# 5^8 = 390,625 combinations
5
6keys = list(large_options.keys())
7for combo in itertools.product(*large_options.values()):
8    config = dict(zip(keys, combo))
9    # Process each config one at a time
10    pass

By iterating directly over itertools.product instead of wrapping it in a list comprehension, you process one combination at a time and keep memory usage constant.

Common Pitfalls

• Materializing huge products into a list causes memory errors. Use the iterator directly when the combination space is large.
• Relying on dictionary insertion order is safe in Python 3.7+ but will produce unpredictable key-value pairings in older versions. Use collections.OrderedDict if you must support Python 3.6 or earlier.
• Confusing itertools.product with itertools.combinations -- product gives the Cartesian product (one pick from each iterable), while combinations gives subsets of a single iterable.
• Forgetting the * unpacking in itertools.product(*values) passes the entire list of lists as one argument instead of separate arguments, producing wrong results.
• Hardcoding key names in list comprehensions breaks when keys change. The dict(zip(keys, combo)) pattern is dynamic and adapts to any dictionary.

Summary

• Use itertools.product(*dict.values()) to compute the Cartesian product of all value lists.
• Reconstruct dictionaries with dict(zip(dict.keys(), combo)) to map each combination back to its keys.
• Filter combinations inside the comprehension to avoid building a full intermediate list.
• For large combination spaces, iterate over the product directly instead of materializing it into a list.
• This pattern is widely used in hyperparameter grid search, A/B test variant generation, and configuration testing.