Introduction

str.join is the standard Python tool for combining many strings efficiently. Its speed comes from avoiding the repeated allocation and copying that happen when you build a long result with + inside a loop.

Why Repeated + Can Be Expensive

Strings in Python are immutable. Every concatenation creates a new string, which means the characters already produced may need to be copied again and again.

1parts = ["ab"] * 5
2result = ""
3
4for part in parts:
5    result += part
6
7print(result)

With short inputs this is fine, but as the result grows, the total amount of copying can become quadratic in the total output size. That is why join is the idiomatic solution when you already have all the pieces.

The High-Level Strategy Used by str.join

At a high level, CPython handles sep.join(iterable) in two logical phases:

1. Read the iterable and verify that every element is a string.
2. Compute the total output size, allocate the destination once, then copy each piece and separator into that buffer.

That is what gives the operation linear behavior in the total number of characters copied.

The implementation includes a few fast paths:

• an empty iterable returns an empty string
• a single exact string item can often be returned directly
• the separator length is accounted for once during size calculation

The important idea is not the exact C function names. It is the allocation strategy: join knows the final size before it starts writing the output.

A Mental Model of the Internal Algorithm

The behavior is roughly equivalent to this conceptual implementation:

1def conceptual_join(separator, items):
2    items = list(items)
3
4    for item in items:
5        if not isinstance(item, str):
6            raise TypeError("sequence item is not a string")
7
8    total_length = sum(len(item) for item in items)
9    total_length += len(separator) * max(len(items) - 1, 0)
10
11    # CPython does not build the result exactly this way in Python,
12    # but it does allocate once after computing the final size.
13    output = []
14    for index, item in enumerate(items):
15        if index:
16            output.append(separator)
17        output.append(item)
18    return "".join(output)

The real interpreter implementation is in C and avoids Python-level overhead, but the shape is similar: validate, size, allocate once, copy once.

One subtle detail is that join may need to materialize the iterable first. If you pass a generator, CPython still must inspect all items to know the final size. That means join is linear, but not streaming.

Compare join with Loop Concatenation

Here is a simple benchmark:

1import timeit
2
3setup = """
4parts = ["hello"] * 10000
5"""
6
7plus_time = timeit.timeit(
8    "result = ''\nfor part in parts:\n    result += part",
9    setup=setup,
10    number=100,
11)
12
13join_time = timeit.timeit(
14    "result = ''.join(parts)",
15    setup=setup,
16    number=100,
17)
18
19print(f"plus: {plus_time:.4f}")
20print(f"join: {join_time:.4f}")

On typical CPython builds, join is clearly faster for large numbers of pieces. You may see optimizations for += in narrow cases, but they are implementation details and not something to rely on for predictable performance.

Type Constraints Matter

join requires strings. It does not silently convert arbitrary objects.

1values = ["user:", 42]
2
3try:
4    print(" ".join(values))
5except TypeError as exc:
6    print(exc)

If conversion is intentional, do it explicitly:

values = ["user:", 42]
print(" ".join(map(str, values)))

This explicitness keeps the method fast and keeps accidental type bugs visible.

Common Pitfalls

• Building strings with + inside a long loop when all pieces are already available.
• Forgetting that join requires strings and raises TypeError for other element types.
• Assuming join streams generator output directly. It still needs enough information to determine total size.
• Overgeneralizing microbenchmarks. Small examples can hide the advantage that appears with larger inputs.

Summary

• 'str.join is efficient because it computes the final size before writing the result.'
• CPython avoids repeated reallocations by allocating the output buffer once.
• The operation is linear in the total number of characters copied.
• 'join is the right tool when you already have many string fragments.'
• Non-string items must be converted explicitly if that behavior is desired.