Introduction

When using os.walk, the correct way to exclude directories is to modify the dirnames list in place while walking top-down. If you only filter results after traversal, Python may still recurse into directories you meant to skip, which wastes time and defeats the purpose of exclusion.

How os.walk decides where to recurse

When topdown=True, os.walk yields three values for each directory:

• 'dirpath'
• 'dirnames'
• 'filenames'

The important detail is that dirnames is not just informational. It is the list Python uses to decide which child directories to visit next.

1import os
2
3root = "/path/to/project"
4excluded = {".git", "node_modules", "__pycache__", ".venv"}
5
6for dirpath, dirnames, filenames in os.walk(root, topdown=True):
7    dirnames[:] = [d for d in dirnames if d not in excluded]
8
9    for filename in filenames:
10        print(os.path.join(dirpath, filename))

The slice assignment is crucial.

Why dirnames[:] = ... matters

This works:

dirnames[:] = [d for d in dirnames if d not in excluded]

This does not control recursion the same way:

dirnames = [d for d in dirnames if d not in excluded]

The second version only rebinds the local variable. It does not mutate the original list that os.walk is using internally.

That is the most important detail in this whole topic.

Exact names versus patterns

If exact directory names are not enough, use pattern matching.

1import fnmatch
2import os
3
4patterns = ["*.egg-info", "build", "dist", ".mypy_cache"]
5
6for dirpath, dirnames, filenames in os.walk("/path/to/project", topdown=True):
7    dirnames[:] = [
8        d for d in dirnames
9        if not any(fnmatch.fnmatch(d, pattern) for pattern in patterns)
10    ]

This is useful when generated folders follow naming patterns rather than one fixed name.

Excluding by full path

Sometimes the directory name alone is not enough because the same name may appear in places you do want to visit.

1import os
2
3root = "/path/to/project"
4excluded_paths = {
5    os.path.normpath("/path/to/project/data/cache"),
6    os.path.normpath("/path/to/project/vendor/tmp"),
7}
8
9for dirpath, dirnames, filenames in os.walk(root, topdown=True):
10    dirnames[:] = [
11        d for d in dirnames
12        if os.path.normpath(os.path.join(dirpath, d)) not in excluded_paths
13    ]

This gives you much more precise control.

A reusable helper

1from pathlib import Path
2import os
3
4
5def walk_files(base: Path, excluded_dirs: set[str]):
6    for dirpath, dirnames, filenames in os.walk(base, topdown=True):
7        dirnames[:] = [d for d in dirnames if d not in excluded_dirs]
8        for filename in filenames:
9            yield Path(dirpath) / filename

This keeps the exclusion logic consistent across scripts instead of re-implementing it each time.

Why filtering afterward is worse

If you walk every directory and only later ignore paths you do not care about, you still pay the recursion cost. That can be slow in repositories with large dependency trees, build outputs, or virtual environments.

In other words, post-filtering hides unwanted results from your output, but it does not prevent the wasted traversal work that caused the slowdown in the first place.

So the rule is simple: prune recursion early rather than post-filtering later.

Common Pitfalls

A common mistake is reassigning dirnames instead of mutating it in place.

That tiny-looking difference is exactly why many first attempts appear correct in code review but still recurse into every directory at runtime.

Another mistake is using topdown=False, which removes the opportunity to prune child recursion before it happens.

A third mistake is filtering file results afterward and assuming excluded directories were never visited.

Summary

• Exclude directories by modifying dirnames in place.
• Use topdown=True so recursion can be pruned before descending.
• Use exact names, patterns, or full paths depending on the use case.
• Do not just rebind dirnames; mutate it with slice assignment.
• Early pruning is more efficient than filtering after traversal.
• Mutate dirnames in place if you want recursion itself to change.