Introduction

pathlib gives Python a cleaner, object-oriented way to work with paths, and it handles recursive traversal well once you know which method matches your goal. The most important choice is whether you want every path under a directory, only certain file types, or a custom traversal that can skip directories or handle errors explicitly.

The Main Recursive Tools in pathlib

The two methods most people reach for are rglob() and glob("**/*"). Both walk subdirectories recursively.

1from pathlib import Path
2
3root = Path("project")
4
5for path in root.rglob("*"):
6    print(path)

That prints every file and directory under project. If you only want Python files, change the pattern:

1from pathlib import Path
2
3root = Path("project")
4
5for path in root.rglob("*.py"):
6    print(path)

The equivalent glob form is:

1from pathlib import Path
2
3root = Path("project")
4
5for path in root.glob("**/*.py"):
6    print(path)

In practice, rglob("*.py") is usually the easiest version to read.

Filtering Files and Directories

Recursive iteration often needs filtering after traversal. For example, you may want only regular files and not directories:

1from pathlib import Path
2
3root = Path("project")
4
5for path in root.rglob("*"):
6    if path.is_file():
7        print(path)

Or perhaps you want only subdirectories:

1from pathlib import Path
2
3root = Path("project")
4
5for path in root.rglob("*"):
6    if path.is_dir():
7        print(path)

This pattern is clearer than trying to force complex logic into the glob expression itself.

When a Manual Recursive Walk Is Better

rglob() is convenient, but a manual recursive function gives you more control. This is useful when you want to skip hidden folders, avoid symbolic links, or recover from permission errors.

1from pathlib import Path
2
3
4def walk_paths(directory: Path):
5    for entry in directory.iterdir():
6        yield entry
7        if entry.is_dir() and not entry.is_symlink():
8            yield from walk_paths(entry)
9
10
11root = Path("project")
12for path in walk_paths(root):
13    print(path)

This version makes the traversal rules explicit. You can insert any condition you need before descending into a directory.

For example, skipping cache directories is straightforward:

1from pathlib import Path
2
3
4def walk_paths(directory: Path):
5    for entry in directory.iterdir():
6        if entry.name == "__pycache__":
7            continue
8        yield entry
9        if entry.is_dir() and not entry.is_symlink():
10            yield from walk_paths(entry)

That kind of logic is harder to express cleanly with rglob() alone.

Building Useful Results

A traversal loop often does more than print paths. You might collect file sizes, count extensions, or search for a name.

1from collections import Counter
2from pathlib import Path
3
4root = Path("project")
5counts = Counter()
6
7for path in root.rglob("*"):
8    if path.is_file():
9        counts[path.suffix or "[no extension]"] += 1
10
11print(counts)

Because Path objects expose properties such as .name, .suffix, .stem, and .parent, post-processing stays readable.

If you only need the relative path from the root, use relative_to:

1from pathlib import Path
2
3root = Path("project")
4
5for path in root.rglob("*.md"):
6    print(path.relative_to(root))

That is helpful when generating manifests or reports.

Error Handling and Symlinks

Recursive filesystem code can fail on unreadable directories or broken links. If you control the environment, rglob() may be enough. In mixed or user-controlled environments, manual recursion gives you room to handle exceptions.

1from pathlib import Path
2
3
4def safe_walk(directory: Path):
5    try:
6        for entry in directory.iterdir():
7            yield entry
8            if entry.is_dir() and not entry.is_symlink():
9                yield from safe_walk(entry)
10    except PermissionError:
11        print(f"Skipping unreadable directory: {directory}")

Avoid following symlinks blindly in recursive code. A symlink can point upward in the directory tree and create repeated traversal or surprising results.

Choosing Between pathlib and os.walk

os.walk is still a good tool, especially if you need in-place control over the list of directories to descend into. pathlib is often preferred because the resulting code is more expressive and path operations are built in.

If you already use Path objects elsewhere in your application, staying inside pathlib avoids constant conversion between strings and paths.

Common Pitfalls

A common mistake is assuming iterdir() is recursive. It is not. It only yields immediate children of one directory level.

Another issue is forgetting to filter with is_file() or is_dir(). A recursive glob can return both, which leads to confusing downstream logic.

Symbolic links deserve special care. If you recurse through them without a rule, traversal can become unexpectedly large or cyclical.

Finally, do not assume recursive scanning is cheap. On large trees, it can be slow, so filter as early as possible and avoid unnecessary work inside the loop.

Summary

• Use Path.rglob() for the shortest recursive traversal code.
• Use a manual recursive generator when you need custom skip rules or error handling.
• Filter with is_file() and is_dir() to make intent explicit.
• Be careful with symbolic links and permission errors.
• Prefer pathlib when you want readable path manipulation alongside traversal.