Python
import modules
dynamic loading
programming
code organization

How to load all modules in a folder?

Master System Design with Codemia

Enhance your system design skills with over 120 practice problems, detailed solutions, and hands-on exercises.

Introduction

Loading all modules in a folder is a common Python plugin pattern. The right solution depends on whether the folder is a real package and whether you want automatic side effects, explicit registration, or simply a list of module objects you can inspect.

Use pkgutil and importlib for Package Modules

If the folder is a Python package, the standard-library approach is to iterate over submodules and import them dynamically.

Suppose you have this layout:

text
1plugins/
2    __init__.py
3    auth.py
4    billing.py
5    search.py

You can load every module in that package like this:

python
1import importlib
2import pkgutil
3
4import plugins
5
6
7def load_all_modules(package):
8    loaded = {}
9
10    for module_info in pkgutil.iter_modules(package.__path__):
11        full_name = f"{package.__name__}.{module_info.name}"
12        loaded[module_info.name] = importlib.import_module(full_name)
13
14    return loaded
15
16
17modules = load_all_modules(plugins)
18print(sorted(modules))

That is usually the cleanest answer because it works with Python’s import system instead of bypassing it.

Why Package Imports Are Better Than Raw File Loading

It is possible to load .py files directly from paths with importlib.util.spec_from_file_location, but that is usually the wrong first choice. When a folder is already a package, importing by package name has clear advantages:

  • normal import caching
  • proper module names
  • easier relative imports inside the loaded modules
  • clearer debugging output

Direct file loading is more useful for one-off tooling than for application architecture.

A Simple Plugin Registration Pattern

Often you do not really want “all modules.” What you want is “all modules that register themselves.” One clean pattern is for each plugin module to expose a known function:

python
# plugins/auth.py
def register():
    return {"name": "auth", "enabled": True}

Then the loader can collect those registrations:

python
1def load_plugins(package):
2    plugins = []
3
4    for module in load_all_modules(package).values():
5        if hasattr(module, "register"):
6            plugins.append(module.register())
7
8    return plugins
9
10
11print(load_plugins(plugins))

This is clearer than importing modules only for their side effects.

Recursive Loading for Nested Packages

If plugins are nested in subpackages, use walk_packages instead of iter_modules:

python
1def load_recursive(package):
2    loaded = {}
3
4    for module_info in pkgutil.walk_packages(package.__path__, package.__name__ + "."):
5        loaded[module_info.name] = importlib.import_module(module_info.name)
6
7    return loaded

That will import deeper packages such as plugins.admin.audit.

When Automatic Loading Is a Bad Idea

Auto-importing every module can trigger expensive setup or unexpected side effects at import time. For example, a module that opens a database connection at import time will do so just because the folder was scanned.

A healthier design is:

  1. import the module
  2. inspect whether it exposes the expected hook
  3. call the hook explicitly

That keeps import behavior predictable.

Alternative: Explicit Imports Through __init__.py

If the module list is small and stable, explicit imports are often better than dynamic discovery:

python
1# plugins/__init__.py
2from . import auth
3from . import billing
4from . import search

Then importing plugins loads the known modules in an obvious way. Dynamic loading is best when modules are added frequently or the folder is meant to be extensible.

Common Pitfalls

The most common mistake is trying to import raw filenames from a directory that is not actually a Python package. If you want package imports, include __init__.py or use namespace-package conventions deliberately.

Another issue is assuming import order. pkgutil.iter_modules gives you discoverable modules, but if order matters you should sort them yourself.

Developers also forget that importing a module runs its top-level code. If a plugin file performs work immediately on import, dynamic loading can create hard-to-debug startup behavior.

Finally, avoid shadowing package names with local variables. Naming a variable plugins after importing the plugins package can make loader code confusing.

Summary

  • For package modules, use pkgutil plus importlib.
  • Prefer package-name imports over raw file-path loading when possible.
  • Use explicit plugin hooks instead of relying on import side effects.
  • Use walk_packages for nested plugin trees.
  • If the module set is fixed, explicit imports may be simpler than dynamic loading.

Course illustration
Course illustration

All Rights Reserved.