Introduction

Python is often used to drive local development environments, test suites, and deployment helpers. When that workflow needs containers, the simplest approach is usually to call Docker Compose from Python rather than trying to reimplement Compose behavior through the Docker API.

Prefer the CLI Over Rebuilding Compose Logic

Even though the old standalone command was named docker-compose, current Docker installations typically expose Compose as docker compose. From Python, that difference only changes the command arguments you pass to subprocess.

For most automation tasks, the CLI is the right abstraction. Compose already knows how to read compose.yaml, resolve environment files, build images, and start dependent services in the correct order. The Docker SDK for Python is useful for container-level operations, but it does not replace the full Compose workflow cleanly.

Running Compose With subprocess

Use subprocess.run when you want a simple call that either succeeds or fails.

1from pathlib import Path
2import shutil
3import subprocess
4
5
6def compose_cmd() -> list[str]:
7    if shutil.which("docker-compose"):
8        return ["docker-compose"]
9    return ["docker", "compose"]
10
11
12def start_stack(project_dir: str) -> None:
13    cmd = compose_cmd() + ["up", "-d", "--build"]
14    result = subprocess.run(
15        cmd,
16        cwd=Path(project_dir),
17        text=True,
18        capture_output=True,
19        check=False,
20    )
21
22    if result.returncode != 0:
23        raise RuntimeError(
24            f"Compose failed with code {result.returncode}\n"
25            f"stdout:\n{result.stdout}\n"
26            f"stderr:\n{result.stderr}"
27        )
28
29
30start_stack("/tmp/demo-app")

There are a few details worth keeping:

• Pass the command as a list instead of one shell string.
• Set cwd to the directory that contains the Compose file.
• Capture output so your Python code can show a useful error message.
• Avoid shell=True unless you truly need shell features.

If your project uses a non-default file name, add -f and the file path to the argument list.

Streaming Logs and Waiting for Readiness

Sometimes you do not just need to start containers; you need to watch the output until a service is ready. In that case, use subprocess.Popen so you can process lines as they arrive.

1from pathlib import Path
2import subprocess
3
4
5def stream_compose_logs(project_dir: str) -> None:
6    cmd = ["docker", "compose", "logs", "--follow", "--no-color"]
7    process = subprocess.Popen(
8        cmd,
9        cwd=Path(project_dir),
10        text=True,
11        stdout=subprocess.PIPE,
12        stderr=subprocess.STDOUT,
13    )
14
15    assert process.stdout is not None
16
17    try:
18        for line in process.stdout:
19            print(line.rstrip())
20            if "server started" in line.lower():
21                break
22    finally:
23        process.terminate()
24        process.wait(timeout=5)

This pattern is common in integration tests. Start the stack with up -d, then follow logs until a known readiness message appears. If the service never becomes healthy, your Python code can time out and tear the stack down.

Cleaning Up Reliably

Compose automation should always include cleanup, especially in tests and CI jobs. The safest pattern is to use try and finally.

1import subprocess
2
3
4def run_test_stack(project_dir: str) -> None:
5    up = ["docker", "compose", "up", "-d"]
6    down = ["docker", "compose", "down", "--volumes"]
7
8    subprocess.run(up, cwd=project_dir, check=True)
9    try:
10        print("Run your tests here")
11    finally:
12        subprocess.run(down, cwd=project_dir, check=True)

Using --volumes is helpful when you want each test run to start from a clean state. In a developer workflow, you may prefer plain down so named volumes persist.

Common Pitfalls

The biggest mistake is assuming the Docker SDK can directly do everything Compose does. It can manage individual containers, networks, and images, but it will not automatically interpret your Compose project the same way the CLI does.

Another common issue is hard-coding docker-compose when the machine only has the plugin form, or hard-coding docker compose on an older machine that still uses the standalone binary. A small detection helper avoids that portability problem.

Developers also get tripped up by running from the wrong working directory. Compose resolves relative paths, environment files, and default project names from the current directory, so an incorrect cwd can make a working command fail mysteriously.

Finally, do not ignore exit codes. If you launch Compose and continue as though it succeeded, your Python script will usually fail later with much less useful diagnostics.

Summary

• Use Python to invoke the Compose CLI, not to reproduce Compose behavior manually.
• Prefer subprocess.run for one-shot commands and subprocess.Popen when you need streaming output.
• Pass arguments as a list, set the correct cwd, and capture or stream logs intentionally.
• Detect whether the system provides docker compose or docker-compose.
• Always clean up containers after tests or temporary automation runs.