Introduction

Python is often used as a glue language, so calling existing command-line tools is a common requirement. The modern way to do this is the subprocess module, which gives you precise control over arguments, output, exit codes, working directory, and environment variables.

Prefer subprocess.run

For most scripts, subprocess.run is the right starting point. It launches a process, waits for it to finish, and returns a CompletedProcess object with the result.

1import subprocess
2
3result = subprocess.run(
4    ["python3", "--version"],
5    capture_output=True,
6    text=True,
7    check=True,
8)
9
10print("stdout:", result.stdout.strip())
11print("return code:", result.returncode)

This style is safer than building a shell string because each argument is passed separately. You do not need to worry about quoting spaces in file names or special shell characters.

The most important arguments are:

• 'capture_output=True to collect standard output and standard error'
• 'text=True to work with Python strings instead of raw bytes'
• 'check=True to raise an exception if the command exits with a non-zero status'
• 'cwd= to run the command from a specific directory'
• 'env= to override or add environment variables'

Passing Input and Reading Output

Some programs expect input on standard input. subprocess.run can send that input directly:

1import subprocess
2
3result = subprocess.run(
4    ["python3", "-c", "print(input().upper())"],
5    input="codemia\n",
6    capture_output=True,
7    text=True,
8    check=True,
9)
10
11print(result.stdout.strip())

If you only care about the output, this pattern is concise and easy to test. You can also direct output to files rather than storing it in memory:

1import subprocess
2from pathlib import Path
3
4output_path = Path("listing.txt")
5
6with output_path.open("w", encoding="utf-8") as fh:
7    subprocess.run(["ls", "-la"], stdout=fh, text=True, check=True)

That is useful when the command may produce large output.

Running in a Specific Directory or Environment

Many tools behave differently depending on the working directory or environment variables. subprocess.run lets you control both without changing global process state for the rest of your Python program.

1import os
2import subprocess
3
4custom_env = os.environ.copy()
5custom_env["APP_MODE"] = "development"
6
7result = subprocess.run(
8    ["python3", "-c", "import os; print(os.getcwd(), os.getenv('APP_MODE'))"],
9    cwd="/tmp",
10    env=custom_env,
11    capture_output=True,
12    text=True,
13    check=True,
14)
15
16print(result.stdout.strip())

Using cwd is usually better than calling os.chdir before spawning a process because it keeps the scope of the change limited to that child process.

When You Need Popen

subprocess.Popen is lower level and should be used when you need streaming or asynchronous behavior. Examples include long-running programs, interactive tools, or cases where you want to read lines as they arrive.

1import subprocess
2
3process = subprocess.Popen(
4    ["python3", "-c", "import time; [print(i) or time.sleep(1) for i in range(3)]"],
5    stdout=subprocess.PIPE,
6    stderr=subprocess.PIPE,
7    text=True,
8)
9
10for line in process.stdout:
11    print("line:", line.strip())
12
13return_code = process.wait()
14print("finished with", return_code)

This is more flexible, but it also creates more ways to make mistakes. Unless you need streaming, stick to subprocess.run.

Should You Use shell=True?

Only use shell=True when you specifically need shell features such as pipes, wildcard expansion, or shell built-ins. Even then, be careful with untrusted input.

1import subprocess
2
3result = subprocess.run(
4    "printf 'a\\nb\\n' | wc -l",
5    shell=True,
6    capture_output=True,
7    text=True,
8    check=True,
9)
10
11print(result.stdout.strip())

If any part of that command string comes from a user, shell=True can become a command injection risk. For normal executables, passing a list of arguments is safer and clearer.

Common Pitfalls

• Using os.system for anything non-trivial gives you less control and weaker error handling than subprocess.
• Forgetting text=True means output arrives as bytes, which surprises many callers.
• Ignoring the exit code can hide command failures until much later in the program.
• Using shell=True with user-provided data creates serious security problems.
• Reading huge output with capture_output=True can consume a lot of memory. Stream to a file or use Popen if needed.

Summary

• Use subprocess.run for most command execution tasks in Python.
• Pass arguments as a list to avoid quoting issues and reduce shell injection risk.
• Use capture_output, text, cwd, and env to control how the child process runs.
• Reach for Popen only when you need streaming or interactive behavior.
• Treat shell=True as a special case, not the default.