Introduction

Python can run shell commands, but the right tool is almost always subprocess, not os.system. The difference matters because subprocess lets you capture output, check exit codes, control environment variables, and avoid common shell-injection mistakes.

Use subprocess.run for most commands

For one-shot commands, subprocess.run is the default choice:

1import subprocess
2
3result = subprocess.run(
4    ["ls", "-la"],
5    capture_output=True,
6    text=True,
7    check=True,
8)
9
10print(result.stdout)

A few details here are important:

• Pass the command as a list so Python does the argument splitting safely.
• Use capture_output=True if you need stdout and stderr in Python.
• Use text=True to get strings instead of raw bytes.
• Use check=True so Python raises an exception when the command fails.

This is much safer than building one shell string and hoping quoting rules work out.

Read exit codes and handle errors

Sometimes you do not want an exception on failure. In that case, inspect the return code yourself:

1import subprocess
2
3result = subprocess.run(["grep", "TODO", "app.py"], capture_output=True, text=True)
4
5if result.returncode == 0:
6    print("Matches found:")
7    print(result.stdout)
8elif result.returncode == 1:
9    print("No matches found.")
10else:
11    print("Command failed:")
12    print(result.stderr)

This is a good pattern when the command's nonzero exit codes have domain-specific meanings rather than representing a hard failure every time.

Use shell=True only when you need real shell features

If you need pipes, glob expansion, or shell built-ins, you may be tempted to use shell=True:

1import subprocess
2
3result = subprocess.run(
4    "cat access.log | grep 500 | wc -l",
5    shell=True,
6    capture_output=True,
7    text=True,
8)
9
10print(result.stdout.strip())

This works, but it is also the risky mode. If any part of that command string contains user input, you may have a shell-injection vulnerability.

Whenever possible, replace shell pipelines with plain Python or separate subprocess calls. For example, a safer version of the previous command could parse the file in Python directly.

Stream output with Popen for long-running commands

When the command runs for a long time and you want to consume output as it arrives, use subprocess.Popen:

1import subprocess
2
3process = subprocess.Popen(
4    ["ping", "-c", "4", "example.com"],
5    stdout=subprocess.PIPE,
6    stderr=subprocess.STDOUT,
7    text=True,
8)
9
10for line in process.stdout:
11    print("OUTPUT:", line.rstrip())
12
13return_code = process.wait()
14print("Finished with:", return_code)

This is useful for build logs, deployment scripts, and any command where waiting for all output at the end would be too late.

Pass environment variables and working directories

Two other subprocess features are extremely useful in automation scripts:

1import os
2import subprocess
3
4env = os.environ.copy()
5env["APP_ENV"] = "staging"
6
7result = subprocess.run(
8    ["python", "manage.py", "check"],
9    cwd="/Users/markqian/project",
10    env=env,
11    capture_output=True,
12    text=True,
13)
14
15print(result.stdout)

cwd changes the working directory for the child process, and env lets you override or add environment variables without affecting the current Python process.

Common Pitfalls

The biggest mistake is using shell=True for commands that could be expressed as a simple argument list. That creates quoting bugs and security problems for no real benefit.

Another common issue is ignoring the exit code. A command may print something useful and still fail, so always check returncode or use check=True.

People also forget the difference between bytes and text. Without text=True, stdout and stderr come back as bytes objects.

Finally, do not use os.system for new code unless you genuinely need the simplest possible fire-and-forget call and do not care about output or structured error handling.

Summary

• Use subprocess.run as the default way to run Bash or shell commands from Python.
• Pass commands as argument lists instead of raw shell strings whenever possible.
• Check exit codes or use check=True so failures do not go unnoticed.
• Use Popen when you need streaming output from long-running processes.
• Avoid shell=True unless you explicitly need shell syntax and trust the input.