Introduction

If you want to reproduce a shell pipeline such as cmd1 | cmd2 in Python, the safest approach is to create separate subprocesses and connect the first process's stdout to the second process's stdin. Python's subprocess module supports this directly with Popen.

You can also use shell=True and pass the whole pipeline as a single string, but that is only appropriate for trusted commands. For general code, explicit pipes are safer and easier to control.

Build a Simple Two-Command Pipeline

The standard pattern is to create the first process with stdout=subprocess.PIPE, then pass that pipe as stdin to the next process.

1import subprocess
2
3p1 = subprocess.Popen(["printf", "apple\nbanana\napple\n"], stdout=subprocess.PIPE)
4p2 = subprocess.Popen(["sort"], stdin=p1.stdout, stdout=subprocess.PIPE, text=True)
5
6p1.stdout.close()
7output, _ = p2.communicate()
8
9print(output)

This is the Python equivalent of:

1printf 'apple
2banana
3apple
4' | sort

Closing p1.stdout in the parent process is important because it lets the upstream process receive normal pipe behavior if the downstream process exits early.

A Longer Pipeline

You can chain more than two processes the same way.

1import subprocess
2
3p1 = subprocess.Popen(["printf", "apple\nbanana\napple\nbanana\nbanana\n"], stdout=subprocess.PIPE)
4p2 = subprocess.Popen(["sort"], stdin=p1.stdout, stdout=subprocess.PIPE)
5p3 = subprocess.Popen(["uniq", "-c"], stdin=p2.stdout, stdout=subprocess.PIPE, text=True)
6
7p1.stdout.close()
8p2.stdout.close()
9output, _ = p3.communicate()
10
11print(output)

This keeps the pipeline explicit, which is helpful when you need to inspect return codes or handle errors process by process.

When shell=True Is Acceptable

For a quick hardcoded command, you can let the shell build the pipeline.

1import subprocess
2
3result = subprocess.run(
4    "printf 'apple\\nbanana\\napple\\n' | sort | uniq -c",
5    shell=True,
6    capture_output=True,
7    text=True,
8    check=True,
9)
10
11print(result.stdout)

This is shorter, but it should not be used with untrusted input. If any part of the command comes from a user, shell=True opens the door to shell injection.

Send Input into the First Process

Sometimes you want Python to provide the input rather than launching a command that generates it.

1import subprocess
2
3p1 = subprocess.Popen(["sort"], stdin=subprocess.PIPE, stdout=subprocess.PIPE, text=True)
4p2 = subprocess.Popen(["uniq", "-c"], stdin=p1.stdout, stdout=subprocess.PIPE, text=True)
5
6p1.stdout.close()
7
8p1.communicate("banana\napple\nbanana\n")
9output, _ = p2.communicate()
10
11print(output)

The general rule is to feed the input to the first process, then collect the final output from the last process.

Check Errors Explicitly

In a pipeline, the last command's success does not automatically tell you whether the earlier commands succeeded. Check return codes if correctness matters.

1import subprocess
2
3p1 = subprocess.Popen(["cat", "missing-file.txt"], stdout=subprocess.PIPE, stderr=subprocess.PIPE, text=True)
4p2 = subprocess.Popen(["wc", "-l"], stdin=p1.stdout, stdout=subprocess.PIPE, stderr=subprocess.PIPE, text=True)
5
6p1.stdout.close()
7out2, err2 = p2.communicate()
8out1, err1 = p1.communicate()
9
10print("p1 return code:", p1.returncode)
11print("p1 stderr:", err1)
12print("p2 return code:", p2.returncode)
13print("p2 stdout:", out2)
14print("p2 stderr:", err2)

If you care about reliability, treat the whole pipeline as more than just the last command.

Prefer Python Logic When a Shell Pipeline Adds No Value

Sometimes the best answer is not to recreate the pipe at all. If you are only filtering lines, sorting data, or counting matches, Python may be clearer.

subprocess is most useful when you truly need existing command-line programs. Do not force everything through shell-style pipelines if a few lines of Python would be simpler and more portable.

Common Pitfalls

• Using shell=True with user input and creating a shell injection risk.
• Forgetting to close unused pipe ends in the parent process.
• Assuming only the last process's return code matters.
• Mixing bytes and text modes in a confusing way.
• Recreating a shell pipeline when pure Python would be simpler and easier to maintain.

Summary

• Use subprocess.Popen with stdout=subprocess.PIPE and stdin=previous.stdout to build safe pipelines.
• Reserve shell=True for trusted, hardcoded commands.
• Feed input to the first process and read output from the last process.
• Check return codes for the whole pipeline when correctness matters.
• Prefer plain Python over subprocess pipelines when no external command is actually needed.