Skip to content

Python subprocess: Run External Commands from Python (Complete Guide)

Soren Atelier
Name
Soren Atelier

Updated on

Python scripts often need to call external programs. You might need to run a shell command to compress files, invoke git for version control, call a system utility like ffmpeg to process video, or execute a compiled binary as part of a data pipeline. But reaching for os.system() or backtick-style hacks leads to code that is brittle, insecure, and impossible to debug when things go wrong.

The pain gets worse fast. Output disappears into the void because you have no way to capture it. Errors pass silently because the return code is ignored. A single user-supplied filename with a space or semicolon in it can turn your innocent script into a shell injection vulnerability. And when a subprocess hangs, your entire Python program hangs with it -- no timeout, no recovery, no explanation.

Python's subprocess module is the standard solution. It replaces older functions like os.system(), os.popen(), and the deprecated commands module with a single, consistent API for spawning processes, capturing their output, handling errors, setting timeouts, and building pipelines. This guide covers everything you need to use it effectively and securely.

📚

    Quick Start with subprocess.run()

    The subprocess.run() function, introduced in Python 3.5, is the recommended way to run external commands. It runs a command, waits for it to finish, and returns a CompletedProcess object.

    import subprocess
 
# Run a simple command
result = subprocess.run(["ls", "-la"], capture_output=True, text=True)
 
print(result.stdout)       # standard output as a string
print(result.stderr)       # standard error as a string
print(result.returncode)   # 0 means success

    Key parameters:

    • capture_output=True captures stdout and stderr (equivalent to stdout=subprocess.PIPE, stderr=subprocess.PIPE)
    • text=True decodes output as strings instead of bytes
    • The command is passed as a list of strings, where each argument is a separate element
    import subprocess
 
# Run a command with arguments
result = subprocess.run(
    ["python", "--version"],
    capture_output=True,
    text=True
)
print(result.stdout.strip())  # e.g., "Python 3.12.1"

    Understanding the Command List vs String

    The first argument to subprocess.run() can be a list or a string. This distinction matters for correctness and security.

    List form (recommended)

    Each element in the list is a separate argument. Python passes them directly to the operating system without shell interpretation.

    import subprocess
 
# Each argument is a separate list element
result = subprocess.run(
    ["grep", "-r", "TODO", "/home/user/project"],
    capture_output=True,
    text=True
)
print(result.stdout)

    Filenames with spaces, quotes, or special characters work correctly because each argument is passed as-is:

    import subprocess
 
# Filename with spaces -- works correctly as a list element
result = subprocess.run(
    ["cat", "my file with spaces.txt"],
    capture_output=True,
    text=True
)

    String form (requires shell=True)

    Passing a single string requires shell=True, which invokes the system shell (/bin/sh on Unix, cmd.exe on Windows) to interpret the command.

    import subprocess
 
# String form requires shell=True
result = subprocess.run(
    "ls -la | grep '.py'",
    shell=True,
    capture_output=True,
    text=True
)
print(result.stdout)

    This enables shell features like pipes (|), redirects (>), globbing (*.py), and environment variable expansion ($HOME). But it also introduces serious security risks, which we cover in the Security section below.

    Capturing Output

    Capture stdout and stderr separately

    import subprocess
 
result = subprocess.run(
    ["python", "-c", "import sys; print('out'); print('err', file=sys.stderr)"],
    capture_output=True,
    text=True
)
 
print(f"stdout: {result.stdout}")   # "out\n"
print(f"stderr: {result.stderr}")   # "err\n"

    Merge stderr into stdout

    Sometimes you want all output in one stream. Use stderr=subprocess.STDOUT:

    import subprocess
 
result = subprocess.run(
    ["python", "-c", "import sys; print('out'); print('err', file=sys.stderr)"],
    stdout=subprocess.PIPE,
    stderr=subprocess.STDOUT,
    text=True
)
 
print(result.stdout)  # Contains both "out\n" and "err\n"

    Discard output

    Send output to subprocess.DEVNULL to suppress it:

    import subprocess
 
# Run silently -- discard all output
result = subprocess.run(
    ["apt-get", "update"],
    stdout=subprocess.DEVNULL,
    stderr=subprocess.DEVNULL
)

    Binary output

    Omit text=True to receive raw bytes. Useful for binary data like images or compressed files:

    import subprocess
 
# Capture binary output (e.g., from curl)
result = subprocess.run(
    ["curl", "-s", "https://example.com/image.png"],
    capture_output=True
)
 
image_bytes = result.stdout  # bytes object
print(f"Downloaded {len(image_bytes)} bytes")

    Error Handling

    Check return codes manually

    By default, subprocess.run() does not raise an exception when a command fails. You must check returncode yourself:

    import subprocess
 
result = subprocess.run(
    ["ls", "/nonexistent/path"],
    capture_output=True,
    text=True
)
 
if result.returncode != 0:
    print(f"Command failed with code {result.returncode}")
    print(f"Error: {result.stderr}")

    Automatic exception on failure with check=True

    The check=True parameter raises subprocess.CalledProcessError when the return code is non-zero:

    import subprocess
 
try:
    result = subprocess.run(
        ["ls", "/nonexistent/path"],
        capture_output=True,
        text=True,
        check=True
    )
except subprocess.CalledProcessError as e:
    print(f"Command failed with return code {e.returncode}")
    print(f"stderr: {e.stderr}")
    print(f"stdout: {e.stdout}")

    This is the recommended pattern for commands that should always succeed. It forces you to handle failures explicitly rather than silently ignoring them.

    Handle command not found

    If the executable does not exist, Python raises FileNotFoundError:

    import subprocess
 
try:
    result = subprocess.run(
        ["nonexistent_command"],
        capture_output=True,
        text=True
    )
except FileNotFoundError:
    print("Command not found -- is it installed and in PATH?")

    Timeouts

    Long-running or hung processes can block your script forever. The timeout parameter (in seconds) kills the process and raises subprocess.TimeoutExpired if it does not finish in time:

    import subprocess
 
try:
    result = subprocess.run(
        ["sleep", "30"],
        timeout=5,
        capture_output=True,
        text=True
    )
except subprocess.TimeoutExpired:
    print("Process timed out after 5 seconds")

    This is essential for network commands, external APIs, or any process that might hang:

    import subprocess
 
def run_with_timeout(cmd, timeout_seconds=30):
    """Run a command with timeout and error handling."""
    try:
        result = subprocess.run(
            cmd,
            capture_output=True,
            text=True,
            timeout=timeout_seconds,
            check=True
        )
        return result.stdout
    except subprocess.TimeoutExpired:
        print(f"Command timed out after {timeout_seconds}s: {' '.join(cmd)}")
        return None
    except subprocess.CalledProcessError as e:
        print(f"Command failed (code {e.returncode}): {e.stderr}")
        return None
    except FileNotFoundError:
        print(f"Command not found: {cmd[0]}")
        return None
 
# Usage
output = run_with_timeout(["ping", "-c", "4", "example.com"], timeout_seconds=10)
if output:
    print(output)

    Passing Input to a Process

    Use the input parameter to send data to a process's stdin:

    import subprocess
 
# Send text to stdin
result = subprocess.run(
    ["grep", "error"],
    input="line 1\nerror on line 2\nline 3\nerror on line 4\n",
    capture_output=True,
    text=True
)
 
print(result.stdout)
# "error on line 2\nerror on line 4\n"

    This replaces the common pattern of piping data through shell commands:

    import subprocess
import json
 
# Send JSON to a processing command
data = {"name": "Alice", "score": 95}
json_string = json.dumps(data)
 
result = subprocess.run(
    ["python", "-c", "import sys, json; d = json.load(sys.stdin); print(d['name'])"],
    input=json_string,
    capture_output=True,
    text=True
)
 
print(result.stdout.strip())  # "Alice"

    Environment Variables

    By default, subprocesses inherit the current environment. You can modify it:

    import subprocess
import os
 
# Add or override environment variables
custom_env = os.environ.copy()
custom_env["API_KEY"] = "secret123"
custom_env["DEBUG"] = "true"
 
result = subprocess.run(
    ["python", "-c", "import os; print(os.environ.get('API_KEY'))"],
    env=custom_env,
    capture_output=True,
    text=True
)
 
print(result.stdout.strip())  # "secret123"

    Always use os.environ.copy() as the base. Passing a dict without the existing environment strips all inherited variables, which can break commands that depend on PATH, HOME, or other system variables.

    Working Directory

    The cwd parameter sets the working directory for the subprocess:

    import subprocess
 
# Run git status in a specific repository
result = subprocess.run(
    ["git", "status", "--short"],
    cwd="/home/user/my-project",
    capture_output=True,
    text=True
)
 
print(result.stdout)

    subprocess.run() vs Popen: When to Use Each

    subprocess.run() is a convenience wrapper around subprocess.Popen. For most use cases, run() is sufficient. Use Popen when you need:

    • Real-time streaming of output (line by line as it is produced)
    • Interacting with a running process (sending input, reading output in a loop)
    • Building multi-step pipelines connecting several processes
    • Non-blocking execution with manual control over process lifecycle

    Comparison Table

    Featuresubprocess.run()subprocess.Popenos.system()
    RecommendedYes (Python 3.5+)Yes (advanced)No (deprecated pattern)
    Capture outputYes (capture_output=True)Yes (via PIPE)No
    Return valueCompletedProcess objectPopen process objectExit code (int)
    Timeout supportYes (timeout param)Manual (via wait/communicate)No
    Error checkingcheck=True raises exceptionManualMust parse exit code
    Input to stdininput parametercommunicate() or stdin.write()No
    Real-time outputNo (waits for completion)Yes (stream line by line)Output goes to terminal
    PipelinesLimited (single command)Yes (chain multiple Popen)Yes (via shell string)
    SecuritySafe with list argsSafe with list argsShell injection risk
    Shell featuresOnly with shell=TrueOnly with shell=TrueAlways uses shell

    Advanced: subprocess.Popen

    Popen gives you full control over the process lifecycle. The constructor starts the process immediately and returns a Popen object you can interact with.

    Basic Popen usage

    import subprocess
 
proc = subprocess.Popen(
    ["ls", "-la"],
    stdout=subprocess.PIPE,
    stderr=subprocess.PIPE,
    text=True
)
 
stdout, stderr = proc.communicate()  # Wait for completion and get output
print(f"Return code: {proc.returncode}")
print(stdout)

    Stream output in real time

    Unlike run(), Popen lets you read output line by line as it is produced:

    import subprocess
 
proc = subprocess.Popen(
    ["ping", "-c", "5", "example.com"],
    stdout=subprocess.PIPE,
    text=True
)
 
# Read output line by line as it arrives
for line in proc.stdout:
    print(f"[LIVE] {line.strip()}")
 
proc.wait()  # Wait for process to finish
print(f"Exit code: {proc.returncode}")

    This is essential for long-running commands where you want to show progress or log output in real time:

    import subprocess
import sys
 
def run_with_live_output(cmd):
    """Run a command and stream its output in real time."""
    proc = subprocess.Popen(
        cmd,
        stdout=subprocess.PIPE,
        stderr=subprocess.STDOUT,
        text=True,
        bufsize=1  # Line-buffered
    )
 
    output_lines = []
    for line in proc.stdout:
        line = line.rstrip()
        print(line)
        output_lines.append(line)
 
    proc.wait()
    return proc.returncode, "\n".join(output_lines)
 
# Usage
code, output = run_with_live_output(["pip", "install", "requests"])
print(f"\nFinished with exit code: {code}")

    Building Pipelines

    Connect multiple commands by piping one process's stdout to another's stdin:

    import subprocess
 
# Equivalent to: cat /var/log/syslog | grep "error" | wc -l
p1 = subprocess.Popen(
    ["cat", "/var/log/syslog"],
    stdout=subprocess.PIPE
)
 
p2 = subprocess.Popen(
    ["grep", "error"],
    stdin=p1.stdout,
    stdout=subprocess.PIPE
)
 
# Allow p1 to receive SIGPIPE if p2 exits early
p1.stdout.close()
 
p3 = subprocess.Popen(
    ["wc", "-l"],
    stdin=p2.stdout,
    stdout=subprocess.PIPE,
    text=True
)
 
p2.stdout.close()
 
output, _ = p3.communicate()
print(f"Error count: {output.strip()}")

    The p1.stdout.close() call after connecting it to p2 is important. It allows p1 to receive a SIGPIPE signal if p2 exits early, preventing deadlocks.

    Interactive process communication

    import subprocess
 
# Start a Python REPL as a subprocess
proc = subprocess.Popen(
    ["python", "-i"],
    stdin=subprocess.PIPE,
    stdout=subprocess.PIPE,
    stderr=subprocess.PIPE,
    text=True
)
 
# Send commands and get results
stdout, stderr = proc.communicate(input="print(2 + 2)\nprint('hello')\n")
print(f"stdout: {stdout}")
print(f"stderr: {stderr}")

    shell=True: Power and Danger

    Setting shell=True runs the command through the system shell, enabling shell features but introducing security risks.

    When shell=True is useful

    import subprocess
 
# Shell features: pipes, redirects, globbing, env vars
result = subprocess.run(
    "ls *.py | wc -l",
    shell=True,
    capture_output=True,
    text=True
)
print(f"Python files: {result.stdout.strip()}")
 
# Environment variable expansion
result = subprocess.run(
    "echo $HOME",
    shell=True,
    capture_output=True,
    text=True
)
print(result.stdout.strip())

    The shell injection problem

    Never pass unsanitized user input to shell=True:

    import subprocess
 
# DANGEROUS -- shell injection vulnerability
user_input = "file.txt; rm -rf /"  # malicious input
subprocess.run(f"cat {user_input}", shell=True)  # Executes "rm -rf /"!
 
# SAFE -- use list form without shell=True
subprocess.run(["cat", user_input])  # Treats entire string as filename

    If you must use shell=True with dynamic values, use shlex.quote():

    import subprocess
import shlex
 
user_input = "file with spaces.txt; rm -rf /"
safe_input = shlex.quote(user_input)
 
# shlex.quote wraps in single quotes, neutralizing shell metacharacters
result = subprocess.run(
    f"cat {safe_input}",
    shell=True,
    capture_output=True,
    text=True
)

    But the safest approach is to avoid shell=True entirely and replicate shell features in Python:

    import subprocess
import glob
 
# Instead of: subprocess.run("ls *.py | wc -l", shell=True)
py_files = glob.glob("*.py")
print(f"Python files: {len(py_files)}")
 
# Instead of: subprocess.run("cat file1.txt file2.txt > combined.txt", shell=True)
with open("combined.txt", "w") as outfile:
    result = subprocess.run(
        ["cat", "file1.txt", "file2.txt"],
        stdout=outfile
    )

    Security Best Practices

    PracticeDoDon't
    Command format["cmd", "arg1", "arg2"]f"cmd {user_input}" with shell=True
    User inputshlex.quote() if shell is neededString concatenation into commands
    Shell modeshell=False (default)shell=True with untrusted input
    Executable pathFull paths like /usr/bin/gitRely on PATH for security-critical code
    Input validationValidate and sanitize before passingPass raw user input to commands

    Using pathlib for path validation makes security checks cleaner:

    import subprocess
import shlex
from pathlib import Path
 
def safe_file_operation(filename):
    """Safely run a command with user-supplied filename."""
    # Validate input
    path = Path(filename)
    if not path.exists():
        raise FileNotFoundError(f"File not found: {filename}")
 
    # Check for path traversal
    resolved = path.resolve()
    allowed_dir = Path("/home/user/uploads").resolve()
    if not str(resolved).startswith(str(allowed_dir)):
        raise PermissionError("Access denied: file outside allowed directory")
 
    # Use list form -- no shell injection possible
    result = subprocess.run(
        ["wc", "-l", str(resolved)],
        capture_output=True,
        text=True,
        check=True
    )
    return result.stdout.strip()

    Real-World Examples

    Run git commands

    import subprocess
 
def git_status(repo_path):
    """Get git status for a repository."""
    result = subprocess.run(
        ["git", "status", "--porcelain"],
        cwd=repo_path,
        capture_output=True,
        text=True,
        check=True
    )
    return result.stdout.strip()
 
def git_log(repo_path, n=5):
    """Get last n commit messages."""
    result = subprocess.run(
        ["git", "log", f"--oneline", f"-{n}"],
        cwd=repo_path,
        capture_output=True,
        text=True,
        check=True
    )
    return result.stdout.strip()
 
status = git_status("/home/user/my-project")
if status:
    print("Uncommitted changes:")
    print(status)
else:
    print("Working directory clean")

    Compress and extract files

    import subprocess
 
def compress_directory(source_dir, output_file):
    """Create a tar.gz archive of a directory."""
    subprocess.run(
        ["tar", "-czf", output_file, "-C", source_dir, "."],
        check=True
    )
    print(f"Created archive: {output_file}")
 
def extract_archive(archive_file, dest_dir):
    """Extract a tar.gz archive."""
    subprocess.run(
        ["tar", "-xzf", archive_file, "-C", dest_dir],
        check=True
    )
    print(f"Extracted to: {dest_dir}")
 
compress_directory("/home/user/data", "/tmp/data_backup.tar.gz")

    Check system information

    import subprocess
 
def get_disk_usage():
    """Get disk usage summary."""
    result = subprocess.run(
        ["df", "-h", "/"],
        capture_output=True,
        text=True,
        check=True
    )
    return result.stdout
 
def get_memory_info():
    """Get memory usage on Linux."""
    result = subprocess.run(
        ["free", "-h"],
        capture_output=True,
        text=True,
        check=True
    )
    return result.stdout
 
def get_process_list(filter_name=None):
    """List running processes, optionally filtered."""
    cmd = ["ps", "aux"]
    result = subprocess.run(cmd, capture_output=True, text=True, check=True)
 
    if filter_name:
        lines = result.stdout.strip().split("\n")
        header = lines[0]
        matching = [line for line in lines[1:] if filter_name in line]
        return header + "\n" + "\n".join(matching)
 
    return result.stdout
 
print(get_disk_usage())

    Process data files with external tools

    import subprocess
import csv
import io
 
def sort_csv_by_column(input_file, column_index=1):
    """Sort a CSV file using the system sort command (fast for large files)."""
    result = subprocess.run(
        ["sort", "-t,", f"-k{column_index}", input_file],
        capture_output=True,
        text=True,
        check=True
    )
    return result.stdout
 
def count_lines(filepath):
    """Count lines in a file using wc (faster than Python for huge files)."""
    result = subprocess.run(
        ["wc", "-l", filepath],
        capture_output=True,
        text=True,
        check=True
    )
    return int(result.stdout.strip().split()[0])
 
def search_in_files(directory, pattern, file_type="*.py"):
    """Search for a pattern in files using grep."""
    result = subprocess.run(
        ["grep", "-rn", "--include", file_type, pattern, directory],
        capture_output=True,
        text=True
    )
    # grep returns exit code 1 if no matches found (not an error)
    if result.returncode == 0:
        return result.stdout
    elif result.returncode == 1:
        return ""  # No matches
    else:
        raise subprocess.CalledProcessError(result.returncode, result.args)
 
matches = search_in_files("/home/user/project", "TODO")
if matches:
    print(matches)
else:
    print("No TODOs found")

    Automated deployment script

    import subprocess
import sys
 
def deploy(repo_path, branch="main"):
    """Simple deployment script using subprocess."""
    steps = [
        (["git", "fetch", "origin"], "Fetching latest changes"),
        (["git", "checkout", branch], f"Switching to {branch}"),
        (["git", "pull", "origin", branch], "Pulling latest code"),
        (["pip", "install", "-r", "requirements.txt"], "Installing dependencies"),
        (["python", "manage.py", "migrate"], "Running migrations"),
        (["python", "manage.py", "collectstatic", "--noinput"], "Collecting static files"),
    ]
 
    for cmd, description in steps:
        print(f"\n--- {description} ---")
        try:
            result = subprocess.run(
                cmd,
                cwd=repo_path,
                capture_output=True,
                text=True,
                check=True,
                timeout=120
            )
            if result.stdout:
                print(result.stdout)
        except subprocess.CalledProcessError as e:
            print(f"FAILED: {e.stderr}")
            sys.exit(1)
        except subprocess.TimeoutExpired:
            print(f"TIMEOUT: {description} took too long")
            sys.exit(1)
 
    print("\nDeployment complete")

    Cross-Platform Considerations

    Commands behave differently on Windows vs Unix. Write portable code:

    import subprocess
import platform
 
def run_command(cmd_unix, cmd_windows=None):
    """Run a command with platform awareness."""
    if platform.system() == "Windows":
        cmd = cmd_windows or cmd_unix
        # Windows often needs shell=True for built-in commands
        return subprocess.run(cmd, shell=True, capture_output=True, text=True)
    else:
        return subprocess.run(cmd, capture_output=True, text=True)
 
# List directory contents
result = run_command(
    cmd_unix=["ls", "-la"],
    cmd_windows="dir"
)
print(result.stdout)

    Key platform differences:

    FeatureUnix/macOSWindows
    Shell/bin/shcmd.exe
    Path separator/\\
    Built-in commands (dir, copy)Not availableRequire shell=True
    Executable extensionNot needed.exe required sometimes
    Signal handlingFull POSIX signalsLimited
    shlex.quote()WorksUse subprocess.list2cmdline()

    Running subprocess in Jupyter Notebooks

    Running shell commands from Jupyter notebooks is a common workflow for data scientists. While Jupyter supports !command syntax for quick shell calls, subprocess gives you proper error handling and output capture within your Python code.

    When debugging subprocess calls in notebooks -- especially when commands fail silently or produce unexpected output -- RunCell (opens in a new tab) can help. RunCell is an AI agent for Jupyter that understands your notebook context. It can diagnose why a subprocess command fails, suggest the correct arguments, and handle platform-specific quirks. Instead of switching between terminal and notebook to debug a shell command, RunCell traces the issue directly in your cell.

    import subprocess
 
# In a Jupyter notebook: capture and display command output
result = subprocess.run(
    ["pip", "list", "--format=columns"],
    capture_output=True,
    text=True
)
 
# Display as formatted output in the notebook
print(result.stdout)

    Common Mistakes and How to Fix Them

    Mistake 1: Forgetting to capture output

    import subprocess
 
# Output goes to terminal, not captured
result = subprocess.run(["ls", "-la"])
print(result.stdout)  # None!
 
# Fix: add capture_output=True
result = subprocess.run(["ls", "-la"], capture_output=True, text=True)
print(result.stdout)  # Actual output

    Mistake 2: Using a string without shell=True

    import subprocess
 
# Fails: string passed without shell=True
# subprocess.run("ls -la")  # FileNotFoundError: "ls -la" is not a program
 
# Fix option 1: use a list
subprocess.run(["ls", "-la"])
 
# Fix option 2: use shell=True (less safe)
subprocess.run("ls -la", shell=True)

    Mistake 3: Ignoring errors

    import subprocess
 
# Bad: silently continues on failure
result = subprocess.run(["rm", "/important/file"], capture_output=True, text=True)
# ... continues even if rm failed
 
# Good: check=True raises exception on failure
try:
    result = subprocess.run(
        ["rm", "/important/file"],
        capture_output=True,
        text=True,
        check=True
    )
except subprocess.CalledProcessError as e:
    print(f"Failed to delete: {e.stderr}")

    Mistake 4: Deadlock with Popen

    import subprocess
 
# DEADLOCK: stdout buffer fills up, process blocks, .wait() waits forever
proc = subprocess.Popen(["command_with_lots_of_output"], stdout=subprocess.PIPE)
proc.wait()  # Deadlock!
 
# Fix: use communicate() which handles buffering
proc = subprocess.Popen(["command_with_lots_of_output"], stdout=subprocess.PIPE, text=True)
stdout, stderr = proc.communicate()  # Safe

    Mistake 5: Not handling encoding

    import subprocess
 
# Bytes output can cause issues
result = subprocess.run(["cat", "data.txt"], capture_output=True)
# result.stdout is bytes, not str
 
# Fix: use text=True or encoding parameter
result = subprocess.run(["cat", "data.txt"], capture_output=True, text=True)
 
# For specific encodings:
result = subprocess.run(
    ["cat", "data.txt"],
    capture_output=True,
    encoding="utf-8",
    errors="replace"  # Handle invalid bytes
)

    subprocess.run() Complete Parameter Reference

    import subprocess
 
result = subprocess.run(
    args,                    # Command as list or string
    stdin=None,              # Input source (PIPE, DEVNULL, file object, or None)
    stdout=None,             # Output destination
    stderr=None,             # Error destination
    capture_output=False,    # Shorthand for stdout=PIPE, stderr=PIPE
    text=False,              # Decode output as strings (alias: universal_newlines)
    shell=False,             # Run through system shell
    cwd=None,                # Working directory
    timeout=None,            # Seconds before TimeoutExpired
    check=False,             # Raise CalledProcessError on non-zero exit
    env=None,                # Environment variables dict
    encoding=None,           # Output encoding (alternative to text=True)
    errors=None,             # Encoding error handling ('strict', 'replace', 'ignore')
    input=None,              # String/bytes to send to stdin
)

    FAQ

    What is the subprocess module in Python?

    The subprocess module is Python's standard library for running external commands and programs from within Python scripts. It replaced older approaches like os.system(), os.popen(), and the commands module. It provides functions to spawn new processes, connect to their input/output/error pipes, obtain return codes, and handle timeouts. The primary interface is subprocess.run() for simple command execution, and subprocess.Popen for advanced use cases requiring real-time I/O or process pipelines.

    What is the difference between subprocess.run() and subprocess.Popen?

    subprocess.run() is a high-level convenience function that runs a command, waits for it to finish, and returns a CompletedProcess object with the output. It is the right choice for most tasks. subprocess.Popen is a lower-level class that gives you direct control over the process: you can stream output line by line, send input interactively, build multi-process pipelines, and manage the process lifecycle manually. Use Popen when you need real-time output streaming or when connecting multiple processes together.

    Is shell=True dangerous in subprocess?

    Yes, using shell=True with untrusted input creates a shell injection vulnerability. When shell=True is set, the command string is passed to the system shell for interpretation, which means shell metacharacters like ;, |, &&, and $() are executed. An attacker could inject arbitrary commands. The safe default is shell=False with commands passed as a list. If you must use shell=True, sanitize input with shlex.quote() and never pass raw user input.

    How do I capture the output of a subprocess command?

    Use capture_output=True and text=True with subprocess.run(). The output is stored in result.stdout (as a string) and errors in result.stderr. For example: result = subprocess.run(["ls", "-la"], capture_output=True, text=True) then access result.stdout. Without text=True, the output is returned as bytes.

    How do I handle subprocess timeouts in Python?

    Pass the timeout parameter (in seconds) to subprocess.run(). If the process exceeds the timeout, Python kills it and raises subprocess.TimeoutExpired. Example: subprocess.run(["slow_command"], timeout=30). For Popen, use proc.communicate(timeout=30) or proc.wait(timeout=30). Always wrap timeout-sensitive code in a try/except block.

    Why does os.system() still work if subprocess is recommended?

    os.system() is not formally deprecated, but it is considered a legacy interface. It runs commands through the shell (like shell=True), cannot capture output, provides no timeout mechanism, and returns only the exit status. subprocess.run() does everything os.system() does, plus output capture, error handling, timeouts, and secure argument passing. All new code should use subprocess.

    Conclusion

    The subprocess module is Python's definitive tool for running external commands. Use subprocess.run() for straightforward command execution -- it handles output capture, error checking, timeouts, and input passing in a single function call. Reach for subprocess.Popen only when you need real-time output streaming, interactive process communication, or multi-step pipelines.

    The most important habit is avoiding shell=True with user-supplied input. Pass commands as lists to eliminate shell injection risks entirely. Use check=True to catch failures early. Set timeout to prevent hung processes. And use text=True to work with strings instead of raw bytes.

    For everything from git automation to data pipeline orchestration, subprocess gives you the control and safety that os.system() never could. Master these patterns and you can confidently integrate any external tool into your Python workflows. When building CLI tools that call subprocess, combine it with argparse for professional command-line interfaces. For concurrent subprocess execution, see our guides on threading and asyncio.

    Related Guides

    📚