Intermediate
Every Python developer reaches the point where they need to run a shell command from inside their script. Maybe you need to call git, compress a file with tar, check disk usage with df, or trigger a build script. The standard approach is subprocess — and while it works, it is notoriously verbose. A simple ls -la turns into five lines of boilerplate just to get the output back as a string.
The sh library solves this with a completely different philosophy: shell commands become Python functions. Instead of importing subprocess and constructing Popen objects, you import sh and call sh.ls("-la") the same way you call any Python function. Arguments map naturally, pipes work, and output comes back as a string-like object you can iterate, slice, or print directly.
In this article you will learn how to install and use sh, how to pass arguments and flags, how to capture output and handle errors, how to use piping and background processes, and how sh compares to subprocess. By the end you will be able to replace most of your subprocess boilerplate with clean, readable sh calls.
Running Shell Commands with sh: Quick Example
If you just want to see sh in action before diving into the details, here is a minimal working example that lists files in the current directory:
# quick_sh_example.py
import sh
# Call ls just like a Python function
result = sh.ls("-la", "/tmp")
print(result)
Output:
total 48
drwxrwxrwt 12 root root 4096 May 10 08:00 .
drwxr-xr-x 20 root root 4096 Apr 15 10:00 ..
-rw-r--r-- 1 user user 123 May 10 07:55 example.txt
The call sh.ls("-la", "/tmp") runs ls -la /tmp in the shell and returns its output as a string-like RunningCommand object. You pass flags as separate string arguments — the same way you would type them on the command line, just split at each space. No shell=True, no Popen, no .communicate(). The sections below cover everything from basic calls to piping, streaming, and error handling.
What Is sh and Why Use It?
sh is a third-party Python library that wraps subprocesses in a function-call interface. Every program on your system’s PATH becomes importable as a callable. When you call that callable, sh launches the program, passes your arguments, captures stdout and stderr, and returns the result — all in one line of Python.
Compare the same task using subprocess versus sh:
| Task | subprocess | sh |
|---|---|---|
| Run a command | subprocess.run(["ls", "-la"]) | sh.ls("-la") |
| Capture output | result = subprocess.run([...], capture_output=True, text=True) | result = sh.ls("-la") |
| Pipe commands | Multiple Popen calls with stdout=PIPE | sh.grep(sh.ls("-la"), "txt") |
| Check exit code | result.returncode | Exception raised automatically on non-zero |
| Stream output | Manual iteration over stdout | _out=callback parameter |
sh is best suited for scripting tasks where you want to glue together shell commands in Python without writing infrastructure code. For security-sensitive production code that validates user input, subprocess with explicit argument lists remains the safer choice. For developer tooling, build scripts, and automation, sh saves significant time.
Installing sh
sh works on Linux and macOS. It does not support Windows (Windows does not have POSIX-style subprocess semantics). If you are on Windows, use subprocess or Windows Subsystem for Linux instead.
Install sh with pip:
# install_sh.py -- run this in your terminal, not as Python
# pip install sh
import sh
print(sh.__version__)
Output:
2.0.7
Once installed, any command available on your PATH is immediately callable via sh. No registration, no configuration. If git is on your path, sh.git("status") works. If ffmpeg is installed, sh.ffmpeg(...) works. The library discovers commands dynamically at call time.
Basic Command Calls
The simplest use of sh is calling a command with no arguments. The return value is a RunningCommand object that behaves like a string — you can print it, iterate over its lines, or compare it to a string.
# basic_sh_calls.py
import sh
# No arguments
hostname = sh.hostname()
print("Hostname:", hostname.strip())
# With arguments
files = sh.ls("-1", "/etc")
for line in files:
print(line.strip())
Output:
Hostname: mycomputer
apt
bash.bashrc
environment
fstab
hosts
passwd
Each positional argument you pass to the sh command maps to a command-line argument. Flags like -1 are just strings. If a flag takes a value (like --timeout=30), pass it as a single string or use keyword argument style (see the next section). The RunningCommand object has a .strip() method, splits on newlines when iterated, and has a .exit_code attribute.
Flags and Keyword Arguments
sh supports a keyword argument syntax for long flags. A keyword argument with a single underscore prefix maps to a short flag, and double underscore maps to a long flag with a dash. This keeps your Python code readable while matching shell conventions exactly.
# sh_keyword_flags.py
import sh
# curl with long flags using double-underscore prefix
# sh.curl maps --silent to _silent=True and --max-time to _max_time=5
result = sh.curl(
"https://httpbin.org/get",
_silent=True,
_timeout=10
)
print(result[:200])
Output:
{
"args": {},
"headers": {
"Accept": "*/*",
"Host": "httpbin.org",
"User-Agent": "curl/7.81.0"
},
...
}
The underscore prefix is a Python convention that sh intercepts before passing to the subprocess. This way --output becomes _output="file.txt" and --verbose becomes _verbose=True. For flags with dashes in the name like --max-time, replace dashes with underscores: _max_time=10.
Piping Commands Together
Piping is one of the most powerful features in shell scripting. In sh, you pipe by passing one command’s output as the first argument to another command. This mirrors the shell pipe operator | in a Python-native way.
# sh_piping.py
import sh
# Equivalent to: ls /etc | grep "conf" | wc -l
conf_count = sh.wc(
sh.grep(
sh.ls("/etc"),
"conf"
),
"-l"
)
print("Files matching 'conf':", conf_count.strip())
Output:
Files matching 'conf': 12
Read the nested calls inside-out: sh.ls("/etc") runs first and its output feeds into sh.grep(..., "conf"), which then feeds into sh.wc(..., "-l"). This is equivalent to ls /etc | grep conf | wc -l in bash. The nesting looks different from shell pipes but the execution order is the same. For complex pipelines, assign intermediate results to variables to keep the code readable.
Error Handling
By default, sh raises an ErrorReturnCode exception whenever a command exits with a non-zero status. This is the opposite of subprocess where you must check returncode manually. The exception carries the exit code, stdout, and stderr, so you have full context for debugging.
# sh_error_handling.py
import sh
try:
# Try to list a directory that doesn't exist
output = sh.ls("/nonexistent_directory_xyz")
except sh.ErrorReturnCode as e:
print("Command failed!")
print("Exit code:", e.exit_code)
print("stderr:", e.stderr.decode().strip())
Output:
Command failed!
Exit code: 2
stderr: ls: cannot access '/nonexistent_directory_xyz': No such file or directory
You can suppress the exception and check the exit code yourself using _ok_code. Pass a list of exit codes that should be treated as success: sh.grep("pattern", "file.txt", _ok_code=[0, 1]) treats exit code 1 (no matches) as non-fatal. This is useful for commands like grep that use non-zero exit codes for semantic results rather than errors.
Streaming Output in Real Time
For long-running commands like builds or log watchers, waiting for all output before printing is frustrating. sh supports streaming via the _out callback parameter. You pass a function, and sh calls it with each line of output as it arrives.
# sh_streaming.py
import sh
def handle_line(line):
"""Called for each line of output as it arrives."""
print(f"[stream] {line}", end="")
# Stream output of a long-running command
# Using 'ping' with count=3 as a demo of streaming
sh.ping("-c", "3", "8.8.8.8", _out=handle_line)
Output:
[stream] PING 8.8.8.8 (8.8.8.8) 56(84) bytes of data.
[stream] 64 bytes from 8.8.8.8: icmp_seq=1 ttl=118 time=14.2 ms
[stream] 64 bytes from 8.8.8.8: icmp_seq=2 ttl=118 time=13.9 ms
[stream] 64 bytes from 8.8.8.8: icmp_seq=3 ttl=118 time=14.1 ms
The _out parameter accepts any callable. You can use a lambda for simple cases or a class method for stateful processing. There is also an _err parameter for stderr. If you want to stream both stdout and stderr to the same handler, use _err_to_out=True to merge them before passing to _out.
Running Commands in the Background
By default, sh blocks until the command finishes. To run a command in the background and continue Python execution immediately, use the _bg=True parameter. The call returns a RunningCommand object immediately, and you can call .wait() on it later.
# sh_background.py
import sh
import time
# Start a background process
process = sh.sleep("5", _bg=True)
print("Sleep started in background")
# Do other work while the command runs
for i in range(3):
print(f"Working... {i+1}")
time.sleep(1)
# Wait for the background process to finish
process.wait()
print("Background process completed")
Output:
Sleep started in background
Working... 1
Working... 2
Working... 3
Background process completed
Background processes are useful for launching servers or watchers while your main script continues with setup. You can also call process.kill() to terminate a background process early, or check process.is_alive() to see if it is still running. Combine _bg=True with _out=callback for non-blocking streaming output.
Real-Life Example: Git Repository Health Check
Here is a practical script that uses sh to check the health of a Git repository — reporting uncommitted changes, recent commits, and branch status:
# git_health_check.py
import sh
import sys
def check_git_repo(path="."):
"""Check health of a git repository using sh."""
try:
# Verify it's a git repo
sh.git("-C", path, "rev-parse", "--is-inside-work-tree", _silent=True)
except sh.ErrorReturnCode:
print(f"Error: {path} is not a git repository")
sys.exit(1)
print(f"=== Git Health Check: {path} ===\n")
# Current branch
branch = sh.git("-C", path, "rev-parse", "--abbrev-ref", "HEAD").strip()
print(f"Branch: {branch}")
# Uncommitted changes
try:
sh.git("-C", path, "diff", "--quiet")
sh.git("-C", path, "diff", "--cached", "--quiet")
print("Working tree: Clean")
except sh.ErrorReturnCode:
changed = sh.git("-C", path, "diff", "--name-only").strip()
staged = sh.git("-C", path, "diff", "--cached", "--name-only").strip()
if changed:
print(f"Modified (unstaged):\n {changed.replace(chr(10), chr(10)+' ')}")
if staged:
print(f"Staged for commit:\n {staged.replace(chr(10), chr(10)+' ')}")
# Last 3 commits
print("\nLast 3 commits:")
log = sh.git("-C", path, "log", "--oneline", "-3")
for line in log:
print(f" {line}", end="")
# Ahead/behind remote
try:
status = sh.git("-C", path, "status", "-sb").split("\n")[0]
print(f"\nRemote status: {status.strip()}")
except sh.ErrorReturnCode:
print("\nRemote status: No remote configured")
if __name__ == "__main__":
repo_path = sys.argv[1] if len(sys.argv) > 1 else "."
check_git_repo(repo_path)
Output:
=== Git Health Check: . ===
Branch: main
Modified (unstaged):
src/utils.py
tests/test_main.py
Staged for commit:
README.md
Last 3 commits:
a1b2c3d Fix pagination bug in API client
e4f5a6b Add retry logic with exponential backoff
c7d8e9f Initial project structure
Remote status: ## main...origin/main [ahead 1]
This script shows how sh makes Git scripting feel natural. Each sh.git() call is a clear, readable expression of the underlying command. Error handling via try/except matches how you would handle git exit codes in bash. You could extend this to compare branches, check for merge conflicts, or send a Slack notification when the working tree is dirty.
Frequently Asked Questions
Does sh work on Windows?
sh does not support Windows. It relies on POSIX semantics for subprocess management that do not exist on Windows. If you are developing on Windows, use subprocess directly, or consider plumbum, which is a cross-platform alternative. On macOS and Linux, sh works fully.
When should I use sh instead of subprocess?
Use sh for developer tooling, automation scripts, and one-off tasks where readability matters more than security hardening. Use subprocess when you need precise control over the environment, when you are processing untrusted user input (to avoid injection risks), or when you are building a library that others will use. Both libraries ultimately launch child processes — sh just wraps the mechanics in a cleaner API.
How do I pass stdin to a command?
Use the _in parameter to pipe a string or file-like object to the command’s stdin. For example, sh.grep("error", _in=open("logfile.txt")) pipes the file to grep. You can also pass a string directly: sh.wc("-l", _in="line1\nline2\nline3"). For interactive commands that expect user input, use the _tty_in=True option.
How do I set environment variables for a command?
Pass a dictionary to the _env parameter: sh.mycommand(_env={"API_KEY": "abc123", "DEBUG": "true"}). Note that this replaces the entire environment, so if you want to extend the current environment, merge it first: import os; env = {**os.environ, "MY_VAR": "value"}; sh.mycommand(_env=env). This keeps all existing environment variables while adding or overriding specific ones.
How do I set a timeout on a command?
Use the _timeout parameter: sh.curl("https://example.com", _timeout=10) raises a TimeoutException if the command runs longer than 10 seconds. You can catch it with except sh.TimeoutException. This is cleaner than using _bg=True and manually monitoring a timer. For streaming commands, combine _timeout with _out=callback to process partial output before the timeout fires.
Conclusion
The sh library transforms shell command execution from a boilerplate chore into expressive Python code. You learned how to install sh, call commands with positional and keyword arguments, capture and stream output, pipe commands together, handle errors cleanly, and run background processes. The git health check project showed how these features combine into practical automation.
The next step is to take the health check script and extend it — add a check for dependency freshness using sh.pip("list", "--outdated"), or send alerts when certain conditions are met using sh.curl() to call a webhook. The sh documentation at sh.readthedocs.io covers advanced features including baking default arguments into reusable command objects using sh.Command.