Beginner
The easiest and simplest mechanism to store data from python is the humble file storage which is often, but does not have to be, text based. There are no libraries that you require, and you can use native python functions to open and write to the file very easily.
There are many use cases for file storage and is usually the “go to” method when hacking a quick solution or prototype together. These are also arguably good solutions for production use cases.
Overview of using storing data to files in Python
The typical use cases has the following commonalities:
- Setup: There’s no setup that is required for files. You can create the file even from python
- Volume: Size Small-ish file size (< 5-10mb). You can go larger of course if your application is not doing heavy reads or writes nor if it doesn’t require fast response (e.g. batch processing)
- Record access: Does not require to search data within the file to extract just portion of the records. You would load or save all the data in the file in one go
- Data Writes: You can either append to the file or you can upload and download all data in the file.
- Write reliability: You do not need to have multiple writes at the same time – there is only possibility (or likelihood) of one person writing at one time, and if there was a case of multiple people writing at once, the consequence are not serious for your application. There are ways to put a lock on a file to prevent conflicts, but you should double check if a file is the write option for you
- Data formats: You may have structured record based (such as comma separated value – CSV or tab delimited) or unstructured (eg document of text or JSON format). You can also store binary data in a file as well – e.g. for images
- Editability: You may want or allow direct editing of the file by other applications or direct editing by people
- Redundancy: There’s no inbuilt redundancy. If there is any failure (data corrupt, the server with the file fails), then you’re out of luck. You need to setup your own mechanisms (e.g. replicate file to another server automatically)
Code examples to read and write to a file
Here are two sets of example code for writing and reading from a file. It is very easy and does not require any libraries. The one thing to be mindful of is what mode you want the file to be opened- read, write, read and write.
Open a text file for (over)writing:
To write to a file, it’s very easy to do so which is to use the ‘w’ switch on the open() function. There are other options as well:
‘r’– Reading‘w’– Writing to a file‘a’– Append to end of file‘r+’– Read and write to the same file‘x’– Used to create and write to a new file
file = open( ‘population.txt’, ‘w’)
file.write(‘Japan’)
file.write(‘United States’)
file.write(‘Australia’)
file.write(‘China’)
file.close() #file is released and closed
You will then have the following output file of population.txt:
Japan
United States
Australia
China
Open a text file fully for reading:
Using the same population.txt file created above –
file = open( ‘population.txt’, ‘r’)
data = file.read() #read full contents of file into a single string
file.close() #file is released and closed
print(“*** file start ***”)
print( data )
print(“*** end file ***”)
The output would be:
*** file start ***
Japan
United States
Australia
China
*** end file ***
Now to explain this a bit further, the open() command helps to open a file where you need to specify how the file is to be opened – in this case with ‘r’ to indicate it is for reading. There are other options as well:
‘r’– Reading‘w’– Writing to a file‘a’– Append to end of file‘r+’– Read and write to the same file‘x’– Used to create and write to a new file
Read a text file line by line:
file = open( ‘population.txt’, ‘r’)
data_list = file.readlines() #read full contents of file into a list of rows
file.close() #file is released and closed
print(“*** file start ***”)
counter = 0
for row in data_list:
counter = counter + 1
print( f”{counter}: {data_list}” )
print(“*** end file ***”)
The output would be:
*** file start ***
1: Japan
2: United States
3: Australia
4: China
*** end file ***
The difference in above to the first example is that the data comes out in a list separated by a newline so that you can process each row. Please note, you can simplify the above using the enumerate to avoid having the separate counter variable setup. E.g.
print(“*** file start ***”)
for index, row in enumerate(data_list):
print( f”{index+1}: {data_list}” ) #Note that when using enumerate, first index is 0
print(“*** end file ***”)
Summary of writing and reading to a file
Reading and writing to a file is a very straightforward native operation in Python. There are many other related operations that you can do ranging from putting a lock on a file to prevent two processes writing to the same file, checking file attributes such as access and size, and many other operations. At the most basic though, you can simply use the “open” statement to do the read/write to satisfy most of your needs.
How To Build CLI Apps with Python Click
Intermediate
Every serious Python developer eventually needs to build a command-line interface. Whether it is a deployment tool, a data processing script, or a developer utility, a well-designed CLI makes the difference between a tool your team actually uses and one that sits forgotten. Python’s standard argparse module works, but it is verbose — you write 20 lines of setup code before you handle your first argument. Click is the modern alternative: decorator-based, expressive, and composable, it cuts that boilerplate in half and adds features argparse simply does not have.
Click was created by the team behind Flask and follows the same philosophy: explicit is better than implicit, but explicit does not have to be painful. You decorate a Python function with @click.command() and @click.option(), and Click handles argument parsing, help text, type conversion, validation, and error messages automatically. Install it with pip install click.
This article covers everything you need to build production-quality CLI tools with Click: basic commands and options, arguments, type validation, prompts, multi-command groups (subcommands), progress bars, and output formatting. By the end, we will build a complete file management CLI that demonstrates all these features working together.
Click Quick Example
Here is a complete Click CLI that greets a user, with an optional count parameter:
# quick_click.py
import click
@click.command()
@click.option('--name', default='World', help='Who to greet.')
@click.option('--count', default=1, type=int, help='Number of greetings.')
@click.option('--loud', is_flag=True, help='Use uppercase.')
def greet(name, count, loud):
"""A friendly greeting command."""
for _ in range(count):
message = f"Hello, {name}!"
if loud:
message = message.upper()
click.echo(message)
if __name__ == '__main__':
greet()
Run it from the terminal:
$ python quick_click.py --name Alice --count 3
Hello, Alice!
Hello, Alice!
Hello, Alice!
$ python quick_click.py --name Bob --loud
HELLO, BOB!
$ python quick_click.py --help
Usage: quick_click.py [OPTIONS]
A friendly greeting command.
Options:
--name TEXT Who to greet.
--count INTEGER Number of greetings.
--loud Use uppercase.
--help Show this message and exit.
Click generated a complete help page automatically from the function’s docstring and decorator metadata. The --help flag, type validation, and default values all come for free.
Options vs Arguments
Click distinguishes between two kinds of inputs: options (named flags like --name Alice) and arguments (positional inputs like a filename). Options are optional by default; arguments are required by default.
| Feature | Option (@click.option) | Argument (@click.argument) |
|---|---|---|
| Syntax | --flag value | Positional: cmd value |
| Required | Optional by default | Required by default |
| Help text | Shown in --help | Shown in usage line |
| Best for | Configuration, flags | Primary inputs (files, names) |
# options_arguments.py
import click
@click.command()
@click.argument('filename') # Required positional arg
@click.option('--output', '-o', default='-', # -o is a short alias
help='Output file (default: stdout)')
@click.option('--lines', '-n', default=10,
type=int, help='Number of lines to show.')
@click.option('--verbose', '-v', is_flag=True,
help='Show extra information.')
def head(filename, output, lines, verbose):
"""Show the first N lines of FILENAME."""
if verbose:
click.echo(f"Reading {filename}, showing {lines} lines")
try:
with open(filename) as f:
for i, line in enumerate(f):
if i >= lines:
break
click.echo(line, nl=False)
except FileNotFoundError:
click.echo(f"Error: {filename} not found", err=True)
raise SystemExit(1)
if __name__ == '__main__':
head()
Run it as python options_arguments.py myfile.txt --lines 5 --verbose. The -o short alias for --output is defined right in the option decorator. Click handles both -o file.txt and --output file.txt automatically.
Types and Validation
Click converts option and argument values to the specified Python type and shows a helpful error if the conversion fails. Beyond basic types, Click has specialized types like click.Path for file paths and click.Choice for enumerated values.
# types_demo.py
import click
@click.command()
@click.argument('input_file', type=click.Path(exists=True, readable=True))
@click.option('--format', 'output_format',
type=click.Choice(['json', 'csv', 'text'], case_sensitive=False),
default='text', help='Output format.')
@click.option('--max-size', type=click.IntRange(1, 1000),
default=100, help='Max size (1-1000).')
@click.option('--scale', type=float, help='Scaling factor.')
def process(input_file, output_format, max_size, scale):
"""Process INPUT_FILE with validation."""
click.echo(f"Processing: {input_file}")
click.echo(f"Format: {output_format}")
click.echo(f"Max size: {max_size}")
if scale:
click.echo(f"Scale: {scale}")
if __name__ == '__main__':
process()
When you pass an invalid value, Click provides a clear error message:
$ python types_demo.py myfile.txt --format xml
Error: Invalid value for '--format': 'xml' is not one of 'json', 'csv', 'text'.
$ python types_demo.py nonexistent.txt
Error: Invalid value for 'INPUT_FILE': Path 'nonexistent.txt' does not exist.
click.Path(exists=True) validates the file exists before your function even runs. click.IntRange(1, 1000) ensures the integer is within bounds. These validations happen automatically and produce user-friendly error messages — no manual error handling needed.
Interactive Prompts and Confirmation
For destructive operations, you often want to confirm with the user. Click provides @click.confirmation_option(), @click.password_option(), and click.prompt() for interactive input collection.
# prompts_demo.py
import click
@click.command()
@click.option('--username', prompt='Username',
help='Your username.')
@click.option('--password', prompt=True,
hide_input=True, confirmation_prompt=True,
help='Your password.')
@click.option('--database', prompt='Database name',
default='mydb', show_default=True)
def setup_connection(username, password, database):
"""Set up a database connection."""
click.echo(f"Connecting to {database} as {username}...")
click.echo(f"Password length: {len(password)} chars")
# In a real app, you'd use these to create a connection
click.echo("Connection configured successfully!")
@click.command()
@click.argument('filename')
@click.confirmation_option(prompt='Are you sure you want to delete this file?')
def delete_file(filename):
"""Permanently delete FILENAME."""
import os
try:
os.remove(filename)
click.echo(f"Deleted: {filename}", err=False)
except FileNotFoundError:
click.echo(f"File not found: {filename}", err=True)
if __name__ == '__main__':
setup_connection()
Run python prompts_demo.py and Click interactively prompts for each required value. The password is hidden during input (no echo to terminal) and asks for confirmation. The @click.confirmation_option adds a yes/no prompt before any destructive action — and automatically processes -y or --yes flags to skip the prompt in automated scripts.
Multi-Command Groups (Subcommands)
Real CLI tools like git and docker use subcommands: git commit, git push, docker build, docker run. Click’s @click.group() decorator creates this structure cleanly. Each subcommand is just another decorated function.
# groups_demo.py
import click
@click.group()
@click.option('--debug/--no-debug', default=False,
help='Enable debug output.')
@click.pass_context
def cli(ctx, debug):
"""Project management tool."""
ctx.ensure_object(dict)
ctx.obj['DEBUG'] = debug
@cli.command()
@click.argument('name')
@click.option('--template', default='basic',
type=click.Choice(['basic', 'flask', 'fastapi']),
help='Project template.')
@click.pass_context
def create(ctx, name, template):
"""Create a new project."""
if ctx.obj['DEBUG']:
click.echo(f"[DEBUG] Creating {name} with template {template}")
click.echo(f"Creating project '{name}'...")
click.echo(f"Template: {template}")
click.echo(f"Done! Run: cd {name} && python main.py")
@cli.command()
@click.argument('name')
@click.pass_context
def delete(ctx, name):
"""Delete a project."""
if ctx.obj['DEBUG']:
click.echo(f"[DEBUG] Deleting {name}")
click.confirm(f"Delete project '{name}'? This cannot be undone.", abort=True)
click.echo(f"Project '{name}' deleted.")
@cli.command()
@click.pass_context
def list_projects(ctx):
"""List all projects."""
click.echo("Projects:")
for project in ['api-service', 'data-pipeline', 'dashboard']:
click.echo(f" - {project}")
# Register the list command with a different name
cli.add_command(list_projects, name='list')
if __name__ == '__main__':
cli()
Run it as:
$ python groups_demo.py --help
Usage: groups_demo.py [OPTIONS] COMMAND [ARGS]...
Project management tool.
Options:
--debug / --no-debug Enable debug output.
--help Show this message and exit.
Commands:
create Create a new project.
delete Delete a project.
list List all projects.
$ python groups_demo.py create myapp --template flask
Creating project 'myapp'...
Template: flask
Done! Run: cd myapp && python main.py
$ python groups_demo.py --debug create myapp
[DEBUG] Creating myapp with template basic
Creating project 'myapp'...
The ctx.pass_context pattern passes a shared context object through all subcommands. The --debug flag is defined on the group level and passed down through context — this is the Click pattern for global flags that affect all subcommands.
Real-Life Example: A File Processing CLI
Here is a complete, practical CLI tool for processing text files — counting words, searching for patterns, and converting case — with progress bars for large files.
# filetools.py
import click
import re
from pathlib import Path
@click.group()
def cli():
"""File processing toolkit."""
@cli.command()
@click.argument('files', nargs=-1, type=click.Path(exists=True), required=True)
@click.option('--words/--no-words', default=True, help='Count words.')
@click.option('--lines/--no-lines', default=True, help='Count lines.')
@click.option('--chars/--no-chars', default=False, help='Count characters.')
def count(files, words, lines, chars):
"""Count words/lines/chars in FILES."""
total_w, total_l, total_c = 0, 0, 0
for filepath in files:
content = Path(filepath).read_text()
w = len(content.split())
l = content.count('\n')
c = len(content)
total_w += w; total_l += l; total_c += c
parts = []
if lines: parts.append(f"{l:>8} lines")
if words: parts.append(f"{w:>8} words")
if chars: parts.append(f"{c:>8} chars")
click.echo(f"{' '.join(parts)} {filepath}")
if len(files) > 1:
click.echo(f"{'':->40}")
click.echo(f"{total_l:>8} lines {total_w:>8} words total")
@cli.command()
@click.argument('pattern')
@click.argument('files', nargs=-1, type=click.Path(exists=True), required=True)
@click.option('--ignore-case', '-i', is_flag=True, help='Case-insensitive.')
@click.option('--count-only', '-c', is_flag=True, help='Print match count only.')
def search(pattern, files, ignore_case, count_only):
"""Search for PATTERN in FILES."""
flags = re.IGNORECASE if ignore_case else 0
for filepath in files:
content = Path(filepath).read_text()
matches = [(i+1, line) for i, line in enumerate(content.splitlines())
if re.search(pattern, line, flags)]
if count_only:
click.echo(f"{len(matches):>5} {filepath}")
else:
for lineno, line in matches:
click.secho(f"{filepath}:{lineno}: ", nl=False, fg='cyan')
# Highlight the match in yellow
highlighted = re.sub(pattern,
lambda m: click.style(m.group(), fg='yellow', bold=True),
line, flags=flags)
click.echo(highlighted)
if __name__ == '__main__':
cli()
Run as:
$ python filetools.py count README.md
45 lines 312 words README.md
$ python filetools.py search "import" *.py --ignore-case
filetools.py:1: import click
filetools.py:2: import re
filetools.py:3: from pathlib import Path
The nargs=-1 pattern on FILES accepts any number of file arguments, like the Unix convention. click.secho() combines echo with styled output (colors). The --ignore-case short alias -i matches grep’s convention, making the tool feel natural to Unix users.
Frequently Asked Questions
When should I use Click instead of argparse?
Use Click for new CLI tools — it is less verbose and more composable. argparse is already in the standard library and requires no installation, so it is better for simple scripts that need zero dependencies. Click shines for multi-command CLIs with many options, complex validation, interactive prompts, and colored output. If you are building something beyond a simple script, Click’s developer experience wins decisively.
How does Click compare to Typer?
Typer is built on top of Click and generates Click CLI definitions from Python function type hints. If you use type annotations throughout your code, Typer reduces Click boilerplate further — you get options and arguments from type hints with no decorators. The trade-off: Typer adds a dependency and is less flexible than Click for complex CLI patterns. Click is more explicit; Typer is more magic. Both are excellent choices.
How do I test Click commands?
Click provides a CliRunner for testing. Use from click.testing import CliRunner; runner = CliRunner(); result = runner.invoke(my_command, ['--option', 'value']). The result object has exit_code, output, and exception attributes. This lets you test CLI behavior in pytest without spawning a subprocess, and it works with input prompts by passing input='yes\n' to invoke().
Can Click read options from environment variables?
Yes. Set auto_envvar_prefix='MYAPP' on the group, and Click automatically reads MYAPP_OPTION_NAME from the environment for any option not provided on the command line. You can also set it per-option: @click.option('--api-key', envvar='API_KEY'). This is the standard pattern for 12-factor applications where configuration comes from the environment.
How do I package a Click app as a proper CLI command?
Add an entry_points section to your pyproject.toml: [project.scripts] mytool = "mypackage.cli:main". After pip install -e ., running mytool in the terminal invokes your Click function directly. This is the standard way to distribute CLI tools on PyPI — users install your package and get the command available system-wide.
Conclusion
We covered the full Click toolkit: defining commands with @click.command(), options with @click.option(), arguments with @click.argument(), type validation with click.Path and click.Choice, interactive prompts, multi-command groups with shared context using @click.pass_context, and colored output with click.secho(). The file processing CLI showed how to compose these features into a tool that feels like a native Unix command.
From here, explore Click’s progress bar support (click.progressbar()), file path handling with lazy file opening, and the CliRunner for testing. Click’s plugin system also allows distributing CLI extensions as separate packages — the same pattern used by Flask extensions.
Official documentation: click.palletsprojects.com
Related Articles
Related Articles
- How To Read and Write JSON Files in Python 3
- How To Use the Logging Module in Python 3
- How To Split And Organise Your Source Code Into Multiple Files in Python 3
Further Reading: For more details, see the Python Input and Output tutorial.
Frequently Asked Questions
How do I read a text file in Python?
Use open('file.txt', 'r') with a with statement: with open('file.txt') as f: content = f.read(). This reads the entire file and automatically closes it. Use f.readlines() to get a list of lines instead.
What is the difference between read(), readline(), and readlines()?
read() returns the entire file as a single string. readline() reads one line at a time. readlines() returns a list of all lines. For large files, iterating with for line in f: is the most memory-efficient approach.
How do I write to a file in Python?
Use open('file.txt', 'w') to write (overwrites existing content) or 'a' to append. Write with f.write('text') or f.writelines(list_of_strings). Always use a with statement to ensure the file is properly closed.
What encoding should I use when reading text files?
Use encoding='utf-8' for most modern text files. UTF-8 handles international characters and is the default on most systems. For legacy files, you may need 'latin-1' or 'cp1252'.
How do I handle file not found errors in Python?
Use a try/except block catching FileNotFoundError. Alternatively, check if the file exists first with pathlib.Path('file.txt').exists() before attempting to read it.