Last Updated: June 01, 2026
Beginner
For most serious applications, you will often have to have persistent storage (storage that still exists after your applications stops running) of some sort. For new developers, it can be quite daunting to decide which option to go for. Is a simple flat file enough? When should you use something like a database? Which database should you use? There are so many options that are available it becomes quite daunting to decide which way to go for.
This is a starting guide to provide an overview of some of the many data storage options that are available for you and how you can go about deciding. One thing to keep in mind is that if you are developing an application which is either planned or has a possibility to scale over time, your underlying database might also grow overtime. It may be quick and easy to implement a file as storage, but as your data grows it might be better to use a relational database but it will take a little bit more effort. Let’s look at this a bit deeper

Python developer and educator with 15+ years building production systems across data engineering, web APIs, and AI tooling. Founder of Python How To Program — 270+ in-depth tutorials covering the modern Python stack.
What are the possible ways to store data?
There are many methods of persistent storage that you can use (persistent storage means that after your program is finished running your data is not lost). The typical ways you can do this is either by using a file which you save data to, or by using the python pickle mechanism. Firstly I will explain what some of the persistent storage options are:
- File: This is where you store the data in a text based file in format such as CSV (comma separated values), JSON, and others
- Python Pickle: A python pickle is a mechanism where you can save a data structure directly to a file, and then you can retrieve the data directly from the file next time you run your program. You can do this with a library called “pickle”
- Config files: config files are similar to File and Python Pickle in that the data is stored in a file format but is intended to be directly edited by a user
- Database SQLite: this is a database where you can run queries to search for data, but the data is stored in a file
- Database Postgres (or other SQL based database): this is a database service where there’s another program that you run to manage the database, and you call functions (or SQL queries) on the database service to get the data back in an efficient manner. SQL based databases are great for structured data – e.g. table-like/excel-like data. You would search for data by category fields as an example
- Key-value database (e.g redis is one of the most famous): A key-value database is exactly that, it contains a database where you search by a key, and then it returns a value. This value can be a single value or it can be a set of fields that are associated with that value. A common use of a key-value database is for hash-based data. Meaning that you have a specific key that you want to search for, and then you get all the related fields associated with that key – much like a dictionary in python, but the benefit being its in a persistent storage
- Graph Database (e.g. Neo4J): A graph database stores data which is built to navigate relationships. This is something that is rather cumbersome to do in a relational database where you need to have many intermediary tables but becomes trivial with GraphQL language
- Text Search (e.g. Elastic Search): A purpose built database for text search which is extremely fast when searching for strings or long text
- Time series database (e.g. influx): For IoT data where each record is stored with a timestamp key and you need to do queries in time blocks, time series databases are ideal. You can do common operations such as to aggregate, search, slice data through specific query operations
- NOSQL document database (e.g. mongodb, couchdb): this is a database that also runs as a separate service but is specifically for “unstructured data” (non-table like data) such as text, images where you search for records in a free form way such as by text strings.
There is no one persistent storage mechanism that fits all, it really depends on your purpose (or “use case”) to determine which database works best for you as there are pros and cons for each.
| Setup | Editable outside Python | Volume | Read Speed | Write Speed | Inbuilt Redundancy | |
| File | None – you can create a file in your python code | For text based | Small | Slow | Slow | No – manual |
| Python Pickle | None- you can create this in your python code | No – only in python | Small | Slow | Slow | No – manual |
| Config File | Optional. You can create a config file before hand | Yes – you can use any text based editor | Small | Slow | Slow | No – manual |
| Database SQLite | None – database created automatically | No – only in python | Small-Med | Slow-Med | Slow-Med | No – manual |
| Relational SQL Database | Separate installation of server | Through the SQL console or other SQL clients | Large | Fast | Fast | Yes, require extra setup |
| NoSQL Column Database | Separate installation of server | Yes, through external client | Very large | Very fast | Very fast | Yes, inbuilt |
| Key-Value database | Separate installation of server | Yes, through external client | Very large | Very fast | Fast-Very Fast | Yes, require extra setup |
| Graph Database | Separate installation of serverSeparate installation of server | Yes, through external client | Large | Med | Med | Yes, require extra setup |
| Time Series Database | Separate installation of server | Yes, through external client | Very large | Very fast | Fast | Yes, require extra setup |
| Text Search Database | Separate installation of server | Yes, through external client | Very large | Very fast | Fast | Yes, require extra setup |
| NoSQL Documet DB | Separate installation of server | Yes, through external client | Very large | Very fast | Fast | Yes, require extra setup |

A big disclaimer here, for some of the responses, the more accurate answer is “it depends”. For example, for redundancy for relational databases, some have it inbuilt such as Oracle RAC enterprise databases and for others you can set up redundancy where you could have an infrastructure solution. However, to provide a simpler guidance, I’ve made this a bit more prescriptive. If you would like to dive deeper, then please don’t rely purely on the table above! Look into the documentation of the particular database product you are considering or reach out to me and I’m happy to provide some advice.
Summary
There are in fact plenty of SaaS-based options for database or persistent storage that are popping up which is exciting. These newer SaaS options (for example, firebase, restdb.io, anvil.works etc) are great in that they save you time on the heavy lifting, but then there may be times you still want to manage your own database. This may be because you want to keep your data yourself, or simply because you want to save costs as you already have an environment either on your own laptop, or you’re paying a fixed price for a virtual machine. Hence, managing your own persistent storage may be more cost effective rather than paying for another SaaS. However, certainly don’t discount the SaaS options altogether, as they will at least help you with things like backups, security updates etc for you.
How To Use Python ward for Modern Python Testing
Intermediate
You have written enough test_something functions to last a lifetime. Your test files are full of names like test_user_creation_when_email_is_valid_and_age_is_over_18 — forty characters of snake_case just to describe one behaviour. When a test fails, the output tells you the function name but not what the test was actually checking. You end up reading the function body to figure it out, which defeats the point of a test name entirely.
Python’s ward library takes a different approach. Instead of naming test functions, you describe them — passing a plain English string to the @test decorator. When a test fails, ward prints exactly what it was supposed to do, in the language you wrote it, not a slug-cased identifier. The library also ships with a fixture system built on function arguments, a fluent expect() assertion API, and parameterized tests that read like a table of examples. Install it with pip install ward — no extra dependencies.
This article walks you through everything you need to use ward effectively. We will cover writing your first tests with the @test decorator, organizing shared setup with @fixture, running parameterized test cases, using the expect() assertion chain, and building a real-world test suite for a small utility library. By the end, you will have a working test suite and a clear picture of where ward fits against pytest.
Writing a ward Test: Quick Example
Before diving into the details, here is a minimal ward test file you can run right now. Create a file called test_math.py and paste in the following:
# test_math.py
from ward import test
@test("adding two positive integers returns their sum")
def _():
assert 1 + 2 == 3
@test("dividing by zero raises ZeroDivisionError")
def _():
try:
_ = 10 / 0
assert False, "should have raised"
except ZeroDivisionError:
pass
Run it with:
ward
Output:
PASS test_math adding two positive integers returns their sum
PASS test_math dividing by zero raises ZeroDivisionError
2 passed in 0.02 seconds
Two things stand out immediately. First, every test function is named _ — the name is irrelevant because ward uses the string you pass to @test instead. Second, the output reads like a sentence, not a mangled identifier. Those two changes alone make failing tests much easier to diagnose at a glance.
The sections below explain each ward feature in depth and show you how to apply them to realistic code.
What Is ward and Why Use It?
Ward is a Python testing framework designed to make test code more readable and test output more useful. It was created as an alternative to pytest and unittest, borrowing the best ideas from both while rethinking the parts that have always felt awkward — especially test naming and fixture injection.
The central idea is that a test is a fact you assert about your code, and that fact deserves to be written in human language. In pytest you write def test_cart_total_is_zero_when_no_items_added():. In ward you write @test("cart total is zero when no items are added"). The string is the documentation; the function body is the proof.
Here is how ward compares to pytest on the features most Python developers care about:
| Feature | pytest | ward |
|---|---|---|
| Test description | Function name (snake_case) | Plain English string |
| Fixtures | @pytest.fixture | @fixture with argument injection |
| Parameterization | @pytest.mark.parametrize | @using with each() |
| Assertions | assert (with rewriting) | assert or fluent expect() |
| Output format | Dots, F, E characters + tracebacks | Coloured PASS/FAIL lines with descriptions |
| Installation | pip install pytest | pip install ward |
Ward is not trying to replace pytest in every project — it is a deliberate choice that pays off most when you want test output to function as living documentation. If you share test runs with non-developers or treat CI output as a changelog, ward’s readable output earns its place immediately.
Installing ward
Ward requires Python 3.6 or later. Install it into your project’s virtual environment:
# install.sh
pip install ward
Verify the installation:
# verify_install.py
import ward
print(ward.__version__)
Output:
0.68.0b0
Ward discovers tests automatically. By default it searches for any file matching the pattern test_*.py in the current directory and its subdirectories — the same convention used by pytest. You can override the search path:
# Run all tests in a specific directory
ward --path tests/
# Run tests matching a keyword in their description
ward --search "total"
# Run tests in a single file
ward --path tests/test_cart.py
There is no configuration file required to get started. For larger projects, ward reads from a pyproject.toml [tool.ward] section if one exists.
Writing Tests with @test
The @test decorator is the foundation of every ward test suite. It takes a single string argument that describes what the test is verifying. The decorated function’s name is ignored entirely — the convention is to name every test function _ to make that clear.
# test_string_utils.py
from ward import test
def shout(text):
"""Convert text to uppercase with an exclamation mark."""
return text.upper() + "!"
@test("shout converts text to uppercase")
def _():
assert shout("hello") == "HELLO!"
@test("shout appends an exclamation mark")
def _():
result = shout("ward")
assert result.endswith("!")
@test("shout works on an already uppercase string")
def _():
assert shout("PYTHON") == "PYTHON!"
Output:
PASS test_string_utils shout converts text to uppercase
PASS test_string_utils shout appends an exclamation mark
PASS test_string_utils shout works on an already uppercase string
3 passed in 0.01 seconds
When a test fails, ward shows you the description, the file and line number, and the values that caused the failure. You get the “what failed” and “what were the values” without reading the traceback from the bottom up. That is a significant quality-of-life improvement when you have a large test suite and a CI run with dozens of failures.
Shared Setup with @fixture
Most tests need shared setup — a database connection, a sample data structure, or a configured object. Ward handles this with the @fixture decorator, which works very similarly to pytest fixtures. You declare a fixture function, then inject it into your test by using the fixture function as a default argument.
# test_user.py
from ward import test, fixture
@fixture
def sample_user():
return {
"name": "Alice",
"email": "alice@example.com",
"age": 25,
"active": True,
}
@test("user name is a string")
def _(user=sample_user):
assert isinstance(user["name"], str)
@test("user age is a positive integer")
def _(user=sample_user):
assert user["age"] > 0
@test("inactive users cannot be created with this fixture")
def _(user=sample_user):
assert user["active"] is True
Output:
PASS test_user user name is a string
PASS test_user user age is a positive integer
PASS test_user inactive users cannot be created with this fixture
3 passed in 0.01 seconds
The fixture function runs fresh for each test that uses it — there is no shared state between tests unless you explicitly use a module-level or session-level scope. To add teardown logic, use yield inside the fixture:
# test_file_fixture.py
import os
import tempfile
from ward import test, fixture
@fixture
def temp_file():
# Setup: create a temporary file
fd, path = tempfile.mkstemp(suffix=".txt")
os.close(fd)
with open(path, "w") as f:
f.write("hello ward")
yield path
# Teardown: delete the file after the test
if os.path.exists(path):
os.remove(path)
@test("temp file exists on disk")
def _(path=temp_file):
assert os.path.isfile(path)
@test("temp file contains the expected text")
def _(path=temp_file):
with open(path) as f:
content = f.read()
assert content == "hello ward"
Output:
PASS test_file_fixture temp file exists on disk
PASS test_file_fixture temp file contains the expected text
2 passed in 0.03 seconds
The code after yield runs automatically after each test that uses temp_file. Ward handles the teardown even if the test throws an exception, so you do not need try/finally blocks in your test functions.
Parameterized Tests with @using and each()
Parameterized tests let you run the same logic against multiple inputs without duplicating code. Ward’s approach is cleaner than pytest’s marker syntax — you use the @using decorator combined with the each() helper to define the value sets.
# test_math_ops.py
from ward import test, each, using
def clamp(value, lo, hi):
"""Clamp value to the range [lo, hi]."""
return max(lo, min(hi, value))
@test("clamp({value}, {lo}, {hi}) returns {expected}")
@using(
value=each(5, -3, 100, 50),
lo=each(0, 0, 0, 0),
hi=each(10, 10, 10, 10),
expected=each(5, 0, 10, 10),
)
def _(value, lo, hi, expected):
assert clamp(value, lo, hi) == expected
Output:
PASS test_math_ops clamp(5, 0, 10) returns 5
PASS test_math_ops clamp(-3, 0, 10) returns 0
PASS test_math_ops clamp(100, 0, 10) returns 10
PASS test_math_ops clamp(50, 0, 10) returns 10
4 passed in 0.01 seconds
Notice that ward interpolates the parameter values into the test description for each run. When one case fails, the output tells you exactly which input caused the problem — clamp(100, 0, 10) returns 10 is far more useful than test_clamp_params[2].
The each() values are matched positionally — the first value from every each() forms the first test case, the second values form the second, and so on. All each() calls in one @using decorator must have the same length.
Fluent Assertions with expect()
Ward provides an optional fluent assertion API called expect(). Instead of writing bare assert statements, you chain methods that read like English and produce more specific error messages when they fail. Use whichever style you prefer — both work the same way under the hood.
# test_expect.py
from ward import test, expect
@test("expect: integer is within range")
def _():
expect(42).to_be_greater_than(40)
expect(42).to_be_less_than(50)
@test("expect: string contains a substring")
def _():
expect("pythonhowtoprogram").to_contain("python")
@test("expect: list has expected length")
def _():
items = ["a", "b", "c"]
expect(items).has_length(3)
@test("expect: dictionary contains a key")
def _():
config = {"debug": True, "timeout": 30}
expect(config).to_contain_key("debug")
@test("expect: value is an instance of a type")
def _():
expect(3.14).to_be_instance_of(float)
Output:
PASS test_expect expect: integer is within range
PASS test_expect expect: string contains a substring
PASS test_expect expect: list has expected length
PASS test_expect expect: dictionary contains a key
PASS test_expect expect: value is an instance of a type
5 passed in 0.02 seconds
When an expect() assertion fails, ward reports both the expected condition and the actual value on separate lines. For example, if expect(15).to_be_greater_than(40) fails, the output reads Expected 15 to be greater than 40 — no need to open the test file and read the assert statement.
The expect() API supports chaining, so you can write several assertions about the same value in sequence. This is useful when a single test function validates multiple properties of one object, and you want each failure to be described independently.
Real-Life Example: Testing a Shopping Cart
Let us build a test suite for a simple shopping cart module using everything covered above: fixtures for shared state, parameterized tests for pricing logic, and expect() for readable assertions.
First, the module under test:
# cart.py
class Cart:
"""A simple shopping cart with item management and total calculation."""
def __init__(self):
self._items = {} # name -> {"price": float, "qty": int}
def add(self, name, price, qty=1):
if name in self._items:
self._items[name]["qty"] += qty
else:
self._items[name] = {"price": price, "qty": qty}
def remove(self, name):
self._items.pop(name, None)
def total(self, tax_rate=0.0):
subtotal = sum(v["price"] * v["qty"] for v in self._items.values())
return round(subtotal * (1 + tax_rate), 2)
def item_count(self):
return sum(v["qty"] for v in self._items.values())
def is_empty(self):
return len(self._items) == 0
Now the ward test suite:
# test_cart.py
from ward import test, fixture, expect, each, using
from cart import Cart
# ----- Fixtures -----
@fixture
def empty_cart():
return Cart()
@fixture
def stocked_cart():
c = Cart()
c.add("apple", price=0.99, qty=3)
c.add("bread", price=2.49)
c.add("milk", price=1.75, qty=2)
return c
# ----- Basic behaviour -----
@test("a new cart is empty")
def _(cart=empty_cart):
expect(cart.is_empty()).to_be_truthy()
expect(cart.item_count()).equals(0)
@test("adding an item increases item count")
def _(cart=empty_cart):
cart.add("apple", price=0.99, qty=4)
expect(cart.item_count()).equals(4)
@test("adding the same item twice increases quantity not entry count")
def _(cart=empty_cart):
cart.add("apple", 0.99, qty=1)
cart.add("apple", 0.99, qty=2)
expect(cart.item_count()).equals(3)
@test("removing an item reduces total")
def _(cart=stocked_cart):
before = cart.total()
cart.remove("bread")
expect(cart.total()).to_be_less_than(before)
# ----- Total calculation (parameterized) -----
@test("total with {rate*100:.0f}% tax is {expected}")
@using(
rate=each(0.0, 0.1, 0.2),
expected=each(9.46, 10.41, 11.35),
)
def _(cart=stocked_cart, rate=None, expected=None):
# stocked: 3*0.99 + 1*2.49 + 2*1.75 = 9.46 subtotal
expect(cart.total(tax_rate=rate)).equals(expected)
Output:
PASS test_cart a new cart is empty
PASS test_cart adding an item increases item count
PASS test_cart adding the same item twice increases quantity not entry count
PASS test_cart removing an item reduces total
PASS test_cart total with 0% tax is 9.46
PASS test_cart total with 10% tax is 10.41
PASS test_cart total with 20% tax is 11.35
7 passed in 0.03 seconds
The test suite demonstrates three things at once: the stocked_cart fixture gives each test a fresh, pre-populated cart without any shared mutation; the parameterized tax tests cover three pricing scenarios without duplicated code; and the expect() calls make each assertion readable without a comment explaining what is being checked. To extend the example, try adding tests for negative quantities, zero-price items, or a discount() method — the fixture and parameterization patterns scale without any structural changes.
Frequently Asked Questions
Can ward tests coexist with pytest tests in the same project?
Yes, but they run in separate commands. Ward discovers test_*.py files using the same convention as pytest, but the two runners are independent — running ward only executes tests decorated with @test, and running pytest only executes functions prefixed with test_. If you are migrating gradually, you can keep both frameworks active during the transition. Ward can also use pytest fixtures defined in a conftest.py file, which eases migration considerably.
How do fixture scopes work in ward?
Ward fixtures support four scopes: Scope.Test (default, reruns for every test), Scope.Module (reruns once per test file), Scope.Global (reruns once per entire test run), and Scope.Call (same as Test). Set the scope with @fixture(scope=Scope.Module). Use module or global scope for expensive operations like database connections or API clients that you do not want to recreate thousands of times.
Should I use expect() or plain assert statements?
Both work correctly and produce useful output. The expect() API gives you method names that double as documentation — expect(value).to_be_greater_than(10) is self-explanatory even in a code review. Plain assert is faster to type and feels more natural if you are coming from pytest. A common middle ground: use assert for straightforward equality checks and expect() when comparing ranges, types, or collection membership where the error message matters most.
How do I run a single test or a subset of tests?
Use the --search flag to filter by description substring: ward --search "cart total" will run every test whose description contains “cart total”. You can also use --path tests/test_cart.py to restrict the discovery scope to a single file. For more precise control, ward supports tags via @test("...", tags=["slow"]) combined with ward --tags slow on the command line.
Does ward support async tests?
Yes. Ward automatically detects async def test functions and fixture functions and runs them using asyncio.run(). You do not need any plugin or marker — just write async def _(): and ward handles the event loop. This works the same way for fixtures, so an async fixture can await a database connection and yield it to an async test without any additional setup.
How do I get code coverage with ward?
Run ward under coverage using the standard coverage.py wrapper: coverage run -m ward. After the run completes, generate the report with coverage report or coverage html for a browsable HTML breakdown. Ward does not bundle its own coverage tool — it relies on coverage.py, which is the same approach pytest recommends and integrates cleanly with CI pipelines and codecov.io.
Conclusion
Python ward replaces the test-naming ceremony of traditional frameworks with something more direct: a plain English description that appears in every test report, every CI log, and every team standup where someone asks “what broke?” The @test decorator, @fixture injection, @using parameterization, and the expect() assertion chain all work together to make tests read like specifications and failures read like bug reports.
The shopping cart example in this article covers the core patterns you will reach for in a real project. Try extending it — add a coupon fixture that returns a discount function, write parameterized tests for boundary cases like zero-quantity items, or switch the Cart storage to a database and test the async fixture pattern. Each of those extensions fits naturally into the structure already in place.
For the full API reference, see the official ward documentation at ward.readthedocs.io. The project is also actively maintained on GitHub at github.com/darrenburns/ward, where you can find the changelog and open issues.
Related Articles
Further Reading: For more details, see the Python sqlite3 documentation.
Frequently Asked Questions
What are the main data storage options in Python?
Python supports flat files (text, CSV, JSON), databases (SQLite, PostgreSQL, MySQL), key-value stores (Redis, shelve), pickle serialization, and cloud storage. The best choice depends on data size, structure, and access patterns.
When should I use SQLite vs a full database?
Use SQLite for single-user apps, prototypes, and small-to-medium datasets. Switch to PostgreSQL or MySQL for concurrent multi-user access, complex queries at scale, or production-grade reliability.
How do I save Python objects to disk?
Use pickle for Python-specific serialization, json for interoperable data, shelve for dictionary-like persistent storage, or databases for structured data. For data analysis, pandas can save to CSV, Parquet, or HDF5.
Is JSON or CSV better for storing data?
JSON handles nested, hierarchical data well. CSV is simpler for tabular, flat data. Use JSON for API data and configuration; use CSV for datasets and spreadsheet-compatible exports.
How do I choose between file storage and a database?
Use file storage for simple, single-user scenarios. Use a database when you need querying, indexing, concurrent access, or ACID transactions. SQLite bridges both worlds for simpler applications.
Related Articles
- How To Read and Write JSON Files in Python
- How To Use TOML for Configuration Files
- How To Use Pathlib for File Paths
Continue Learning Python
Tutorials you might also find useful:
- How To Work With JSON Data in Python Using the json Module
- How To Use Python PyArrow for Columnar Data Processing
- How To Use Python Plotly for Interactive Data Visualizations
- How To Use Python Streamlit for Interactive Data Apps
- How To Use Python cattrs for Structured Data Conversion
- How To Use Python glom for Nested Data Access