Intermediate

You’ve probably seen the @ symbol above function definitions in Python code and wondered what it does. That’s a decorator — one of Python’s most powerful and elegant features. Decorators let you wrap a function with additional behavior (logging, caching, access control, rate limiting, timing) without modifying the function’s code. They’re the reason you can add authentication to a Flask route with a single line, or enable caching with @functools.lru_cache.

Decorators are a pure Python feature — no installation required. They’re built on Python’s first-class functions (functions that can be passed as arguments and returned from other functions). Once you understand how decorators work mechanically, you’ll be able to read and write the patterns used by virtually every Python framework, from Django’s @login_required to FastAPI’s @app.get() to pytest’s @pytest.fixture.

In this tutorial, you’ll learn how decorators work from first principles, how to use functools.wraps to preserve function metadata, how to write parameterized decorators (decorators that take arguments), how to stack multiple decorators, how to use class-based decorators, and how to apply these techniques in real-world scenarios like timing, retry logic, and access control.

Decorators: Quick Example

Here’s the simplest useful decorator — one that logs when a function is called:

# decorator_quick.py
import functools

def log_calls(func):
    @functools.wraps(func)
    def wrapper(*args, **kwargs):
        print(f"Calling {func.__name__}({args}, {kwargs})")
        result = func(*args, **kwargs)
        print(f"{func.__name__} returned {result}")
        return result
    return wrapper

@log_calls
def add(a, b):
    return a + b

# This is equivalent to: add = log_calls(add)
result = add(3, 4)
print(f"Final result: {result}")

# The function's identity is preserved
print(f"Function name: {add.__name__}")

Output:

Calling add((3, 4), {})
add returned 7
Final result: 7
Function name: add

The @log_calls syntax is shorthand for add = log_calls(add). The decorator receives the original function, returns a new wrapper function that adds behavior before and after calling the original, and replaces the name add with the wrapper. The @functools.wraps(func) line copies the original function’s name, docstring, and other metadata onto the wrapper — always include this.

How Decorators Work: First Principles

To truly understand decorators, you need to understand that in Python, functions are objects — they can be passed as arguments and returned from other functions. This is called “first-class functions.” Decorators are just a syntax shortcut for a function transformation pattern.

# first_class_functions.py

# Functions can be passed as arguments
def apply_twice(func, value):
    return func(func(value))

def double(x):
    return x * 2

result = apply_twice(double, 3)
print(f"Apply twice: {result}")  # 3 -> 6 -> 12

# Functions can be returned from other functions
def make_multiplier(n):
    def multiplier(x):
        return x * n
    return multiplier  # Returns the inner function

triple = make_multiplier(3)
print(f"Triple 5: {triple(5)}")  # 15

# The decorator pattern manually, without @ syntax
def shout(func):
    def wrapper(*args, **kwargs):
        result = func(*args, **kwargs)
        return result.upper() + "!!!"
    return wrapper

def greet(name):
    return f"Hello, {name}"

# Without @ syntax -- same result
greet = shout(greet)
print(greet("alice"))  # HELLO, ALICE!!!

Output:

Apply twice: 12
Triple 5: 15
HELLO, ALICE!!!

The key insight: @shout above a function definition is exactly equivalent to writing greet = shout(greet) after the definition. The @ syntax just makes it more readable and places the decoration visually near the function definition where it belongs.

Always Use functools.wraps

Without @functools.wraps(func), your decorator replaces the original function’s metadata with the wrapper’s. This causes problems with debugging, documentation, and tools that inspect function names. Always include it:

# wraps_example.py
import functools

# WITHOUT functools.wraps -- breaks function identity
def bad_decorator(func):
    def wrapper(*args, **kwargs):
        return func(*args, **kwargs)
    return wrapper

# WITH functools.wraps -- preserves identity
def good_decorator(func):
    @functools.wraps(func)
    def wrapper(*args, **kwargs):
        return func(*args, **kwargs)
    return wrapper

@bad_decorator
def my_function_bad():
    """This function does something important."""
    pass

@good_decorator
def my_function_good():
    """This function does something important."""
    pass

print(f"Bad decorator name:    {my_function_bad.__name__}")
print(f"Bad decorator docstr:  {my_function_bad.__doc__}")
print()
print(f"Good decorator name:   {my_function_good.__name__}")
print(f"Good decorator docstr: {my_function_good.__doc__}")

Output:

Bad decorator name:    wrapper
Bad decorator docstr:  None

Good decorator name:   my_function_good
Good decorator docstr: This function does something important.

Practical Decorator Examples

Timing Functions

A timer decorator measures how long a function takes to execute — great for performance monitoring and identifying bottlenecks:

# timer_decorator.py
import functools
import time

def timer(func):
    @functools.wraps(func)
    def wrapper(*args, **kwargs):
        start = time.perf_counter()
        result = func(*args, **kwargs)
        elapsed = time.perf_counter() - start
        print(f"{func.__name__} took {elapsed:.4f} seconds")
        return result
    return wrapper

@timer
def slow_function():
    time.sleep(0.1)
    return "done"

@timer
def sum_million():
    return sum(range(1_000_000))

slow_function()
result = sum_million()
print(f"Sum result: {result:,}")

Output:

slow_function took 0.1002 seconds
sum_million took 0.0312 seconds
Sum result: 499,999,500,000

Retry Logic

A retry decorator automatically re-runs a function if it raises an exception — essential for network calls, database operations, and any code that can fail transiently:

# retry_decorator.py
import functools
import time
import random

def retry(max_attempts=3, delay=1.0, exceptions=(Exception,)):
    """Decorator factory: retries a function up to max_attempts times."""
    def decorator(func):
        @functools.wraps(func)
        def wrapper(*args, **kwargs):
            last_error = None
            for attempt in range(1, max_attempts + 1):
                try:
                    return func(*args, **kwargs)
                except exceptions as e:
                    last_error = e
                    print(f"Attempt {attempt}/{max_attempts} failed: {e}")
                    if attempt < max_attempts:
                        time.sleep(delay)
            raise last_error
        return wrapper
    return decorator

# Simulated unreliable function (fails 70% of the time)
call_count = 0

@retry(max_attempts=5, delay=0.1, exceptions=(ValueError,))
def unreliable_api_call():
    global call_count
    call_count += 1
    if random.random() < 0.7:
        raise ValueError(f"API timeout on call #{call_count}")
    return f"Success on call #{call_count}"

random.seed(42)
result = unreliable_api_call()
print(f"Final result: {result}")

Output:

Attempt 1/5 failed: API timeout on call #1
Attempt 2/5 failed: API timeout on call #2
Attempt 3/5 failed: API timeout on call #3
Final result: Success on call #4

Notice the decorator factory pattern: retry(max_attempts=5, delay=0.1) returns a decorator, which then returns a wrapper. This is a three-level nesting -- outer function configures, middle function receives the function to decorate, inner function is what actually runs. This is the standard pattern for parameterized decorators.

Parameterized Decorators

When your decorator needs configuration (like the number of retries in the example above), you add one more level of nesting -- a "decorator factory" that takes the parameters and returns the actual decorator:

# parameterized_decorator.py
import functools

def repeat(n):
    """Call the decorated function n times."""
    def decorator(func):
        @functools.wraps(func)
        def wrapper(*args, **kwargs):
            results = []
            for _ in range(n):
                results.append(func(*args, **kwargs))
            return results
        return wrapper
    return decorator

@repeat(3)
def say_hello(name):
    return f"Hello, {name}!"

results = say_hello("Alice")
for r in results:
    print(r)

Output:

Hello, Alice!
Hello, Alice!
Hello, Alice!

Stacking Multiple Decorators

You can apply multiple decorators to the same function by stacking them. They apply from bottom to top (closest to the function first):

# stacking_decorators.py
import functools
import time

def timer(func):
    @functools.wraps(func)
    def wrapper(*args, **kwargs):
        start = time.perf_counter()
        result = func(*args, **kwargs)
        print(f"  [timer] {func.__name__}: {time.perf_counter()-start:.4f}s")
        return result
    return wrapper

def log_result(func):
    @functools.wraps(func)
    def wrapper(*args, **kwargs):
        result = func(*args, **kwargs)
        print(f"  [log] {func.__name__} returned: {result}")
        return result
    return wrapper

# Applied bottom-up: log_result wraps the original,
# then timer wraps log_result's wrapper
@timer
@log_result
def compute(x, y):
    return x ** y

result = compute(2, 10)
print(f"Final result: {result}")

Output:

  [log] compute returned: 1024
  [timer] compute: 0.0001s
Final result: 1024

Real-Life Example: Access Control Decorators

Here's a practical access control system using decorators -- the same pattern used by web frameworks for route authentication:

# access_control.py
import functools

# Simulated current user session
current_user = {'name': 'alice', 'roles': ['user', 'editor'], 'logged_in': True}

def login_required(func):
    """Decorator that requires the user to be logged in."""
    @functools.wraps(func)
    def wrapper(*args, **kwargs):
        if not current_user.get('logged_in'):
            print(f"Access denied: login required for {func.__name__}")
            return None
        return func(*args, **kwargs)
    return wrapper

def require_role(role):
    """Decorator factory: requires the user to have a specific role."""
    def decorator(func):
        @functools.wraps(func)
        def wrapper(*args, **kwargs):
            if role not in current_user.get('roles', []):
                print(f"Access denied: '{role}' role required for {func.__name__}")
                return None
            return func(*args, **kwargs)
        return wrapper
    return decorator

@login_required
def view_dashboard():
    return f"Dashboard for {current_user['name']}"

@login_required
@require_role('admin')
def delete_user(user_id):
    return f"Deleted user {user_id}"

@login_required
@require_role('editor')
def publish_post(post_id):
    return f"Published post {post_id}"

# Alice is logged in and has 'editor' but not 'admin'
print(view_dashboard())
print(delete_user(42))
print(publish_post(101))

# Simulate a logged-out user
current_user['logged_in'] = False
print(view_dashboard())

Output:

Dashboard for alice
Access denied: 'admin' role required for delete_user
Published post 101
Access denied: login required for view_dashboard

This is the exact pattern used by Flask's @login_required and Django's @permission_required. The decorators are reusable across any number of functions -- add access control to a new function by adding one line above its definition. The stacked @login_required @require_role('admin') means the user must pass both checks: logged in AND has the required role.

Frequently Asked Questions

When should I use a decorator instead of a helper function?

Use a decorator when you want to add the same cross-cutting behavior (logging, timing, validation, caching) to multiple functions without repeating the logic. If you find yourself writing the same "before" and "after" code in many functions, that's a strong signal to extract it into a decorator. For one-off or highly specific behavior, a regular helper function is simpler.

Can I use a class as a decorator?

Yes -- any callable can be a decorator. A class with a __call__ method works as a decorator. Class-based decorators are useful when you need to maintain state between calls (like call counts or cached results). Define __init__(self, func) to receive the function and __call__(self, *args, **kwargs) to wrap it. The @functools.wraps(func) approach works on __call__ too.

Do decorators work on class methods?

Yes, but with one caveat: the first argument of instance methods is self. Since decorators use *args, **kwargs, this is handled automatically. However, @staticmethod and @classmethod are themselves decorators. When stacking with them, always place @staticmethod or @classmethod outermost (closest to the def).

What is @functools.lru_cache and when should I use it?

@functools.lru_cache(maxsize=128) memoizes a function's return values -- if the function is called again with the same arguments, it returns the cached result instead of recomputing. Use it for pure functions (no side effects) that are called repeatedly with the same inputs. It's especially powerful for recursive functions like Fibonacci where the same sub-problems repeat many times.

Why does my IDE show wrong type hints after applying a decorator?

Without @functools.wraps, the decorated function's signature shows as (*args, **kwargs) -- losing the original type hints. With @functools.wraps, the function identity is preserved, but the signature the type checker sees is still the wrapper's. For full type hint preservation in decorated functions, use typing.ParamSpec and typing.Concatenate (Python 3.10+) to annotate the wrapper correctly.

Conclusion

Decorators are one of Python's most powerful code-reuse mechanisms. In this tutorial, you learned how Python's first-class functions make decorators possible, why @functools.wraps(func) is essential in every decorator, how to write practical decorators for timing, retry logic, and logging, how to create parameterized decorators using a decorator factory pattern, how to stack multiple decorators on a single function, and how the access control pattern mirrors real framework implementations.

The access control project is a foundation you can extend: add role inheritance, time-based access restrictions, or rate limiting. Every web framework you'll encounter -- Flask, Django, FastAPI -- relies heavily on decorators for its most important features.

For deeper coverage, see the functools module documentation and PEP 318 which introduced decorator syntax to Python.

Related Articles

• How To Use Python functools for Higher-Order Functions
• How To Use Python Property Decorators and Descriptors
• How To Use Python Closures and Nested Functions

Intermediate

Imagine you need to process a log file with 10 million lines. The naive approach — reading the whole file into a list first — would use several gigabytes of memory before you even start processing. Python generators solve this elegantly: instead of building the entire sequence in memory, they produce values one at a time, on demand. This “lazy evaluation” approach lets you work with datasets of any size using a constant, tiny amount of memory.

Generators are a core Python feature that you’ll find everywhere in Python’s standard library — range(), zip(), enumerate(), and file iteration all use generator-like lazy evaluation. Understanding generators not only helps you write memory-efficient code, it also makes you a better Python developer because you’ll understand why these built-in tools work the way they do. No installation needed — generators are a built-in language feature.

In this tutorial, you’ll learn how to create generators with yield, understand how the generator protocol works under the hood, use generator expressions as compact alternatives to list comprehensions, delegate to sub-generators with yield from, build generator pipelines for data processing, and apply all of this in a practical log file analysis project.

Generators: Quick Example

Here’s a generator function next to its equivalent list-based function, demonstrating the memory difference:

# generators_quick.py
import sys

# List version: builds entire sequence in memory
def first_n_squares_list(n):
    return [i * i for i in range(n)]

# Generator version: produces values one at a time
def first_n_squares_gen(n):
    for i in range(n):
        yield i * i

# Compare memory usage
squares_list = first_n_squares_list(1000000)
squares_gen = first_n_squares_gen(1000000)

print(f"List size:      {sys.getsizeof(squares_list):,} bytes")
print(f"Generator size: {sys.getsizeof(squares_gen):,} bytes")

# Use the generator just like any iterable
total = sum(squares_gen)
print(f"Sum of first 1M squares: {total:,}")

Output:

List size:      8,448,728 bytes
Generator size: 104 bytes
Sum of first 1M squares: 333,332,833,333,500,000

The generator object itself is only 104 bytes regardless of how many values it will produce. The list consumed over 8 MB. Both can be iterated the same way — sum() works with any iterable. The key difference is when and how the values are created.

What Are Generators and How Do They Work?

A generator function looks like a regular function, but uses yield instead of return. When you call a generator function, it doesn’t execute the function body — it returns a generator object. The body executes lazily, only when you ask for the next value.

The yield keyword does two things: it sends a value out of the generator, and it pauses execution at that point. The generator’s local variables and execution state are preserved between yields. When next() is called again, execution resumes from right after the yield statement.

The Generator Protocol

Generators implement Python’s iterator protocol: they have a __next__() method. You can call next() manually to see exactly how this works step by step:

# generator_protocol.py

def countdown(n):
    print(f"Starting countdown from {n}")
    while n > 0:
        print(f"  About to yield {n}")
        yield n
        print(f"  Resumed after yielding {n}")
        n -= 1
    print("Countdown complete!")

# Create the generator object (nothing runs yet)
gen = countdown(3)
print(f"Generator object: {gen}")
print()

# Manually advance the generator
val1 = next(gen)
print(f"Got: {val1}\n")

val2 = next(gen)
print(f"Got: {val2}\n")

val3 = next(gen)
print(f"Got: {val3}\n")

# One more next() raises StopIteration
try:
    next(gen)
except StopIteration:
    print("Generator exhausted -- StopIteration raised")

Output:

Generator object: <generator object countdown at 0x7f8b1c2d3a50>

Starting countdown from 3
  About to yield 3
Got: 3

  Resumed after yielding 3
  About to yield 2
Got: 2

  Resumed after yielding 2
  About to yield 1
Got: 1

  Resumed after yielding 1
Countdown complete!
Generator exhausted -- StopIteration raised

This output reveals the exact sequence: calling countdown(3) did nothing. The first next(gen) started execution, ran until the first yield 3, and paused. The second next(gen) resumed right after that yield. For loops handle StopIteration automatically — they call next() and stop when the exception is raised.

Generator Expressions

Generator expressions are the compact syntax for creating simple generators — they look exactly like list comprehensions but use parentheses instead of square brackets. They’re ideal for single-use transformations passed directly to functions:

# generator_expressions.py

# List comprehension: builds all values immediately
squares_list = [x*x for x in range(10)]
print(f"List: {squares_list}")

# Generator expression: lazy, no brackets
squares_gen = (x*x for x in range(10))
print(f"Generator: {squares_gen}")
print(f"Sum via generator: {sum(squares_gen)}")

# Use generator expressions inline -- no extra parentheses needed
total = sum(x*x for x in range(1000000))
print(f"Sum of 1M squares: {total:,}")

# Filter with generator expressions
big_squares = (x*x for x in range(100) if x*x > 500)
print(f"Squares > 500: {list(big_squares)[:5]}...")

# Chained transformations (memory-efficient pipeline)
data = range(1, 1000001)
even_nums = (x for x in data if x % 2 == 0)
squared = (x*x for x in even_nums)
under_million = (x for x in squared if x < 1_000_000)
result = list(under_million)
print(f"Even squares under 1M: {len(result)} values")

Output:

List: [0, 1, 4, 9, 16, 25, 36, 49, 64, 81]
Generator: <generator object <genexpr> at 0x7f8b1c2d3b60>
Sum via generator: 285
Sum of 1M squares: 333,332,833,333,500,000
Squares > 500: [529, 576, 625, 676, 729]...
Even squares under 1M: 499 values

Delegating with yield from

yield from lets a generator delegate to another iterable -- it's cleaner than looping and yielding each item manually. This is especially useful when building recursive generators or combining multiple generators:

# yield_from.py

# Without yield from -- verbose
def flatten_manual(nested):
    for sublist in nested:
        for item in sublist:
            yield item

# With yield from -- cleaner
def flatten(nested):
    for sublist in nested:
        yield from sublist

data = [[1, 2, 3], [4, 5], [6, 7, 8, 9]]
print("Flattened:", list(flatten(data)))

# Chain multiple generators with yield from
def chain_generators(*iterables):
    for it in iterables:
        yield from it

gen1 = (x for x in range(3))
gen2 = (x*10 for x in range(3))
gen3 = ['a', 'b', 'c']

chained = list(chain_generators(gen1, gen2, gen3))
print("Chained:", chained)

Output:

Flattened: [1, 2, 3, 4, 5, 6, 7, 8, 9]
Chained: [0, 1, 2, 0, 10, 20, 'a', 'b', 'c']

Real-Life Example: Log File Analyzer

Here's a practical generator pipeline that processes a large log file without loading it all into memory at once. This pattern handles files of any size efficiently:

# log_analyzer.py
import re
from datetime import datetime

# Sample log data (representing a large file in real use)
SAMPLE_LOG = """2026-04-17 09:00:01 INFO  User alice logged in
2026-04-17 09:01:15 ERROR Database connection timeout
2026-04-17 09:01:16 ERROR Retrying connection (attempt 1/3)
2026-04-17 09:01:17 INFO  Database reconnected successfully
2026-04-17 09:02:33 WARNING High memory usage: 87%
2026-04-17 09:03:45 INFO  User bob logged in
2026-04-17 09:04:12 ERROR Disk write failed: /var/log/app.log
2026-04-17 09:05:00 INFO  Backup completed successfully
2026-04-17 09:06:22 ERROR Authentication failed for user charlie
2026-04-17 09:07:11 INFO  Scheduled job completed in 1.23s"""

# Generator: read lines one at a time (use open() for real files)
def read_lines(text):
    for line in text.strip().splitlines():
        yield line

# Generator: parse each line into a structured dict
LOG_PATTERN = re.compile(
    r'(\d{4}-\d{2}-\d{2} \d{2}:\d{2}:\d{2}) (\w+)\s+(.+)'
)

def parse_logs(lines):
    for line in lines:
        match = LOG_PATTERN.match(line)
        if match:
            timestamp_str, level, message = match.groups()
            yield {
                'timestamp': datetime.strptime(timestamp_str, '%Y-%m-%d %H:%M:%S'),
                'level': level,
                'message': message
            }

# Generator: filter to only error entries
def filter_level(entries, level):
    for entry in entries:
        if entry['level'] == level:
            yield entry

# Build the pipeline (nothing runs yet)
lines = read_lines(SAMPLE_LOG)
parsed = parse_logs(lines)
errors = filter_level(parsed, 'ERROR')

# Consume the pipeline -- data flows through all stages
print("ERROR log entries:")
for entry in errors:
    time_str = entry['timestamp'].strftime('%H:%M:%S')
    print(f"  [{time_str}] {entry['message']}")

# Rebuild and get counts (generator is exhausted after one pass)
lines = read_lines(SAMPLE_LOG)
all_entries = list(parse_logs(lines))
counts = {}
for entry in all_entries:
    counts[entry['level']] = counts.get(entry['level'], 0) + 1

print("\nLog level summary:")
for level, count in sorted(counts.items()):
    print(f"  {level}: {count}")

Output:

ERROR log entries:
  [09:01:15] Database connection timeout
  [09:01:16] Retrying connection (attempt 1/3)
  [09:04:12] Disk write failed: /var/log/app.log
  [09:06:22] Authentication failed for user charlie

Log level summary:
  ERROR: 4
  INFO: 5
  WARNING: 1

This pipeline pattern is memory-efficient because at any moment, only one log line exists in memory as it flows through read_lines -> parse_logs -> filter_level. For a 10 GB log file, this approach uses the same tiny amount of memory as it does for a 10-line file. To use this with a real log file, replace read_lines(SAMPLE_LOG) with open('app.log', 'r') -- file objects are themselves iterable generators in Python.

Frequently Asked Questions

Can I iterate a generator multiple times?

No -- generators are single-use. Once exhausted, calling next() always raises StopIteration. If you need to iterate multiple times, either store the values in a list first (items = list(my_gen())), or call the generator function again to create a fresh generator object. This is why generator expressions are usually passed directly to functions like sum() or list() that consume them in one pass.

What is the difference between a generator and an iterator?

Every generator is an iterator, but not every iterator is a generator. An iterator is any object with a __next__() and __iter__() method. A generator is a specific way to create an iterator using a function with yield -- Python automatically implements the iterator protocol for you. Generators are the most convenient way to create iterators in Python.

What does generator.send() do?

The .send(value) method resumes a generator and sends a value in as the result of the current yield expression. This turns generators into coroutines -- two-way communication channels. It's used in advanced patterns like cooperative multitasking. For most use cases, you'll never need .send() -- standard next() iteration is sufficient.

When should I use a generator vs. a list?

Use a generator when: you're processing a large sequence and don't need all values in memory at once, you're reading from a file or network stream, you only need to iterate once, or you're building a pipeline of transformations. Use a list when: you need to index into the sequence, iterate multiple times, get its length with len(), or pass it to code that explicitly expects a list.

Can generators produce infinite sequences?

Yes -- this is one of the most powerful uses of generators. A generator can loop indefinitely, yielding values forever, because it never builds a finite collection in memory. The standard library's itertools.count() and itertools.cycle() are examples of infinite generators. Just make sure your consuming code has a termination condition (like islice or a loop with a break) to stop pulling values.

Conclusion

Generators are one of Python's most elegant features. In this tutorial, you learned how yield turns a function into a lazy generator, how the generator protocol works with next() and StopIteration, how generator expressions provide compact syntax for simple generators, how yield from delegates to sub-iterables, and how to compose generators into data processing pipelines.

The log analyzer project shows the real-world payoff: a memory-efficient pipeline the scales to gigabyte files with no changes. Try extending it to count errors per hour, find the longest gap between errors, or write the filtered entries to a new file.

For more on generators and iteration in Python, see the Python HOWTO: Generators and the itertools documentation for powerful generator-based utilities.

Related Articles

• How To Use Python itertools for Efficient Looping
• How To Master Python List Comprehensions with Examples
• How To Use Python contextlib for Resource Management

Intermediate

Every production Python application needs logging. Not print() statements that vanish when your script closes — real, structured logs with timestamps, severity levels, file rotation, and the ability to turn verbosity up or down without touching your code. When something goes wrong at 2am on a production server, your log file is the only witness. If all you left behind are a few print("here") calls, you’re debugging blind.

Python ships with a powerful, flexible logging module in its standard library. It’s built around a hierarchy of loggers, handlers, and formatters that you configure once and use everywhere. The learning curve is a bit steeper than print(), but the payoff — structured, timestamped, level-filtered, file-backed logs — is enormous. No third-party packages are required to get started.

In this article we’ll cover the five log levels, the basicConfig shortcut, named loggers and the logger hierarchy, handlers (console, file, rotating), formatters, logging from multiple modules, and a complete real-world logging setup for a data pipeline application. By the end, you’ll have a professional logging setup you can drop into any project.

Python Logging: Quick Example

Here’s the fastest way to get meaningful logging output with timestamps and levels:

# quick_logging.py
import logging

logging.basicConfig(
    level=logging.DEBUG,
    format='%(asctime)s - %(levelname)s - %(message)s',
    datefmt='%Y-%m-%d %H:%M:%S'
)

logging.debug('This is a debug message')
logging.info('Application started')
logging.warning('Disk space running low')
logging.error('Failed to connect to database')
logging.critical('Application cannot continue')

Output:

2026-04-16 09:00:01 - DEBUG - This is a debug message
2026-04-16 09:00:01 - INFO - Application started
2026-04-16 09:00:01 - WARNING - Disk space running low
2026-04-16 09:00:01 - ERROR - Failed to connect to database
2026-04-16 09:00:01 - CRITICAL - Application cannot continue

basicConfig() sets up the root logger — the single logging object all other loggers inherit from if not configured themselves. The format string uses special tokens like %(asctime)s, %(levelname)s, and %(message)s. In the sections below we’ll go beyond the root logger to set up named, per-module loggers and file handlers.

What Is the logging Module and Why Use It?

The logging module provides a standardized way to emit messages from your application at different severity levels. Unlike print(), log messages carry metadata (timestamp, level, logger name, file, line number), can be routed to multiple destinations simultaneously (console AND file), and can be filtered by level without changing any code.

The level you set on a logger or handler acts as a filter: only messages at that level or higher are processed. Set to DEBUG in development to see everything; set to WARNING or ERROR in production to reduce noise. No code changes required — just a config change.

Named Loggers and the Logger Hierarchy

The best practice is to create a named logger for each module using __name__. This gives every log message a module-level identifier and lets you control logging granularity per module in large applications.

# named_logger.py
import logging

# Create a logger named after this module
logger = logging.getLogger(__name__)
logger.setLevel(logging.DEBUG)

# Create a console handler with formatting
handler = logging.StreamHandler()
handler.setLevel(logging.DEBUG)
formatter = logging.Formatter('%(asctime)s - %(name)s - %(levelname)s - %(message)s')
handler.setFormatter(formatter)
logger.addHandler(handler)

# Use the logger
logger.info('Module initialized')
logger.debug('Loading configuration from file')
logger.warning('Config file not found, using defaults')

Output:

2026-04-16 09:00:01 - __main__ - INFO - Module initialized
2026-04-16 09:00:01 - __main__ - DEBUG - Loading configuration from file
2026-04-16 09:00:01 - __main__ - WARNING - Config file not found, using defaults

Loggers form a hierarchy based on their names. A logger named myapp.database is a child of myapp, which is a child of the root logger. Messages propagate up the hierarchy by default — so configuring handlers on the root logger or a parent logger affects all children. This hierarchy is what makes it possible to configure logging once in your main module and have it work across all your imports.

Logging to a File

Writing logs to a file ensures you have a record of what happened, even after the terminal session closes. The FileHandler writes log messages to a file you specify.

# file_logging.py
import logging

logger = logging.getLogger('myapp')
logger.setLevel(logging.DEBUG)

# Console handler -- only show WARNING and above in the terminal
console_handler = logging.StreamHandler()
console_handler.setLevel(logging.WARNING)
console_handler.setFormatter(logging.Formatter('%(levelname)s: %(message)s'))

# File handler -- write everything DEBUG and above to a file
file_handler = logging.FileHandler('app.log', encoding='utf-8')
file_handler.setLevel(logging.DEBUG)
file_handler.setFormatter(
    logging.Formatter('%(asctime)s - %(name)s - %(levelname)s - %(message)s')
)

logger.addHandler(console_handler)
logger.addHandler(file_handler)

# Now emit messages
logger.debug('Processing item 1')       # Only in file
logger.info('Item 1 processed OK')     # Only in file
logger.warning('Item 2 skipped')        # Console AND file
logger.error('Item 3 failed: timeout') # Console AND file

Terminal output:

WARNING: Item 2 skipped
ERROR: Item 3 failed: timeout

app.log contents:

2026-04-16 09:00:01 - myapp - DEBUG - Processing item 1
2026-04-16 09:00:01 - myapp - INFO - Item 1 processed OK
2026-04-16 09:00:01 - myapp - WARNING - Item 2 skipped
2026-04-16 09:00:01 - myapp - ERROR - Item 3 failed: timeout

This dual-handler pattern is extremely common in production: the console shows only what operators need to see in real time (warnings and errors), while the file captures the full diagnostic history for post-mortem debugging.

Rotating Log Files

Log files grow indefinitely if nothing manages them. The RotatingFileHandler automatically rotates log files when they hit a size limit, keeping a configurable number of backup files. The TimedRotatingFileHandler rotates on a schedule (daily, hourly, etc.).

# rotating_logs.py
import logging
from logging.handlers import RotatingFileHandler, TimedRotatingFileHandler

logger = logging.getLogger('rotating_demo')
logger.setLevel(logging.DEBUG)
formatter = logging.Formatter('%(asctime)s - %(levelname)s - %(message)s')

# Rotate when file hits 1MB, keep 5 backups
size_handler = RotatingFileHandler(
    'app_size.log',
    maxBytes=1_000_000,   # 1 MB
    backupCount=5,
    encoding='utf-8'
)
size_handler.setFormatter(formatter)

# Rotate daily at midnight, keep 7 days of logs
time_handler = TimedRotatingFileHandler(
    'app_daily.log',
    when='midnight',
    interval=1,
    backupCount=7,
    encoding='utf-8'
)
time_handler.setFormatter(formatter)

logger.addHandler(size_handler)
logger.addHandler(time_handler)

for i in range(100):
    logger.info(f'Processing record {i}')

print('Logging complete. Check app_size.log and app_daily.log')

Output:

Logging complete. Check app_size.log and app_daily.log

When app_size.log reaches 1MB, it’s renamed to app_size.log.1, then app_size.log.2, and so on up to the backupCount. Older backups are deleted automatically. For long-running services like web servers or data pipelines, TimedRotatingFileHandler with when='midnight' and backupCount=30 gives you a month of daily logs with zero maintenance.

Logging Exceptions

One of the most valuable logging features is capturing full exception tracebacks. Use logger.exception() inside an except block — it logs the message at ERROR level and automatically appends the full traceback.

# exception_logging.py
import logging

logging.basicConfig(level=logging.DEBUG, format='%(asctime)s - %(levelname)s - %(message)s')
logger = logging.getLogger(__name__)

def divide(a, b):
    try:
        result = a / b
        logger.info(f'Divided {a} / {b} = {result}')
        return result
    except ZeroDivisionError:
        logger.exception(f'Failed to divide {a} by {b}')
        return None

divide(10, 2)
divide(10, 0)  # Will log the traceback

Output:

2026-04-16 09:00:01 - INFO - Divided 10 / 2 = 5.0
2026-04-16 09:00:01 - ERROR - Failed to divide 10 by 0
Traceback (most recent call last):
  File "exception_logging.py", line 8, in divide
    result = a / b
ZeroDivisionError: division by zero

logger.exception() is equivalent to logger.error(msg, exc_info=True). The traceback is automatically included — you don’t need to call traceback.format_exc() or format it yourself. This is the pattern every production application should use inside exception handlers.

Real-Life Example: Data Pipeline Logger

Here’s a complete logging setup for a data pipeline that processes records from a source file, transforms them, and writes them to an output file — with full logging at every stage.

# data_pipeline.py
import logging
import logging.config
import json
import os
from logging.handlers import RotatingFileHandler

def setup_logger(name, log_file, level=logging.DEBUG):
    """Create a configured logger with console and rotating file handlers."""
    logger = logging.getLogger(name)
    logger.setLevel(level)

    if logger.handlers:
        return logger  # Prevent duplicate handlers on re-import

    formatter = logging.Formatter(
        '%(asctime)s | %(name)-20s | %(levelname)-8s | %(message)s',
        datefmt='%Y-%m-%d %H:%M:%S'
    )

    # Console: WARNING and above
    console = logging.StreamHandler()
    console.setLevel(logging.WARNING)
    console.setFormatter(formatter)

    # File: everything, rotate at 500KB, keep 3 backups
    fh = RotatingFileHandler(log_file, maxBytes=500_000, backupCount=3, encoding='utf-8')
    fh.setLevel(logging.DEBUG)
    fh.setFormatter(formatter)

    logger.addHandler(console)
    logger.addHandler(fh)
    return logger

def process_records(records, logger):
    """Process a list of record dicts. Returns (success_count, error_count)."""
    success = 0
    errors = 0

    for i, record in enumerate(records):
        try:
            if 'name' not in record:
                raise ValueError(f'Missing required field: name')
            if not isinstance(record.get('age', 0), int):
                raise TypeError(f'age must be an integer, got {type(record["age"]).__name__}')

            # Simulate transform
            transformed = {
                'id': i + 1,
                'name': record['name'].strip().title(),
                'age': record['age'],
                'status': 'active'
            }
            logger.debug(f'Processed record {i+1}: {transformed["name"]}')
            success += 1

        except (ValueError, TypeError) as e:
            logger.warning(f'Skipping record {i+1}: {e}')
            errors += 1
        except Exception as e:
            logger.exception(f'Unexpected error on record {i+1}')
            errors += 1

    return success, errors

def run_pipeline(input_records, output_path):
    logger = setup_logger('pipeline', 'pipeline.log')

    logger.info(f'Pipeline started. Input records: {len(input_records)}')

    success, errors = process_records(input_records, logger)

    logger.info(f'Pipeline complete. Success: {success}, Errors: {errors}')
    if errors > 0:
        logger.warning(f'{errors} records skipped due to errors')

    return success, errors

# Run the pipeline
sample_data = [
    {'name': 'alice johnson', 'age': 30},
    {'name': 'bob smith', 'age': 25},
    {'age': 40},                         # Missing name -- will warn
    {'name': 'charlie brown', 'age': 'thirty'},  # Wrong type -- will warn
    {'name': 'diana prince', 'age': 28},
]

success, errors = run_pipeline(sample_data, 'output.json')
print(f'\nFinal result: {success} processed, {errors} skipped')
print('Check pipeline.log for full details')

Output:

WARNING | pipeline             | WARNING  | Skipping record 3: Missing required field: name
WARNING | pipeline             | WARNING  | Skipping record 4: age must be an integer, got str

Final result: 3 processed, 2 skipped
Check pipeline.log for full details

The setup_logger() function uses a guard (if logger.handlers: return logger) to prevent duplicate handlers when the function is called multiple times, which is a common gotcha in larger projects. The pipeline logs every step to the file (DEBUG level) while showing only warnings and errors on the console, giving operators a clean output while preserving the full diagnostic trail in the log file.

Frequently Asked Questions

When should I use basicConfig vs named loggers?

Use basicConfig() for scripts and quick tools where you just need output to the console. For any application with multiple modules, use named loggers (logging.getLogger(__name__)) so you can identify which module emitted each message and control logging per module. Named loggers are the standard for libraries and production code.

Why am I seeing duplicate log messages?

This almost always happens because you added a handler to a logger that also propagates to the root logger, which has its own handler. Fix it either by setting logger.propagate = False on your named logger, or by removing the handler from the root logger. The pattern if logger.handlers: return logger in a setup function also prevents duplicate handlers when the function is called more than once.

How do I turn off all logging in production?

Set the root logger level to logging.CRITICAL + 1 or logging.NOTSET and remove all handlers: logging.disable(logging.CRITICAL) silences everything at CRITICAL and below (effectively everything). More typically, just set the production handler level to WARNING or ERROR rather than disabling logging entirely — you want errors logged even in production.

How do I configure logging from a config file?

Use logging.config.fileConfig('logging.ini') for INI-format config files, or logging.config.dictConfig(config_dict) for dictionary-based config (which you can load from a JSON or YAML file). Dictionary config is the modern approach — it’s more flexible and easier to version-control alongside your application code.

Does logging slow down my application?

At DEBUG level with many log messages, yes, logging adds overhead — especially if writing to disk. In production, set the level to WARNING or ERROR so most logging.debug() and logging.info() calls return immediately without any I/O. For extremely hot code paths, check if logger.isEnabledFor(logging.DEBUG): before constructing expensive log messages.

Conclusion

Python’s logging module gives you a production-grade observability system built into the standard library. We covered the five log levels (DEBUG through CRITICAL), setting up named loggers with getLogger(__name__), combining console and file handlers with different levels, automatic log rotation with RotatingFileHandler and TimedRotatingFileHandler, capturing exception tracebacks with logger.exception(), and building a complete pipeline logger. Replace every print() statement in your applications with the appropriate logging call — your future debugging self will thank you.

Extend the data pipeline example by loading the logging configuration from a JSON file so it can be adjusted without code changes, or add a SMTPHandler to email you when a CRITICAL event fires. The logging module’s handler ecosystem is extensive.

See the official logging documentation and the logging cookbook for advanced patterns including thread-safe logging and multiprocessing log handlers.

Related Articles

• How To Use Python subprocess Module to Run System Commands
• How To Read and Write JSON Files in Python
• How To Use Python Regular Expressions with the re Module

Intermediate

You wrote unit tests. You covered the happy path. You even tested a few edge cases — empty strings, zero values, negative numbers. But then production breaks on an input you never imagined: a Unicode string with a zero-width space, a list with 2 billion elements, or a float that is technically not-a-number. Property-based testing is the approach that finds these bugs before your users do. Instead of you specifying test inputs, the library generates hundreds of random inputs automatically and searches for ones that break your code.

Hypothesis is Python’s leading property-based testing library. You describe the shape of valid inputs using strategies, and Hypothesis generates inputs of that shape, tries to break your code, and if it finds a failing case, automatically shrinks it to the smallest possible example that still fails. This gives you a precise, minimal reproduction case instead of a random mess. Install it with pip install hypothesis. It works alongside pytest and unittest with zero configuration.

In this article, we will cover: what property-based testing is and when to use it, how to write your first Hypothesis test, how to use built-in strategies for common types, how to compose strategies for custom data structures, how to use stateful testing for sequences of operations, and how to apply Hypothesis to real code to find real bugs. By the end, you will have a new tool that makes your test suite dramatically more thorough.

Hypothesis Quick Example

Here is a Hypothesis test that checks a property of Python’s built-in sorted() function — that sorting a list and then reversing it should equal sorting in reverse order:

# quick_hypothesis.py
from hypothesis import given
from hypothesis import strategies as st

@given(st.lists(st.integers()))
def test_sort_reverse_equivalent(numbers):
    """sorted then reversed == sorted with reverse=True"""
    sorted_then_reversed = list(reversed(sorted(numbers)))
    sorted_reversed = sorted(numbers, reverse=True)
    assert sorted_then_reversed == sorted_reversed

# Run with pytest: pytest quick_hypothesis.py -v
# Or call directly:
test_sort_reverse_equivalent()
print("All tests passed!")

Output:

All tests passed!

Hypothesis ran this function hundreds of times with randomly generated lists — empty lists, lists with one element, lists with thousands of integers, lists with negative numbers, lists with duplicates. It found no counterexample, so the property holds. If the property had been wrong, Hypothesis would show you the smallest list that breaks it.

What Is Property-Based Testing?

Traditional unit tests are example-based: you write specific inputs and expected outputs. Property-based tests are contract-based: you describe invariants that must hold for ANY valid input. The library’s job is to find inputs that violate those invariants.

Property-based testing does not replace example-based tests — it complements them. Use both together. Write example-based tests for known requirements, add property-based tests for algorithmic invariants and data transformations.

Understanding Strategies

A strategy tells Hypothesis how to generate values for a particular type. The hypothesis.strategies module (conventionally imported as st) provides strategies for all Python built-in types, plus tools to compose them into complex structures.

# strategies_demo.py
from hypothesis import given, settings
from hypothesis import strategies as st

# Basic strategies
@given(st.integers(min_value=0, max_value=100))
def test_squares_are_positive(n):
    assert n * n >= 0

# Text strategies
@given(st.text(min_size=1, max_size=50))
def test_strip_never_longer(s):
    assert len(s.strip()) <= len(s)

# Float strategies
@given(st.floats(allow_nan=False, allow_infinity=False))
def test_abs_never_negative(f):
    assert abs(f) >= 0

# Lists of specific type
@given(st.lists(st.integers(), min_size=1))
def test_max_is_in_list(lst):
    assert max(lst) in lst

# Run all tests
test_squares_are_positive()
test_strip_never_longer()
test_abs_never_negative()
test_max_is_in_list()
print("All strategy tests passed!")

Output:

All strategy tests passed!

The constraints in strategies are important. st.floats(allow_nan=False, allow_infinity=False) excludes the special IEEE 754 values that would break most arithmetic. min_size=1 on a list ensures max() does not raise a ValueError on an empty list — though you might want to test that case separately.

Composing Custom Strategies

Real applications use complex data structures, not just integers. Hypothesis lets you compose strategies using st.fixed_dict(), st.builds(), and the @st.composite decorator to generate custom objects.

# custom_strategies.py
from hypothesis import given
from hypothesis import strategies as st
from dataclasses import dataclass

@dataclass
class Product:
    name: str
    price: float
    quantity: int

# Strategy for valid products
product_strategy = st.builds(
    Product,
    name=st.text(alphabet=st.characters(whitelist_categories=('Lu', 'Ll', 'Nd', 'Zs')),
                 min_size=1, max_size=50),
    price=st.floats(min_value=0.01, max_value=10000.0, allow_nan=False),
    quantity=st.integers(min_value=0, max_value=10000)
)

def calculate_total(products):
    """Calculate total value of product inventory."""
    return sum(p.price * p.quantity for p in products)

@given(st.lists(product_strategy, min_size=1, max_size=20))
def test_total_always_non_negative(products):
    """Total inventory value must never be negative."""
    total = calculate_total(products)
    assert total >= 0, f"Negative total: {total}"

@given(st.lists(product_strategy, min_size=2))
def test_adding_strategy-product_increases_total(products):
    """Adding a product with positive price and quantity increases total."""
    base_total = calculate_total(products[:-1])
    last = products[-1]
    if last.price > 0 and last.quantity > 0:
        full_total = calculate_total(products)
        assert full_total > base_total

test_total_always_non_negative()
test_adding_strategy-product_increases_total()
print("Custom strategy tests passed!")

Output:

Custom strategy tests passed!

The st.builds() strategy calls the Product constructor with generated values for each field. You can nest strategies arbitrarily — a list of products, each with a composed strategy for its fields. This mirrors how your real application data is structured, so Hypothesis generates realistic test data automatically.

Finding Real Bugs with Hypothesis

The real value of property-based testing shows when Hypothesis finds a bug you would never have written a test for. Here is an example with a buggy encoding function:

# finding_bugs.py
from hypothesis import given
from hypothesis import strategies as st

def encode(data: list) -> str:
    """Run-length encode a list of integers. E.g., [1,1,2,3,3] -> '2x1,1x2,2x3'"""
    if not data:
        return ''
    result = []
    count = 1
    for i in range(1, len(data)):
        if data[i] == data[i-1]:
            count += 1
        else:
            result.append(f"{count}x{data[i-1]}")
            count = 1
    result.append(f"{count}x{data[-1]}")
    return ','.join(result)

def decode(encoded: str) -> list:
    """Decode run-length encoded string back to list."""
    if not encoded:
        return []
    result = []
    for part in encoded.split(','):
        count_str, val_str = part.split('x')
        result.extend([int(val_str)] * int(count_str))
    return result

# Property: encode then decode should give back the original
@given(st.lists(st.integers(min_value=-100, max_value=100)))
def test_encode_decode_roundtrip(data):
    encoded = encode(data)
    decoded = decode(encoded)
    assert decoded == data, f"Roundtrip failed: {data} -> '{encoded}' -> {decoded}"

test_encode_decode_roundtrip()
print("Roundtrip test passed!")

Output:

Roundtrip test passed!

Hypothesis tested this function with hundreds of inputs including empty lists, single-element lists, all-same lists, and alternating values — and the roundtrip property held for all of them. If decode() had a bug (say, only handling positive integers), Hypothesis would immediately find a minimal failing input like [-1] and show you the exact failing case with the encoded string.

Controlling Hypothesis Settings

Hypothesis provides a settings decorator to control how many examples are generated, the maximum shrink time, and the verbosity of output. You can also use @example() to always include specific cases alongside the generated ones.

# settings_demo.py
from hypothesis import given, settings, example
from hypothesis import strategies as st

def divide(a: int, b: int) -> float:
    """Divide a by b, raise ValueError if b is zero."""
    if b == 0:
        raise ValueError("Cannot divide by zero")
    return a / b

# Always test these specific cases, plus 200 random ones
@settings(max_examples=200)
@example(a=0, b=1)
@example(a=-10, b=2)
@given(
    a=st.integers(),
    b=st.integers().filter(lambda x: x != 0)
)
def test_divide_properties(a, b):
    result = divide(a, b)
    # Property 1: result times b should equal a (within float precision)
    assert abs(result * b - a) < 1e-9
    # Property 2: dividing by positive number preserves sign
    if b > 0:
        assert (result >= 0) == (a >= 0)

test_divide_properties()
print("Divide properties verified over 200+ examples!")

Output:

Divide properties verified over 200+ examples!

The .filter(lambda x: x != 0) call on the strategy excludes zero from the generated values. The @example() decorator guarantees that specific cases always run, even if Hypothesis would not randomly generate them. Combined with max_examples=200, this gives you a thorough test with precise control.

Real-Life Example: Testing a Sorted Data Structure

We will test a custom SortedList class that maintains elements in sorted order. Hypothesis will verify several invariants hold under all valid operations.

# sorted_list_test.py
from hypothesis import given, assume
from hypothesis import strategies as st
import bisect

class SortedList:
    """A list that maintains sorted order on insert."""
    def __init__(self):
        self._data = []

    def insert(self, value):
        bisect.insort(self._data, value)

    def remove(self, value):
        idx = bisect.bisect_left(self._data, value)
        if idx < len(self._data) and self._data[idx] == value:
            self._data.pop(idx)
        else:
            raise ValueError(f"{value} not in list")

    def contains(self, value) -> bool:
        idx = bisect.bisect_left(self._data, value)
        return idx < len(self._data) and self._data[idx] == value

    def to_list(self) -> list:
        return list(self._data)

    def __len__(self):
        return len(self._data)

# Property 1: after inserting a value, the list is always sorted
@given(st.lists(st.integers()), st.integers())
def test_insert_preserves_order(existing, new_value):
    sl = SortedList()
    for v in existing:
        sl.insert(v)
    sl.insert(new_value)
    lst = sl.to_list()
    assert lst == sorted(lst), f"Not sorted after insert: {lst}"

# Property 2: after inserting, the value is always present
@given(st.integers())
def test_insert_then_contains(value):
    sl = SortedList()
    sl.insert(value)
    assert sl.contains(value)

# Property 3: length matches number of inserts
@given(st.lists(st.integers()))
def test_length_matches_inserts(values):
    sl = SortedList()
    for v in values:
        sl.insert(v)
    assert len(sl) == len(values)

test_insert_preserves_order()
test_insert_then_contains()
test_length_matches_inserts()
print("SortedList: all 3 properties verified!")

Output:

SortedList: all 3 properties verified!

These three properties — sorted order preserved, inserted value found, length consistent — form a behavioral contract for SortedList. Any implementation that passes all three is correct by definition. You can refactor the internals (swap bisect.insort for a balanced BST, for example) and use these property tests as your regression suite — they will catch any violation of the contract.

Frequently Asked Questions

Hypothesis makes my test suite slow. How do I speed it up?

Hypothesis caches its database of found examples between runs, so later runs are faster. Use @settings(max_examples=50) for fast CI runs and max_examples=1000 for deeper local testing. The suppress_health_check option disables specific health checks if they are triggering false positives. In CI, set the environment variable HYPOTHESIS_DATABASE_DIRECTORY to a cached location to preserve learned examples across runs.

What is shrinking and why does it matter?

When Hypothesis finds a failing input, it automatically tries to shrink it — finding a smaller or simpler input that still triggers the same failure. Instead of showing you a list of 1,000 random integers that caused a bug, it will show you the 2-element list [0, -1] that triggers the same bug. This makes debugging dramatically easier, because you can immediately understand what property of the input caused the failure.

What is stateful testing?

Stateful testing (also called model-based testing) lets you test sequences of operations, not just single function calls. Use hypothesis.stateful.RuleBasedStateMachine to define rules (operations like insert, delete, query) and invariants. Hypothesis generates sequences of these operations and checks invariants after each step. This is powerful for testing state machines, databases, queues, and any system where order of operations matters.

Should I replace my existing tests with Hypothesis?

No — use both. Example-based tests document specific known behaviors and are fast to run. Property-based tests explore unknown edge cases and validate invariants. A typical approach: write a few example-based tests to nail down the specification, then add property-based tests to verify invariants hold broadly. Your test suite becomes both precise (example-based) and thorough (property-based).

How does Hypothesis remember failing examples?

Hypothesis stores its discovered failing examples in a database directory (by default, .hypothesis/ in your project root). When you run the tests again, it tries the previously failing examples first. This means once a bug is found, it is retested every run — even if the main random generation would not have generated that input again. Add .hypothesis/ to .gitignore for local databases, or commit it to retain team-shared learned examples.

Conclusion

Property-based testing with Hypothesis changes how you think about test coverage. Instead of asking “did I test this specific case?” you ask “what properties must always hold?” We covered the basics: the @given decorator and strategies for common types, composing custom strategies with st.builds(), writing meaningful properties (roundtrip, ordering, length consistency), and controlling test settings. The real payoff comes when Hypothesis finds a minimal failing input you never would have written yourself.

From here, explore Hypothesis’s stateful testing with RuleBasedStateMachine for testing complex state machines, and the st.data() strategy for dynamic input generation within tests. The Hypothesis documentation includes a gallery of real-world bugs found by property testing that is worth reading for inspiration.

Official documentation: hypothesis.readthedocs.io

Related Articles

• How To Write Unit Tests with pytest in Python
• How To Mock API Calls in Python Tests
• Python Exception Handling Best Practices