Intermediate
Standard Python logging gives you timestamped text messages, but when you are debugging a production incident at 2am, those messages rarely give you the context you need. Which user triggered the error? Which request ID? What was the state of the application at that moment? Structured logging solves this by making every log event a machine-readable data record — a dict of key-value pairs instead of a raw string. structlog is the leading Python library for structured logging, and it makes this easy while staying fully compatible with the standard logging module.
structlog works by building up a context dictionary as your code executes, then rendering it as JSON (or pretty-printed for development) when you emit a log event. You bind values once — like a request ID or user ID — and they appear automatically in every subsequent log message from that context. No more copy-pasting request IDs into every log call.
In this article, you will learn how to configure structlog, bind context with bind and new, set up processors for development and production output, integrate with the standard logging module, add request context in a Flask app, and build a practical async worker with rich structured logging. Install structlog with pip install structlog before starting.
Structured Logging: Quick Example
# structlog_quick.py
import structlog
log = structlog.get_logger()
log.info("user_login", user_id=42, email="alice@example.com", source="web")
log.warning("login_failed", user_id=99, reason="invalid_password", attempt=3)
Output (development mode — pretty printed):
2026-05-02 10:00:00 [info ] user_login email=alice@example.com source=web user_id=42
2026-05-02 10:00:00 [warning ] login_failed attempt=3 reason=invalid_password user_id=99
Each log call takes a positional event string followed by keyword arguments that become fields in the structured record. In development, structlog renders these in a readable format; in production, you configure it to output JSON. The same code, different renderers — you never change your log calls based on environment.
What Is structlog and Why Use It?
structlog replaces the pattern of building log messages from string concatenation (f"User {user_id} failed to login with error {error}") with a pattern of logging discrete fields. This makes logs searchable and filterable in tools like Elasticsearch, Datadog, or CloudWatch Logs Insights — you can query user_id=42 and get all events for that user, rather than running regex searches over raw strings.
| Feature | stdlib logging | structlog |
|---|---|---|
| Output format | Text strings | JSON or text, configurable |
| Context binding | LoggerAdapter (verbose) | bind() / new() (clean) |
| Processors/middleware | Handlers and Filters | Processor pipeline |
| stdlib compatibility | Native | Full integration available |
| Async support | Thread-safe only | async-aware with asyncio |
structlog does not replace stdlib logging — it wraps around it. In most production setups, structlog acts as the frontend you write to, while stdlib logging handles the backend (file handlers, syslog, etc.). This means you can adopt structlog incrementally in an existing codebase.
Configuring structlog for Development and Production
structlog is configured once at application startup. The key is the processor chain — a list of functions that transform the log event dict before rendering. Here is a complete configuration for both environments:
# structlog_config.py
import structlog
import logging
import sys
import os
def configure_logging() -> None:
"""Configure structlog for the current environment."""
is_production = os.getenv('ENV', 'development') == 'production'
shared_processors = [
structlog.contextvars.merge_contextvars, # Thread/async context
structlog.stdlib.add_log_level, # Add 'level' field
structlog.stdlib.add_logger_name, # Add 'logger' field
structlog.processors.TimeStamper(fmt='iso'), # ISO 8601 timestamp
structlog.processors.StackInfoRenderer(), # Stack traces
structlog.processors.format_exc_info, # Exception formatting
]
if is_production:
# JSON output for log aggregation tools
renderer = structlog.processors.JSONRenderer()
else:
# Human-readable colored output for terminals
renderer = structlog.dev.ConsoleRenderer(colors=True)
structlog.configure(
processors=shared_processors + [renderer],
wrapper_class=structlog.make_filtering_bound_logger(logging.DEBUG),
context_class=dict,
logger_factory=structlog.PrintLoggerFactory(),
cache_logger_on_first_use=True,
)
configure_logging()
log = structlog.get_logger("myapp")
log.info("app_started", version="1.2.3", environment=os.getenv('ENV', 'development'))
log.debug("config_loaded", database="postgres", cache="redis")
Output (development):
2026-05-02T10:00:00Z [debug ] config_loaded cache=redis database=postgres logger=myapp
2026-05-02T10:00:00Z [info ] app_started environment=development logger=myapp version=1.2.3
Output (production, ENV=production):
{"cache": "redis", "database": "postgres", "event": "config_loaded", "level": "debug", "logger": "myapp", "timestamp": "2026-05-02T10:00:00Z"}
{"environment": "production", "event": "app_started", "level": "info", "logger": "myapp", "timestamp": "2026-05-02T10:00:00Z", "version": "1.2.3"}
Binding Context with bind() and new()
The most powerful feature of structlog is context binding. Use bind() to add fields that persist for the lifetime of a logger, and new() to create a fresh logger with a clean context:
# structlog_bind.py
import structlog
log = structlog.get_logger()
def process_order(order_id: str, user_id: int) -> None:
# Bind context once -- all subsequent logs from this logger include it
order_log = log.bind(order_id=order_id, user_id=user_id)
order_log.info("order_processing_started")
# Further bind additional context
item_log = order_log.bind(item_count=3, total_amount=99.99)
item_log.info("items_validated")
try:
# Simulate an operation
if order_id == "ORD-999":
raise ValueError("Invalid payment method")
item_log.info("payment_processed", gateway="stripe")
except ValueError as e:
item_log.error("payment_failed", error=str(e), retry=False)
raise
order_log.info("order_processing_complete")
process_order("ORD-101", user_id=42)
Output:
2026-05-02 [info ] order_processing_started order_id=ORD-101 user_id=42
2026-05-02 [info ] items_validated item_count=3 order_id=ORD-101 total_amount=99.99 user_id=42
2026-05-02 [info ] payment_processed gateway=stripe item_count=3 order_id=ORD-101 total_amount=99.99 user_id=42
2026-05-02 [info ] order_processing_complete order_id=ORD-101 user_id=42
Notice how order_id and user_id appear in every log message without being passed repeatedly. When the error occurs, the full context is already present — you see exactly which order failed, for which user, with which item count. This is what makes structured logging so powerful for debugging production issues.
Integrating with stdlib logging
Most production codebases already use stdlib logging — your application, your third-party libraries, the frameworks they depend on. structlog plays nicely with all of them: it can wrap stdlib loggers (so existing code keeps working) AND redirect stdlib output through structlog’s pipeline (so everything ends up as JSON):
# File: hybrid_logging.py
import logging
import structlog
# 1) Configure stdlib logging to forward to structlog's processor chain
logging.basicConfig(
format="%(message)s",
level=logging.INFO,
)
# 2) Configure structlog to render as JSON
structlog.configure(
processors=[
structlog.stdlib.filter_by_level,
structlog.stdlib.add_logger_name,
structlog.stdlib.add_log_level,
structlog.processors.TimeStamper(fmt="iso"),
structlog.processors.StackInfoRenderer(),
structlog.processors.format_exc_info,
structlog.processors.JSONRenderer(),
],
logger_factory=structlog.stdlib.LoggerFactory(),
wrapper_class=structlog.stdlib.BoundLogger,
cache_logger_on_first_use=True,
)
# 3) Use either API — same JSON output
struct_logger = structlog.get_logger("api")
struct_logger.info("via structlog", user_id=42)
stdlib_logger = logging.getLogger("legacy")
stdlib_logger.info("via stdlib")The processor chain is the heart of structlog. Each processor receives the event dict and returns the modified dict. JSONRenderer at the end serializes it. By inserting format_exc_info and StackInfoRenderer in the chain, every logger.exception() call gets a clean JSON traceback without you having to do anything.
Custom Processors for Filtering and Enrichment
A processor is just a function with signature (logger, method_name, event_dict) -> event_dict. You write them like middleware — each one inspects the dict, can mutate it, and returns either the dict or raises DropEvent to drop the line entirely. Two patterns you’ll use constantly:
# File: custom_processors.py
import os
import structlog
def add_environment(logger, method_name, event_dict):
"""Inject deploy environment into every log line."""
event_dict["env"] = os.environ.get("APP_ENV", "dev")
event_dict["service"] = "payments-api"
return event_dict
def redact_secrets(logger, method_name, event_dict):
"""Strip values for any key that looks sensitive."""
for key in list(event_dict):
if any(s in key.lower() for s in ("password", "token", "secret", "card")):
event_dict[key] = "***REDACTED***"
return event_dict
structlog.configure(
processors=[
add_environment,
redact_secrets,
structlog.processors.TimeStamper(fmt="iso"),
structlog.processors.JSONRenderer(),
],
)
log = structlog.get_logger()
log.info("charge attempted", user_id=42, card_number="4111111111111111", amount=99)Output:
{"env": "dev", "service": "payments-api", "user_id": 42, "card_number": "***REDACTED***", "amount": 99, "timestamp": "2026-05-16T12:34:56", "event": "charge attempted"}The redact_secrets processor is the most useful one most projects don’t have. Run every log through it and you can’t accidentally ship raw tokens or PII to your log aggregator. The add_environment processor solves the “which deploy did this come from” question that always shows up during incident response.
Async, FastAPI, and Per-Request Context
In an async web app, the killer feature of structlog is contextvars-based logging — bind context once per request and every log line inside that request inherits it automatically, across await boundaries:
# File: fastapi_structlog.py
from uuid import uuid4
import structlog
from fastapi import FastAPI, Request
structlog.configure(
processors=[
structlog.contextvars.merge_contextvars,
structlog.processors.TimeStamper(fmt="iso"),
structlog.processors.JSONRenderer(),
],
)
log = structlog.get_logger()
app = FastAPI()
@app.middleware("http")
async def log_context(request: Request, call_next):
structlog.contextvars.clear_contextvars()
structlog.contextvars.bind_contextvars(
request_id=str(uuid4()),
path=request.url.path,
method=request.method,
)
log.info("request_started")
response = await call_next(request)
log.info("request_completed", status=response.status_code)
return response
@app.get("/items/{item_id}")
async def get_item(item_id: int):
log.info("looking up item", item_id=item_id) # request_id auto-included
return {"item_id": item_id}Every log line emitted from anywhere inside the request — your handler, a service called by the handler, a coroutine awaited deep inside — automatically carries request_id, path, and method. When something breaks at 3 AM, you grep one request_id across the log aggregator and see the full timeline of a single request.
Production Patterns and Performance
- Configure once at startup.
structlog.configure()should be called from your main entry point before anything else logs. Re-configuring at runtime is supported but confusing. - Cache the logger. Set
cache_logger_on_first_use=Trueinconfigure(). Without it, structlog rebuilds the bound logger on every call, which is slow under load. - Use lazy formatting. structlog already does this for free — you pass kwargs, not f-strings.
log.info("query ran", duration=t)is faster and safer thanlog.info(f"query ran {t}s"). - Avoid logging in tight loops. JSON serialization isn’t free. If you’re logging inside a 1M-iteration loop, log a summary at the end instead.
- Pin your processors in tests. Use
structlog.testing.LogCaptureas a test fixture to assert that specific events were logged with specific fields. Don’t grep stdout in tests.
FAQ
Q: structlog or python-json-logger — which one?
A: python-json-logger is a tiny adapter that just makes stdlib logging output JSON. structlog is a full structured-logging system with bound context, processor chains, and contextvars integration. For “I just want JSON”, use python-json-logger. For anything beyond that, structlog.
Q: Does structlog replace the stdlib logging module?
A: It can, but it doesn’t have to. The recommended setup is hybrid: structlog handles your code’s logs, stdlib logging keeps working for third-party libraries, and both end up in the same JSON output via the processor chain.
Q: How do I get structlog working with Django?
A: Configure structlog in settings.py alongside Django’s LOGGING dict. Use structlog.stdlib.LoggerFactory() as the factory so Django’s existing log levels and handlers still apply. Add structlog.contextvars.merge_contextvars as the first processor and bind request_id in middleware.
Q: Can I use structlog in scripts/CLIs, not just web apps?
A: Yes — structlog has no web-framework dependencies. The contextvars features are also useful in any async or threaded code, not just HTTP request handlers.
Q: What’s the performance cost vs stdlib logging?
A: With cache_logger_on_first_use=True and a JSON renderer, structlog is roughly 2-3x slower than bare logger.info("msg"). In practice, log emission is rarely a hotpath — you’ll feel it only if you’re logging inside a per-millisecond loop, which you shouldn’t be doing anyway.
Wrapping Up
structlog turns Python logging from “strings that you grep later” into “events that you query.” The two features that matter most are bind() (carry context forward) and the processor pipeline (filter, enrich, and redact in one place). Combined with contextvars integration in web apps, you get request-scoped log context for free — no manual passing of correlation IDs through every function.
The official structlog documentation has the full reference, including integrations with sentry, OpenTelemetry, and various web frameworks beyond FastAPI.