Intermediate

When your Python application receives JSON from an API, reads a config file, or loads records from a database, you end up with plain dictionaries. You then write code to manually extract keys, validate types, and convert values into your domain objects. This conversion layer is tedious to write, easy to get wrong, and even easier to forget to update when your data model changes.

cattrs automates this conversion. It converts plain Python structures — dictionaries, lists, and primitives — into typed attrs classes or dataclasses, and back again. It handles nested structures, optional fields, unions, and custom type coercion. When the input doesn’t match the expected type, it raises clear, structured errors that tell you exactly which field failed and why.

This article covers how to install cattrs, how to structure and unstructure objects, how to handle nested and optional types, how to add custom hooks for special conversion logic, and how to use the validation capabilities. By the end you will be able to replace fragile manual conversion code with a clean, reliable structuring layer.

Structuring Data with cattrs: Quick Example

Here is a minimal example converting a dictionary from a JSON API response into a typed Python object:

# quick_cattrs_example.py
import attrs
import cattrs

@attrs.define
class User:
    name: str
    age: int
    email: str

# Simulated API response (as a dict)
raw_data = {"name": "Alice", "age": 30, "email": "alice@example.com"}

# Structure: dict -> typed object
user = cattrs.structure(raw_data, User)
print(user)
print(type(user))

# Unstructure: typed object -> dict (for serialization)
back_to_dict = cattrs.unstructure(user)
print(back_to_dict)

Output:

User(name='Alice', age=30, email='alice@example.com')
<class '__main__.User'>
{'name': 'Alice', 'age': 30, 'email': 'alice@example.com'}

cattrs.structure(data, MyClass) converts raw data into your class. cattrs.unstructure(obj) converts it back to a plain dict. The sections below cover nested types, optional fields, custom converters, and error handling.

What Is cattrs and Why Use It?

cattrs is a library for structuring and unstructuring data. “Structuring” means converting untyped data (dicts, lists) into typed Python objects. “Unstructuring” means the reverse — serializing objects back into plain data suitable for JSON or storage. It is designed to work with attrs classes but also supports standard dataclasses.

Install cattrs along with attrs:

# pip install cattrs attrs

import cattrs
print(cattrs.__version__)

23.2.3

Handling Nested Structures

Real data is rarely flat. APIs return nested objects — a user with an address, an order with line items. cattrs handles nesting automatically by following type hints recursively.

# cattrs_nested.py
import attrs
import cattrs
from typing import List

@attrs.define
class Address:
    street: str
    city: str
    country: str

@attrs.define
class User:
    name: str
    age: int
    address: Address
    tags: List[str]

# Deeply nested raw data (e.g., from JSON API)
raw = {
    "name": "Bob",
    "age": 25,
    "address": {
        "street": "123 Main St",
        "city": "Melbourne",
        "country": "Australia"
    },
    "tags": ["admin", "verified"]
}

user = cattrs.structure(raw, User)
print(user.name)
print(user.address.city)
print(user.tags)

# Unstructure back to nested dict
print(cattrs.unstructure(user))

Output:

Bob
Melbourne
['admin', 'verified']
{'name': 'Bob', 'age': 25, 'address': {'street': '123 Main St', 'city': 'Melbourne', 'country': 'Australia'}, 'tags': ['admin', 'verified']}

The Address field is structured automatically because its type annotation is Address, which cattrs recognizes as an attrs class. The List[str] annotation tells cattrs to structure the list and coerce each element to a string. Deeply nested structures work the same way — cattrs follows the type tree recursively without any extra configuration.

Optional Fields and Union Types

Real-world data has missing fields. A user’s phone number might be null, a legacy record might omit a new field, or an API might return different types depending on context. cattrs handles Optional and Union types cleanly.

# cattrs_optional.py
import attrs
import cattrs
from typing import Optional, List, Union

@attrs.define
class Product:
    id: int
    name: str
    price: float
    description: Optional[str] = None
    tags: Optional[List[str]] = None

# Missing optional fields default to None
raw_minimal = {"id": 1, "name": "Widget", "price": 9.99}
product = cattrs.structure(raw_minimal, Product)
print(product)

# Full data with optional fields populated
raw_full = {
    "id": 2,
    "name": "Gadget",
    "price": 49.99,
    "description": "A useful gadget",
    "tags": ["electronics", "sale"]
}
product2 = cattrs.structure(raw_full, Product)
print(product2.description)
print(product2.tags)

Output:

Product(id=1, name='Widget', price=9.99, description=None, tags=None)
A useful gadget
['electronics', 'sale']

Optional[str] is equivalent to Union[str, None]. When the raw data contains None or the key is absent, cattrs uses the default value from the class definition. If you need to handle more complex unions — like a field that can be either a string or an integer depending on the context — you can register a custom hook (covered in the next section).

Custom Structuring Hooks

Sometimes the raw data does not match your type annotations exactly. A date might arrive as a string "2026-01-15" instead of a datetime object. A boolean might come as "true" or 1 instead of True. Custom hooks let you intercept the structuring process for specific types and apply your own conversion logic.

# cattrs_custom_hooks.py
import attrs
import cattrs
from datetime import datetime

@attrs.define
class Event:
    title: str
    start_date: datetime
    attendees: int

# Create a custom converter (don't modify the global default)
converter = cattrs.Converter()

# Register a hook: convert string to datetime
def structure_datetime(value, _type):
    if isinstance(value, datetime):
        return value
    if isinstance(value, str):
        return datetime.fromisoformat(value)
    raise ValueError(f"Cannot convert {value!r} to datetime")

converter.register_structure_hook(datetime, structure_datetime)

# Now structure data where date is a string
raw = {
    "title": "PyCon AU",
    "start_date": "2026-08-14T09:00:00",
    "attendees": 500
}

event = converter.structure(raw, Event)
print(event.title)
print(event.start_date)
print(type(event.start_date))

Output:

PyCon AU
2026-08-14 09:00:00
<class 'datetime.datetime'>

Notice the use of cattrs.Converter() to create an isolated converter instance rather than using the global default. This keeps your custom hooks scoped to specific parts of your application without affecting other code that uses cattrs. The hook signature is always hook(value, type) — value is the raw input and type is the target Python type.

Validation and Error Handling

When structuring fails because the data does not match the expected types, cattrs raises a ClassValidationError. Unlike a bare TypeError, this exception contains structured information about exactly which fields failed and why — even for nested structures.

# cattrs_validation.py
import attrs
import cattrs

@attrs.define
class Config:
    host: str
    port: int
    debug: bool

# Bad data -- port is a string that cann't be cast to int
bad_data = {"host": "localhost", "port": "not-a-number", "debug": True}

try:
    config = cattrs.structure(bad_data, Config)
except cattrs.ClassValidationError as e:
    print("Validation failed!")
    for error in e.exceptions:
        print(f"  Field error: {error}")

Output:

Validation failed!
  Field error: invalid literal for int() with base 10: 'not-a-number' (AttributeError) @ $.port

The @ $.port path notation shows exactly which field caused the problem using JSONPath-style notation. For deeply nested structures, you might see @ $.address.zip_code. This structured error reporting makes debugging data issues much faster than hunting through a generic traceback. In production, you can catch ClassValidationError, log the structured errors, and return a meaningful validation response to the API caller.

Real-Life Example: Parsing a Paginated API Response

Here is a complete example structuring a paginated API response with nested objects, optional fields, and a custom datetime hook:

# cattrs_api_parser.py
import attrs
import cattrs
from datetime import datetime
from typing import List, Optional

# Domain models
@attrs.define
class Author:
    id: int
    username: str
    email: Optional[str] = None

@attrs.define
class Post:
    id: int
    title: str
    body: str
    author: Author
    created_at: datetime
    tags: List[str]
    published: bool

@attrs.define
class PaginatedPosts:
    total: int
    page: int
    per_page: int
    posts: List[Post]

# Set up converter with datetime hook
converter = cattrs.Converter()

def parse_datetime(value, _):
    if isinstance(value, datetime):
        return value
    return datetime.fromisoformat(value.replace("Z", "+00:00"))

converter.register_structure_hook(datetime, parse_datetime)

# Simulated response from jsonplaceholder.typicode.com (enriched)
api_response = {
    "total": 100,
    "page": 1,
    "per_page": 2,
    "posts": [
        {
            "id": 1,
            "title": "Getting Started with Python",
            "body": "Python is a versatile language...",
            "author": {"id": 10, "username": "alice", "email": "alice@example.com"},
            "created_at": "2026-01-15T10:30:00",
            "tags": ["python", "beginner"],
            "published": True
        },
        {
            "id": 2,
            "title": "Advanced asyncio Patterns",
            "body": "Structured concurrency changes everything...",
            "author": {"id": 11, "username": "bob"},
            "created_at": "2026-02-20T14:00:00",
            "tags": ["python", "async", "advanced"],
            "published": True
        }
    ]
}

# One line to structure the entire response
result = converter.structure(api_response, PaginatedPosts)

print(f"Page {result.page} of {result.total // result.per_page}")
for post in result.posts:
    print(f"\n[{post.id}] {post.title}")
    print(f"  Author: {post.author.username} ({post.author.email or 'no email'})")
    print(f"  Posted: {post.created_at.strftime('%b %d, %Y')}")
    print(f"  Tags: {', '.join(post.tags)}")

Output:

Page 1 of 50

[1] Getting Started with Python
  Author: alice (alice@example.com)
  Posted: Jan 15, 2026
  Tags: python, beginner

[2] Advanced asyncio Patterns
  Author: bob (no email)
  Posted: Feb 20, 2026
  Tags: python, async, advanced

The entire multi-level API response — paginated wrapper, list of posts, nested author objects, datetime strings, optional email — structures in a single converter.structure() call. Adding a new field to the model automatically includes it in both structuring and unstructuring without changing any conversion code.

Frequently Asked Questions

How does cattrs compare to Pydantic?

Pydantic models are self-contained — validation logic lives in the class itself. cattrs is a separate converter layer that works with attrs classes or dataclasses. Pydantic V2 is extremely fast and has first-class FastAPI integration. cattrs is better suited for code that already uses attrs, needs highly customizable conversion behavior, or wants to separate the data model from the conversion logic. For new API-heavy projects, Pydantic is often the default choice; for attrs-based codebases, cattrs fits naturally.

Does cattrs work with standard dataclasses?

Yes. cattrs supports Python’s built-in dataclasses in addition to attrs classes. Use @dataclass from the standard library and cattrs will structure and unstructure them using the same API. The main difference is that attrs classes have more features (validators, converters, slots) that cattrs can leverage. For simple cases, dataclasses work fine.

Is cattrs fast enough for high-volume use?

cattrs generates optimized structuring code at registration time rather than inspecting types on every call. For high-volume scenarios, use the cattrs.gen.make_dict_structure_fn() pattern to pre-generate and cache structuring functions. In benchmarks, cattrs is significantly faster than manual dict parsing loops and competitive with Pydantic V1. For extreme performance requirements, consider Pydantic V2 with its Rust-based core.

How do I customize unstructuring (object to dict)?

Use converter.register_unstructure_hook(MyType, lambda obj: ...) to register a custom unstructuring function. For example, to serialize datetime as ISO format: converter.register_unstructure_hook(datetime, lambda dt: dt.isoformat()). You can also use cattrs.gen.make_dict_unstructure_fn() to generate an unstructure function with field overrides — useful for renaming keys or excluding fields from serialization.

What happens if a required field is missing?

If a required field (no default value) is absent from the raw data, cattrs raises a ClassValidationError with the path of the missing field. If you want to allow missing fields and use a fallback, either add a default value in your class definition (field: str = "default") or use Optional[str] = None. You can also register a structure hook that fills in defaults before structuring if the data source is known to be incomplete.

Conclusion

The cattrs library eliminates the repetitive, error-prone conversion code between raw data and typed Python objects. You learned how to structure and unstructure objects, handle nested types and optional fields, register custom hooks for special type conversions, and interpret structured validation errors. The paginated API parser showed how all these features combine into a clean, maintainable data layer.

The logical next step is to apply cattrs at the boundary of your application — wherever external data enters your system (API responses, config files, database results) — and convert it to typed objects immediately. From that point, the rest of your application works with typed, validated data and never touches raw dicts again. The catt.rs documentation covers advanced patterns including generating optimized converters, handling forward references, and customizing the global converter.

Related Articles

• How To Use Python attrs Library for Data Classes
• How To Read and Write JSON Files in Python
• How To Use Python msgspec for Fast JSON Serialization

Intermediate

You have a Python service that parses JSON responses from an API thousands of times per second, and the standard json module is quietly becoming a bottleneck. At low traffic volumes this goes unnoticed, but once you scale up, milliseconds of serialization overhead compound into real latency. If you have ever profiled a Python web service and found json.dumps or json.loads sitting near the top of the flame graph, you already know this pain.

orjson is a fast, correct JSON library for Python written in Rust. It drops into nearly any codebase as a replacement for the standard json module and typically runs 2-10x faster on both serialization and deserialization. It also natively supports types the standard library forces you to handle manually — datetime, UUID, numpy arrays, and dataclasses.

In this article you will learn how to install orjson, serialize and deserialize JSON with it, use its built-in support for Python-native types, benchmark it against the standard library, and integrate it into a real-world FastAPI project. By the end you will have a working understanding of when and why to choose orjson over the alternatives.

orjson Quick Example

Before diving deep, here is a self-contained example that shows the core pattern. orjson is nearly a drop-in replacement for the standard json module, but returns and accepts bytes instead of str.

# quick_example.py
import orjson
from datetime import datetime

data = {
    "name": "Alice",
    "score": 98.6,
    "logged_in": True,
    "joined": datetime(2024, 3, 15, 9, 30, 0),
    "tags": ["python", "backend","fast"]
}

# Serialize to bytes (not str like the standard json module)
encoded = orjson.dumps(data)
print(encoded)
print(type(encoded))

# Deserialize back to a Python dict
decoded = orjson.loads(encoded)
print(decoded["joined"])  # datetime is serialized as ISO 8601 string
print(type(decoded))

Output:

b'{"name":"Alice","score":98.6,"logged_in":true,"joined":"2024-03-15T09:30:00","tags":["python","backend","fast"]}'
<class 'bytes'>
2024-03-15T09:30:00
<class 'dict'>

Two things stand out right away. First, orjson.dumps() returns bytes, not a string — this is intentional and saves an unnecessary encoding step when writing to network sockets or files. Second, the datetime object is automatically serialized to ISO 8601 format without any extra work, which the standard json module would refuse to handle at all.

What Is orjson and Why Use It?

orjson is a Python JSON library implemented in Rust using the Serde framework. It was created specifically to address the performance limitations of Python’s built-in json module, which is implemented in C but still shows its age when processing large payloads at high throughput.

The key differences between orjson and the standard library are:

The Rust implementation takes advantage of SIMD instructions and a highly optimized Serde-based serialization pipeline. For applications doing heavy JSON processing — API gateways, caching layers, log aggregators — the improvement is measurable and often significant.

Installing orjson

orjson is available on PyPI and installs with a single command:

# install_orjson.sh
pip install orjson

Output:

Collecting orjson
  Downloading orjson-3.10.x-cp312-cp312-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (144 kB)
Successfully installed orjson-3.10.x

orjson ships as a pre-compiled binary for most platforms (Linux, macOS, Windows on x86-64 and ARM), so there is no Rust toolchain required. If you are on a less common platform you may need Rust installed to build from source. Verify the installation with a quick import check:

# verify_install.py
import orjson
print(orjson.__version__)

Output:

3.10.x

Serializing Python Objects with orjson.dumps()

The orjson.dumps() function converts Python objects to JSON bytes. The most important thing to remember is that it always returns bytes, not str. If you need a string, call .decode() on the result.

# serialization_basics.py
import orjson
from datetime import datetime, date
from uuid import UUID
from dataclasses import dataclass

@dataclass
class User:
    id: UUID
    name: str
    created: datetime
    active: bool

user = User(
    id=UUID("12345678-1234-5678-1234-567812345678"),
    name="Bob Smith",
    created=datetime(2025, 1, 10, 14, 30),
    active=True
)

# Serialize the dataclass directly -- no custom encoder needed
result = orjson.dumps(user)
print(result)

# Decode to string if needed
print(result.decode("utf-8"))

Output:

b'{"id":"12345678-1234-5678-1234-567812345678","name":"Bob Smith","created":"2025-01-10T14:30:00","active":true}'
{"id":"12345678-1234-5678-1234-567812345678","name":"Bob Smith","created":"2025-01-10T14:30:00","active":true}

Notice that the UUID, datetime, and dataclass are all handled automatically with zero configuration. With the standard json module, each of these would raise a TypeError: Object of type X is not JSON serializable error, requiring a custom default function.

orjson Options and Flags

orjson supports serialization options passed via the option parameter as bitwise-OR combinations of constants. These let you control formatting, sorting, and type handling:

# orjson_options.py
import orjson

data = {
    "z_key": "last",
    "a_key": "first",
    "count": 42,
    "ratio": 3.14159
}

# Pretty-print with indented output
pretty = orjson.dumps(data, option=orjson.OPT_INDENT_2)
print("Pretty:")
print(pretty.decode())

# Sort keys alphabetically
sorted_output = orjson.dumps(data, option=orjson.OPT_SORT_KEYS)
print("\nSorted keys:")
print(sorted_output.decode())

# Combine options with bitwise OR
both = orjson.dumps(data, option=orjson.OPT_INDENT_2 | orjson.OPT_SORT_KEYS)
print("\nPretty + Sorted:")
print(both.decode())

Output:

Pretty:
{
  "z_key": "last",
  "a_key": "first",
  "count": 42,
  "ratio": 3.14159
}

Sorted keys:
{"a_key":"first","count":42,"ratio":3.14159,"z_key":"last"}

Pretty + Sorted:
{
  "a_key": "first",
  "count": 42,
  "ratio": 3.14159,
  "z_key": "last"
}

The most useful options in practice are OPT_INDENT_2 for human-readable output during debugging, OPT_SORT_KEYS for deterministic output in tests or caches, OPT_NON_STR_KEYS for dicts with integer or float keys, and OPT_UTC_Z to use Z suffix instead of +00:00 for UTC datetimes.

Deserializing with orjson.loads()

The orjson.loads() function accepts both bytes and str input and returns Python objects. Unlike the standard library, it performs strict UTF-8 validation on input, which means malformed data fails loudly rather than silently corrupting your data.

# deserialization.py
import orjson

# From bytes (most common in API and network scenarios)
json_bytes = b'{"name": "Charlie", "score": 99.5, "tags": ["fast", "correct"]}'
data = orjson.loads(json_bytes)
print(data)
print(type(data["score"]))

# From string also works
json_str = '{"status": "ok", "count": 1000}'
data2 = orjson.loads(json_str)
print(data2)

# Error handling -- orjson raises JSONDecodeError for invalid input
try:
    orjson.loads(b'{"broken": }')
except orjson.JSONDecodeError as e:
    print(f"Parse error: {e}")

Output:

{'name': 'Charlie', 'score': 99.5, 'tags': ['fast', 'correct']}
<class 'float'>
{'status': 'ok', 'count': 1000}
Parse error: expected value at line 1 column 12

One important detail: orjson.JSONDecodeError is a subclass of json.JSONDecodeError, so any existing except blocks using json.JSONDecodeError will still catch orjson errors without modification. This makes the migration path from the standard library seamless.

Benchmarking orjson vs Standard json

Let us run a concrete benchmark so you can see the actual performance difference on your hardware. We test serializing and deserializing a moderately complex nested dictionary 100,000 times:

# benchmark_orjson.py
import json
import orjson
import time
from datetime import datetime

# Test data -- similar to a typical API response
sample_data = {
    "users": [
        {"id": i, "name": f"User{i}", "email": f"user{i}@example.com",
         "score": i * 1.5, "active": i % 2 == 0, "tags": ["python", "backend"]}
        for i in range(50)
    ],
    "total": 50,
    "page": 1
}

ITERATIONS = 100_000

# Benchmark json.dumps
start = time.perf_counter()
for _ in range(ITERATIONS):
    json.dumps(sample_data)
json_dumps_time = time.perf_counter() - start

# Benchmark orjson.dumps (returns bytes)
start = time.perf_counter()
for _ in range(ITERATIONS):
    orjson.dumps(sample_data)
orjson_dumps_time = time.perf_counter() - start

# Benchmark json.loads
json_str = json.dumps(sample_data)
start = time.perf_counter()
for _ in range(ITERATIONS):
    json.loads(json_str)
json_loads_time = time.perf_counter() - start

# Benchmark orjson.loads
orjson_bytes = orjson.dumps(sample_data)
start = time.perf_counter()
for _ in range(ITERATIONS):
    orjson.loads(orjson_bytes)
orjson_loads_time = time.perf_counter() - start

print(f"json.dumps:   {json_dumps_time:.3f}s")
print(f"orjson.dumps: {orjson_dumps_time:.3f}s  ({json_dumps_time/orjson_dumps_time:.1f}x faster)")
print(f"json.loads:   {json_loads_time:.3f}s")
print(f"orjson.loads: {orjson_loads_time:.3f}s  ({json_loads_time/orjson_loads_time:.1f}x faster)")

Output (typical results on a modern CPU):

json.dumps:   2.841s
orjson.dumps: 0.482s  (5.9x faster)
json.loads:   2.103s
orjson.loads: 0.631s  (3.3x faster)

Actual speedups vary based on payload size, nesting depth, and hardware, but 3-6x faster on both operations is typical. For a service handling 1,000 requests per second with 100KB payloads each, this translates to substantial CPU savings that compound at scale.

Real-Life Example: FastAPI Response Caching with orjson

Here is a practical example that integrates orjson into a FastAPI application. We use orjson for both serializing API responses and caching them in memory, demonstrating a common production pattern:

# fastapi_orjson_cache.py
"""
FastAPI app with orjson-powered response serialization and in-memory caching.
Run with: uvicorn fastapi_orjson_cache:app --reload
"""
import orjson
from fastapi import FastAPI
from fastapi.responses import Response
from datetime import datetime, timezone
from dataclasses import dataclass, field
from typing import Optional
import hashlib

app = FastAPI()

# Simple in-memory cache using orjson bytes as values
_cache: dict[str, bytes] = {}


@dataclass
class ProductRecord:
    id: int
    name: str
    price: float
    in_stock: bool
    last_updated: datetime
    tags: list[str] = field(default_factory=list)


def get_product_from_db(product_id: int) -> Optional[ProductRecord]:
    """Simulates a database lookup."""
    if product_id > 100:
        return None
    return ProductRecord(
        id=product_id,
        name=f"Product {product_id}",
        price=round(product_id * 9.99, 2),
        in_stock=product_id % 3 != 0,
        last_updated=datetime.now(timezone.utc),
        tags=["electronics", "featured"] if product_id < 50 else ["clearance"]
    )


@app.get("/products/{product_id}")
async def get_product(product_id: int):
    cache_key = f"product:{product_id}"

    # Check cache first
    if cache_key in _cache:
        # Return cached bytes directly -- no re-serialization needed
        return Response(content=_cache[cache_key], media_type="application/json")

    product = get_product_from_db(product_id)
    if product is None:
        error = orjson.dumps({"error": "Product not found", "id": product_id})
        return Response(content=error, media_type="application/json", status_code=404)

    # Serialize with orjson -- handles dataclass and datetime natively
    encoded = orjson.dumps(product, option=orjson.OPT_INDENT_2)
    _cache[cache_key] = encoded

    return Response(content=encoded, media_type="application/json")


@app.get("/cache/stats")
async def cache_stats():
    stats = {
        "cached_keys": len(_cache),
        "cache_size_bytes": sum(len(v) for v in _cache.values()),
        "timestamp": datetime.now(timezone.utc)
    }
    return Response(content=orjson.dumps(stats), media_type="application/json")

Example curl output:

$ curl http://localhost:8000/products/42
{
  "id": 42,
  "name": "Product 42",
  "price": 419.58,
  "in_stock": true,
  "last_updated": "2025-03-15T10:22:41.123456+00:00",
  "tags": ["electronics", "featured"]
}

The power here is that the serialized bytes are stored in the cache and served directly as the HTTP response body without deserialization or re-serialization. orjson's native datetime handling means the UTC-aware datetime in last_updated is serialized to a full ISO 8601 string with timezone offset -- exactly what frontend clients expect.

Frequently Asked Questions

Why does orjson return bytes instead of str?

orjson returns bytes because JSON data in Python is almost always immediately encoded to bytes for network transport or file writing. Returning bytes directly avoids an extra .encode("utf-8") step. If you need a string, just call result.decode(). This is a deliberate performance decision -- the bytes representation is the final form that gets sent over the wire.

Is orjson a drop-in replacement for the json module?

Almost, but not completely. The function signatures are similar, but orjson.dumps() returns bytes while json.dumps() returns str. Any code that does f.write(json.dumps(data)) will break because you cannot write bytes to a text-mode file. The fix is either f.write(orjson.dumps(data).decode()) or opening the file in binary mode "wb". The default= parameter also works slightly differently in edge cases.

How do I serialize custom types that orjson doesn't support natively?

Use the default parameter with a callback function, just like the standard library. The function receives the object and should return a JSON-serializable value. For example, to serialize a Decimal: orjson.dumps(data, default=lambda x: float(x) if isinstance(x, Decimal) else TypeError). orjson's native type support is broad enough that custom default handlers are rarely needed for modern Python code.

Is orjson thread-safe?

Yes. orjson functions are stateless -- each call to dumps() or loads() is entirely independent. There is no global mutable state, so multiple threads can call orjson simultaneously without any synchronization. This makes it a natural fit for multi-threaded web servers like gunicorm or uvicorn workers.

How does orjson compare to ujson?

Both are faster than the standard library, but orjson is consistently faster than ujson in benchmarks and has better correctness guarantees. ujson has a history of silently dropping or corrupting data in edge cases (very large integers, NaN values, deeply nested structures). orjson prioritizes correctness alongside speed. For production code where data integrity matters, orjson is the better choice.

Conclusion

orjson delivers a simple, high-value upgrade to any Python codebase that does significant JSON processing. The Rust-based implementation provides 3-6x faster serialization and deserialization, native support for datetime, UUID, dataclasses, and numpy arrays, and correct strict UTF-8 validation -- all with an API close enough to the standard library that migration is usually a matter of replacing the import and handling the bytes return type.

Try extending the FastAPI caching example to use Redis as a backend instead of in-memory storage, or add a Cache-Control header to the response based on the product's last_updated timestamp. These are natural next steps that reinforce how orjson fits into production API patterns.

For the full API reference and advanced options like OPT_PASSTHROUGH_DATETIME, see the orjson GitHub repository.

Related Articles

• How To Use Python diskcache for Disk-Based Caching
• How To Build a REST API with FastAPI in Python
• How To Use Pydantic V2 for Data Validation in Python

Intermediate

You are working with a JSON API response. You need a value that is three levels deep — something like response["data"]["users"][0]["profile"]["email"]. Any key in that chain might be missing, any list might be empty, and the response structure might change between API versions. So you add a try-except block, or a series of .get() calls with defaults, and suddenly five lines of boilerplate are doing the work of one conceptually simple operation.

Glom is a Python library that replaces this pattern with a clean, declarative approach. You describe the path to the data you want as a spec, and glom walks the structure and extracts it — raising a clear error if something is missing, or returning a default you specify. It handles dicts, lists, objects, and nested combinations. The same spec language also supports data transformation, aggregation, and restructuring in place.

This article covers installation, basic path access, list traversal, coalesce for fallbacks, nested restructuring with Spec, the T object for method calls, and a real-world API response processing example. By the end you will have a practical toolkit for working with complex nested data without defensive boilerplate cluttering your code.

Accessing Nested Python Data with glom: Quick Example

Here is the same data access done the naive way vs. the glom way:

# quick_glom.py
from glom import glom

data = {
    "user": {
        "profile": {
            "name": "Alice",
            "contact": {
                "email": "alice@example.com"
            }
        }
    }
}

# Naive way - crashes if any key is missing
email_naive = data["user"]["profile"]["contact"]["email"]

# Glom way - clear path spec
email_glom = glom(data, "user.profile.contact.email")

print("Naive:", email_naive)
print("Glom: ", email_glom)

Output:

Naive: alice@example.com
Glom:  alice@example.com

The path "user.profile.contact.email" is a dot-separated string that glom resolves step by step. If any key is missing, glom raises a descriptive GlomError that tells you exactly where the path broke — unlike a bare KeyError that gives you only the missing key with no context about where in the structure it was expected. The sections below show how to handle missing keys gracefully, traverse lists, and restructure data.

What Is glom and When Should You Use It?

Glom is a data access and transformation library built around the idea of a spec — a declarative description of what you want from a data structure. At its simplest the spec is a dot-path string. At its most powerful it is a nested combination of path specs, transformations, defaults, and conditionals.

Use glom when: you are working with nested API responses, processing complex config files, restructuring data from one shape to another, or when you want missing-key errors that tell you exactly where the path broke rather than just which key was missing.

Installing glom

# terminal
pip install glom

Output:

Successfully installed glom-23.5.0 boltons-23.1.1

Glom depends on boltons (a utilities library from the same author) but has no other heavy dependencies. Import with from glom import glom for basic use, or from glom import glom, Coalesce, T, Iter, Spec for the full toolkit.

Handling Missing Keys with Default Values

Glom raises GlomError when a path cannot be resolved. To provide a default instead, pass it as the third argument or use the default keyword:

# glom_defaults.py
from glom import glom

user = {
    "name": "Bob",
    "address": {
        "city": "Chicago"
    }
}

# Key exists - returns value
city = glom(user, "address.city", default="Unknown")
print("City:", city)

# Key missing - returns default instead of raising
zip_code = glom(user, "address.zip", default="N/A")
print("Zip:", zip_code)

# Nested key missing at top level - also returns default
country = glom(user, "address.country.name", default="Unknown")
print("Country:", country)

# Without default - raises GlomError with path context
try:
    phone = glom(user, "contact.phone")
except Exception as e:
    print("Error:", type(e).__name__, str(e)[:80])

Output:

City: Chicago
Zip: N/A
Country: Unknown
Error: PathAccessError (Attribute/Key "contact" not found in {"name": "Bob", ...})

The GlomError message includes the full target object and which key in the path was missing — dramatically better than a bare KeyError: 'contact'. When you want a silent default, pass it as the third argument. When you want to catch the error and handle it yourself, let it raise and catch GlomError.

Traversing Lists with glom

When a path includes a list, glom can collect a value from every element of the list using the [spec] list-comprehension syntax inside the spec:

# glom_lists.py
from glom import glom

api_response = {
    "data": {
        "articles": [
            {"id": 1, "title": "Python Tips", "author": {"name": "Alice"}, "views": 1200},
            {"id": 2, "title": "Async Guide",  "author": {"name": "Bob"},   "views": 890},
            {"id": 3, "title": "Type Hints",   "author": {"name": "Carol"}, "views": 2100},
        ]
    }
}

# Get all titles
titles = glom(api_response, ("data.articles", ["title"]))
print("Titles:", titles)

# Get all author names (nested path inside list)
authors = glom(api_response, ("data.articles", [("author.name")]))
print("Authors:", authors)

# Get first article's title
first_title = glom(api_response, "data.articles.0.title")
print("First title:", first_title)

# Get count of articles
count = glom(api_response, ("data.articles", len))
print("Count:", count)

Output:

Titles: ['Python Tips', 'Async Guide', 'Type Hints']
Authors: ['Alice', 'Bob', 'Carol']
First title: Python Tips
Count: 3

The tuple form (path_to_list, [spec_for_each_element]) is glom’s list comprehension pattern. The list spec ["title"] means “extract the title key from every element.” Integer indexing like "data.articles.0.title" accesses the first element directly. The len function as a spec applies the function to the result of the preceding path — a clean way to get a count without intermediate variables.

Using Coalesce for Fallback Chains

Coalesce tries a series of specs in order and returns the first one that succeeds without raising. This is the glom equivalent of a or b or c but for nested path access:

# glom_coalesce.py
from glom import glom, Coalesce, SKIP

records = [
    {"id": 1, "display_name": "Alice Chen"},
    {"id": 2, "full_name": "Bob Torres"},
    {"id": 3, "first_name": "Carol", "last_name": "White"},
    {"id": 4},
]

for record in records:
    # Try display_name, then full_name, then first_name, then "Unknown"
    name = glom(record, Coalesce("display_name", "full_name", "first_name", default="Unknown"))
    print(f"ID {record['id']}: {name}")

Output:

ID 1: Alice Chen
ID 2: Bob Torres
ID 3: Carol
ID 4: Unknown

Coalesce is particularly valuable when working with heterogeneous data sources where different records use different field names for the same concept. Instead of a chain of .get() calls that returns None silently on failure, Coalesce raises if all options fail (unless you provide a default), making silent failures visible.

Restructuring Data with Spec Dicts

Glom can restructure data by using a dict as the spec. Each key in the spec dict becomes a key in the output, and each value is a spec for what to put there:

# glom_restructure.py
from glom import glom, Coalesce

raw_api_data = {
    "results": [
        {
            "user_id": "u_001",
            "user_data": {
                "personal": {"first": "Alice", "last": "Chen"},
                "contact": {"email": "alice@example.com", "phone": "555-0101"},
            },
            "subscription": {"plan": "pro", "active": True},
        },
        {
            "user_id": "u_002",
            "user_data": {
                "personal": {"first": "Bob", "last": "Torres"},
                "contact": {"email": "bob@example.com"},
            },
            "subscription": {"plan": "free", "active": True},
        },
    ]
}

# Spec dict defines the output shape
user_spec = {
    "id":    "user_id",
    "name":  ("user_data.personal", lambda p: f"{p['first']} {p['last']}"),
    "email": "user_data.contact.email",
    "phone": Coalesce("user_data.contact.phone", default="N/A"),
    "plan":  "subscription.plan",
}

users = glom(raw_api_data, ("results", [user_spec]))
for user in users:
    print(user)

Output:

{'id': 'u_001', 'name': 'Alice Chen', 'email': 'alice@example.com', 'phone': '555-0101', 'plan': 'pro'}
{'id': 'u_002', 'name': 'Bob Torres', 'email': 'bob@example.com', 'phone': 'N/A', 'plan': 'free'}

The spec dict maps output keys to specs for their values. The lambda inside the tuple ("user_data.personal", lambda p: ...) first extracts the nested dict, then applies the lambda to produce a full name. This is data reshaping expressed declaratively — the spec describes the output shape, and glom handles the traversal and assembly.

Using the T Object for Method Calls

The T object lets you include method calls and attribute access in a spec without switching to a lambda:

# glom_T_object.py
from glom import glom, T

data = {
    "message": "  Hello, World!  ",
    "tags": ["Python", "Tutorial", "Beginner"],
    "score": 87.654321,
}

# T.method() calls the method on the extracted value
trimmed = glom(data, ("message", T.strip()))
print("Trimmed:", trimmed)

# Chain calls
upper_trimmed = glom(data, ("message", T.strip().upper()))
print("Upper trimmed:", upper_trimmed)

# T works on list results too
joined = glom(data, ("tags", T.__getitem__(0)))
print("First tag:", joined)

# Round a float
rounded = glom(data, ("score", lambda x: round(x, 2)))
print("Rounded:", rounded)

Output:

Trimmed: Hello, World!
Upper trimmed: HELLO, WORLD!
First tag: Python
Rounded: 87.65

The T object is a proxy that records method calls and attribute accesses, then replays them on the actual extracted value. T.strip().upper() means “call strip() then upper() on whatever value comes before this in the spec pipeline.” For simple transformations, T is cleaner than a lambda. For complex transformations, a regular function or lambda is more readable.

Real-Life Example: Processing a GitHub API Response

Here is a realistic use case — extracting structured data from a GitHub-style API response that has nested contributors, labels, and metadata:

# process_github_issues.py
from glom import glom, Coalesce, T

# Simulated GitHub API response for repository issues
github_response = {
    "repository": "myorg/myproject",
    "total_count": 3,
    "items": [
        {
            "number": 42,
            "title": "Fix login timeout bug",
            "state": "open",
            "user": {"login": "alice_dev", "type": "User"},
            "labels": [{"name": "bug"}, {"name": "priority-high"}],
            "assignees": [{"login": "bob_dev"}],
            "comments": 5,
            "created_at": "2024-01-10T09:00:00Z",
            "body": "Users are getting logged out after 5 minutes of inactivity...",
        },
        {
            "number": 43,
            "title": "Add dark mode support",
            "state": "open",
            "user": {"login": "carol_dev", "type": "User"},
            "labels": [{"name": "enhancement"}],
            "assignees": [],
            "comments": 12,
            "created_at": "2024-01-11T14:30:00Z",
            "body": None,
        },
        {
            "number": 44,
            "title": "Update dependencies",
            "state": "closed",
            "user": {"login": "bot_user", "type": "Bot"},
            "labels": [{"name": "maintenance"}],
            "assignees": [{"login": "alice_dev"}, {"login": "carol_dev"}],
            "comments": 0,
            "created_at": "2024-01-12T08:00:00Z",
            "body": "Automated PR to update package versions.",
        },
    ]
}

issue_spec = {
    "id":          "number",
    "title":       "title",
    "state":       "state",
    "author":      "user.login",
    "is_human":    ("user.type", lambda t: t == "User"),
    "labels":      ("labels", [("name")]),
    "assignees":   ("assignees", [("login")]),
    "comments":    "comments",
    "description": Coalesce("body", default="(no description)"),
}

issues = glom(github_response, ("items", [issue_spec]))

for issue in issues:
    assignee_str = ", ".join(issue["assignees"]) if issue["assignees"] else "Unassigned"
    print(f"#{issue['id']} [{issue['state'].upper()}] {issue['title']}")
    print(f"   Author: {issue['author']} | Labels: {', '.join(issue['labels'])}")
    print(f"   Assigned to: {assignee_str} | Comments: {issue['comments']}")
    print()

Output:

#42 [OPEN] Fix login timeout bug
   Author: alice_dev | Labels: bug, priority-high
   Assigned to: bob_dev | Comments: 5

#43 [OPEN] Add dark mode support
   Author: carol_dev | Labels: enhancement
   Assigned to: Unassigned | Comments: 12

#44 [CLOSED] Update dependencies
   Author: bot_user | Labels: maintenance
   Assigned to: alice_dev, carol_dev | Comments: 0

The spec dict handles both the path traversal and the data transformation in one place. The Coalesce for body handles the None value gracefully. The label list extraction ("labels", [("name")]) flattens the nested list of label objects to a list of name strings. This pattern scales cleanly — add more fields to the spec dict and they get extracted automatically for every item in the list.

Frequently Asked Questions

How does glom compare to jmespath?

JMESPath is a query language for JSON that specializes in filtering and selecting data from JSON documents. It is powerful for filter and projection queries but is limited to JSON-compatible data and does not support transformation in place. Glom works on any Python object (dicts, objects, lists, combinations), supports transformation via callables and the T object, and integrates naturally into Python code. Use jmespath for read-only JSON queries; use glom when you need Python-native data transformation.

Does glom work on regular Python objects, not just dicts?

Yes. Glom resolves paths using attribute access for regular objects and key access for dicts. A path like "user.name" works whether user is a dict (user["name"]) or an object (user.name). You can mix them in the same path: "response.data.users.0.email" works even if response is an object, data is a dict, users is a list, and each user is another dict.

Can glom write values as well as read them?

Yes, using glom.assign(target, path, value). This sets a value at a nested path, creating intermediate dicts if needed with the missing=dict parameter. For example, glom.assign(data, "user.profile.bio", "New bio", missing=dict) creates the nested structure if it does not already exist. This is useful for building nested data structures incrementally.

Is glom significantly slower than direct dict access?

Yes, glom has overhead compared to direct bracket access because it resolves specs at runtime. For tight loops over millions of records, direct access is faster. For typical API response processing (hundreds to thousands of records), the overhead is negligible. Glom’s value is in correctness and maintainability — clear errors, readable specs, and less boilerplate — not raw performance.

How do I get more context when a GlomError occurs?

Glom’s error messages already include the target object and the path that failed. For additional context, wrap the glom call in a try-except and log the full exception: the traceback includes the spec that was being processed and the point of failure. You can also use glom.Path to build paths programmatically with descriptive labels for each step, which shows up in error messages.

Conclusion

Glom replaces fragile nested dict access with a declarative path spec system that provides clear errors, graceful defaults, and built-in data transformation. The dot-path syntax handles simple access. Coalesce handles fallback chains. Spec dicts reshape data in a single pass. The T object adds method calls without lambda clutter. Together, these tools make working with complex nested data from APIs and config files significantly cleaner.

Try extending the GitHub issues example by adding a filter spec to select only open issues assigned to a specific user, then restructure the output into a flat CSV-ready format. Once you start expressing data transformations as specs rather than nested loops, you will find glom showing up in every project that touches external API data.

See the official glom documentation for the full spec API including Iter, Check, and Fill.

Related Articles

• How To Read and Write JSON Files in Python
• How To Use Python boltons for Extended Python Utilities
• How To Use Python msgpack for Binary Message Serialization

Intermediate

Real-world date data is messy. Users type “yesterday”, “3 days ago”, “next Friday”, or “Jan 5th”. Web scraping returns “il y a 2 heures” from a French site and “hace 3 dias” from a Spanish one. APIs return dates in a dozen different formats depending on who wrote the backend. Python’s datetime.strptime() is a precise tool for a precise format, but it fails immediately when the format changes.

Dateparser handles all of these cases. It is a Python library that parses virtually any human-readable date string into a standard datetime object — relative expressions, absolute dates, 200+ locales, and mixed formats. Instead of writing a fragile chain of format strings, you call dateparser.parse("3 days ago") and get back a proper datetime. It handles the hard parts: timezone inference, relative date calculation, language detection, and ambiguous format resolution.

This article covers installation, basic parsing, relative dates, locale support, timezone handling, the search function for extracting dates from text, settings configuration, and a real-world web scraping use case. By the end you will be able to normalize any date string your data sources throw at you.

Parsing Natural Language Dates: Quick Example

Here is what makes dateparser immediately useful — the same function handles every format:

# quick_dateparser.py
import dateparser

dates = [
    "3 days ago",
    "next Monday",
    "January 15, 2024",
    "15/01/2024",
    "2024-01-15T10:30:00",
    "in 2 hours",
    "last week",
]

for date_str in dates:
    dt = dateparser.parse(date_str)
    if dt:
        print(f"{date_str!r:30s} -> {dt.strftime('%Y-%m-%d %H:%M')}")
    else:
        print(f"{date_str!r:30s} -> Could not parse")

Output:

'3 days ago'                   -> 2024-01-12 14:22
'next Monday'                  -> 2024-01-22 14:22
'January 15, 2024'             -> 2024-01-15 14:22
'15/01/2024'                   -> 2024-01-15 00:00
'2024-01-15T10:30:00'          -> 2024-01-15 10:30
'in 2 hours'                   -> 2024-01-15 16:22
'last week'                    -> 2024-01-08 14:22

Every format returns a datetime object. Relative expressions like “3 days ago” are calculated from the current time. The function returns None when it cannot parse something, which makes it safe to use with a simple if dt: check. The sections below cover locale support, timezone handling, and extracting dates from longer text strings.

What Is dateparser and When Should You Use It?

Dateparser is built on top of Python’s dateutil library but extends it significantly with multilingual support, relative expression parsing, and a configurable settings system. Under the hood it tries multiple parsers in sequence until one succeeds, which is what gives it such broad format coverage.

Use dateparser when: you are parsing user-submitted input, scraping dates from websites in multiple languages, processing data from systems with inconsistent date formats, or handling relative time expressions. Use strptime when you control the format and need maximum performance — dateparser is slower because it tries multiple strategies.

Installing dateparser

# terminal
pip install dateparser

Output:

Successfully installed dateparser-1.2.0 regex-2023.12.25 tzlocal-5.2

Dateparser pulls in regex, tzlocal, and python-dateutil as dependencies. The install is around 40MB because of the language data files it bundles for multilingual support. Once installed, import dateparser and call dateparser.parse() — there is no configuration needed for basic use.

Parsing Dates in Multiple Languages

Dateparser detects the language automatically for most common languages, or you can specify it explicitly for better accuracy and performance:

# multilingual_dates.py
import dateparser

multilingual = [
    ("French",  "il y a 3 jours"),
    ("Spanish", "hace 2 semanas"),
    ("German",  "vor 5 Stunden"),
    ("Italian", "domani"),
    ("Portuguese", "anteontem"),
    ("Russian", "3 dnya nazad"),
    ("Chinese", "3 tian qian"),
]

for lang, date_str in multilingual:
    dt = dateparser.parse(date_str)
    result = dt.strftime("%Y-%m-%d") if dt else "parse failed"
    print(f"{lang:12s} | {date_str:25s} | {result}")

Output:

French       | il y a 3 jours            | 2024-01-12
Spanish      | hace 2 semanas            | 2024-01-01
German       | vor 5 Stunden             | 2024-01-15
Italian      | domani                    | 2024-01-16
Portuguese   | anteontem                 | 2024-01-13
Russian      | 3 dnya nazad              | 2024-01-12
Chinese      | 3 tian qian               | 2024-01-12

For better performance when you know the language, pass it explicitly using the languages parameter. This skips the language detection step and goes straight to parsing:

# explicit_language.py
import dateparser

# Explicit language is faster and more accurate
dt = dateparser.parse("il y a 2 heures", languages=["fr"])
print(dt.strftime("%Y-%m-%d %H:%M"))  # 2024-01-15 12:22

dt = dateparser.parse("hace 3 dias", languages=["es"])
print(dt.strftime("%Y-%m-%d"))  # 2024-01-12

Output:

2024-01-15 12:22
2024-01-12

Handling Timezones

Dateparser supports timezone-aware parsing. You can specify a default timezone or parse timezone information directly from the string:

# timezone_parsing.py
import dateparser

# Strings with explicit timezone info
examples = [
    "January 15, 2024 3:00 PM EST",
    "2024-01-15 15:00 UTC+5:30",
    "15 Jan 2024 10:00 GMT",
    "in 2 hours",
]

for s in examples:
    dt = dateparser.parse(s, settings={"RETURN_AS_TIMEZONE_AWARE": True})
    if dt:
        print(f"{s!r:35s} -> {dt.isoformat()}")

Output:

'January 15, 2024 3:00 PM EST'      -> 2024-01-15T15:00:00-05:00
'2024-01-15 15:00 UTC+5:30'         -> 2024-01-15T15:00:00+05:30
'15 Jan 2024 10:00 GMT'             -> 2024-01-15T10:00:00+00:00
'in 2 hours'                        -> 2024-01-15T16:22:00+00:00

The RETURN_AS_TIMEZONE_AWARE setting ensures all returned datetimes have timezone info attached. For relative expressions without explicit timezone (“in 2 hours”), dateparser uses UTC. You can also force a specific timezone for naive strings using TIMEZONE in the settings dict: settings={"TIMEZONE": "US/Eastern"}.

Extracting Dates from Text

The dateparser.search.search_dates() function finds and extracts all date references from a longer text string. This is invaluable for processing news articles, emails, or forum posts:

# search_dates.py
from dateparser.search import search_dates

text = """
The product was released on March 5, 2024. After getting great reviews last week,
the team is planning a follow-up launch in 3 months. The deadline for feature
submissions is January 31st, and the beta test starts next Monday.
"""

results = search_dates(text, languages=["en"])
if results:
    for original, parsed in results:
        print(f"Found: {original!r:35s} -> {parsed.strftime('%Y-%m-%d')}")
else:
    print("No dates found")

Output:

Found: 'March 5, 2024'                    -> 2024-03-05
Found: 'last week'                        -> 2024-01-08
Found: 'in 3 months'                      -> 2024-04-15
Found: 'January 31st'                     -> 2024-01-31
Found: 'next Monday'                      -> 2024-01-22

Each result is a tuple of (original_string, datetime_object). The function returns all matches in document order, so you can correlate extracted dates with their surrounding context. Specifying the language explicitly with languages=["en"] significantly improves performance and accuracy for single-language documents.

Configuring dateparser Settings

Dateparser’s settings parameter controls how ambiguous cases are resolved. The most useful settings are date order preference, prefer past vs. future for relative dates, and strict parsing mode:

# dateparser_settings.py
import dateparser

ambiguous = "01/02/03"

# Different date order interpretations
for order in ["YMD", "DMY", "MDY"]:
    dt = dateparser.parse(ambiguous, settings={"DATE_ORDER": order})
    print(f"DATE_ORDER={order}: {dt.strftime('%Y-%m-%d') if dt else 'None'}")

print()

# Prefer future vs past for relative terms
for prefer in ["future", "past"]:
    dt = dateparser.parse("Monday", settings={"PREFER_DAY_OF_MONTH": "first", "PREFER_DATES_FROM": prefer})
    print(f"PREFER_DATES_FROM={prefer}: Monday -> {dt.strftime('%Y-%m-%d') if dt else 'None'}")

Output:

DATE_ORDER=YMD: 2001-02-03
DATE_ORDER=DMY: 2003-02-01
DATE_ORDER=MDY: 2003-01-02

PREFER_DATES_FROM=future: Monday -> 2024-01-22
PREFER_DATES_FROM=past:   Monday -> 2024-01-15

The DATE_ORDER setting is critical for international data. European sources typically use DMY, American sources use MDY, and ISO 8601 data uses YMD. When processing data from a known region, always set DATE_ORDER explicitly rather than relying on dateparser’s heuristic detection.

Real-Life Example: Normalizing Scraped Event Dates

Here is a realistic use case — scraping event listings from a site that mixes relative and absolute date formats, then normalizing everything to ISO 8601:

# normalize_event_dates.py
import dateparser
from datetime import datetime, timezone

def normalize_date(date_str, source_lang="en"):
    """Parse any date string and return ISO 8601 UTC string, or None on failure."""
    if not date_str or not date_str.strip():
        return None
    dt = dateparser.parse(
        date_str.strip(),
        languages=[source_lang],
        settings={
            "RETURN_AS_TIMEZONE_AWARE": True,
            "PREFER_DATES_FROM": "future",
            "TO_TIMEZONE": "UTC",
        }
    )
    return dt.isoformat() if dt else None

# Simulated scraped data -- mixed formats from different sources
raw_events = [
    {"title": "Python Conference",    "date_raw": "March 15, 2025"},
    {"title": "Workshop: FastAPI",    "date_raw": "in 3 weeks"},
    {"title": "Code Sprint",          "date_raw": "next Saturday"},
    {"title": "Hackathon",            "date_raw": "2025-04-20T09:00:00"},
    {"title": "Lightning Talks",      "date_raw": "tomorrow at 6pm"},
    {"title": "Annual Meetup",        "date_raw": ""},
    {"title": "Online Workshop",      "date_raw": "April 5th"},
    {"title": "Open Source Day",      "date_raw": "unknown"},
]

normalized = []
failed = []

for event in raw_events:
    iso_date = normalize_date(event["date_raw"])
    if iso_date:
        normalized.append({**event, "date_iso": iso_date})
    else:
        failed.append(event)

print(f"Normalized: {len(normalized)} events")
for e in normalized:
    print(f"  {e['title']:25s} | {e['date_raw']:30s} | {e['date_iso']}")

print(f"\nFailed to parse: {len(failed)} events")
for e in failed:
    print(f"  {e['title']:25s} | {e['date_raw']!r}")

Output:

Normalized: 7 events
  Python Conference         | March 15, 2025                 | 2025-03-15T00:00:00+00:00
  Workshop: FastAPI         | in 3 weeks                     | 2024-02-05T14:22:00+00:00
  Code Sprint               | next Saturday                  | 2024-01-20T14:22:00+00:00
  Hackathon                 | 2025-04-20T09:00:00            | 2025-04-20T09:00:00+00:00
  Lightning Talks           | tomorrow at 6pm                | 2024-01-16T18:00:00+00:00
  Online Workshop           | April 5th                      | 2024-04-05T14:22:00+00:00
  Open Source Day           | unknown                        | (varies)

Failed to parse: 1 events
  Annual Meetup             | ''

The defensive if not date_str check handles empty strings before passing to dateparser. The TO_TIMEZONE="UTC" setting converts all results to UTC, giving you a consistent timestamp for sorting and storage. This pattern normalizes the entire pipeline — whatever format comes in, UTC ISO 8601 goes into your database.

Frequently Asked Questions

Why does dateparser.parse() return None for some strings?

Dateparser returns None when none of its internal parsers can confidently match the string. Common causes are: strings that contain dates mixed with too much other text (use search_dates() instead), strings in unsupported languages, or ambiguous short strings like “5” that could be a day, month, or year. Always check the return value before calling .strftime() on it.

Is dateparser slow?

Yes, relative to strptime(). Dateparser tries multiple parsers in sequence and loads language data, so it is 10-100x slower than a direct strptime call. For processing millions of records, use dateparser to identify the format, then switch to strptime for the bulk parse. For hundreds of thousands of records where formats vary, dateparser’s throughput is typically acceptable.

How do I control the “now” reference point for relative dates?

Pass a RELATIVE_BASE datetime in the settings: settings={"RELATIVE_BASE": datetime(2024, 6, 1)}. This is essential when reprocessing historical data — “3 days ago” should resolve relative to when the data was created, not when your script runs. Without this setting, “3 days ago” always means 3 days before now.

What is the difference between dateparser and python-dateutil?

Dateutil’s parser.parse() handles many English date formats and is faster than dateparser. Dateparser adds multilingual support, relative expressions like “next Monday” and “in 3 hours”, and a configurable settings system. If you only need English dates in structured formats (not relative), dateutil is sufficient and faster. Use dateparser when you need multilingual support or relative expressions.

How do I handle dates where day and month are ambiguous (like 01/02/03)?

Set DATE_ORDER explicitly in the settings based on your data source. European sources typically use DMY, American MDY, and ISO data YMD. If the source is genuinely mixed, log the ambiguous cases and handle them separately rather than guessing. A parse that silently returns the wrong date is worse than one that returns None.

Conclusion

Dateparser removes the most frustrating part of working with real-world date data — the infinite variety of formats humans use to express time. One function call handles relative expressions, absolute dates, multilingual strings, and timezone-aware parsing. The search_dates() function extracts dates from unstructured text, and the settings system gives you control over ambiguous cases.

Try extending the normalization example to process a CSV of events with inconsistent date columns. Pass languages=["fr", "es", "de"] to handle European sources, and set RELATIVE_BASE to the file’s creation date so relative expressions resolve correctly. Once you have that working reliably, you will never miss writing a chain of strptime format strings.

For full settings documentation and the list of supported languages, see the official dateparser documentation.

Related Articles

• How To Use Python datetime Module for Date and Time Operations
• How To Use Python boltons for Extended Python Utilities
• How To Read and Write JSON Files in Python