Intermediate

JSON serialization is in the hot path of almost every Python web service: every API response encodes data to JSON, every incoming request decodes it. If your service handles thousands of requests per second, JSON encoding time adds up fast. Python’s built-in json module is correct and convenient, but it was not built for speed — and libraries like orjson are fast but handle only basic Python types. What if you want both speed and type safety?

msgspec is a high-performance serialization library that encodes and decodes JSON (and MessagePack) 5-10x faster than the standard library while also providing automatic type validation. You define your data shape once using msgspec.Struct — like a Pydantic model but with a smaller memory footprint and faster instantiation — and msgspec handles encoding, decoding, and validation in a single C-extension call.

This article covers installing msgspec, defining Struct classes, encoding and decoding JSON, using type annotations for automatic validation, handling optional and nested fields, working with MessagePack, and benchmarking against the standard library. By the end you will have a complete toolkit for high-performance, type-safe JSON serialization in Python.

msgspec Quick Example

# quick_msgspec.py
import msgspec
import msgspec.json

class User(msgspec.Struct):
    name: str
    email: str
    age: int
    is_active: bool = True

# Encode to JSON bytes
user = User(name="Alice", email="alice@example.com", age=30)
encoded = msgspec.json.encode(user)
print(encoded)
print(type(encoded))

# Decode from JSON bytes -- returns a User instance with validation
raw = b'{"name":"Bob","email":"bob@example.com","age":25}'
decoded = msgspec.json.decode(raw, type=User)
print(decoded)
print(type(decoded))

b'{"name":"Alice","email":"alice@example.com","age":30,"is_active":true}'
<class 'bytes'>
User(name='Bob', email='bob@example.com', age=25, is_active=True)
<class '__main__.User'>

The encode() function serializes a Struct (or any supported Python type) to JSON bytes. The decode() function deserializes JSON bytes and validates them against a type — if the JSON does not match the expected shape, a ValidationError is raised. Unlike the stdlib json module, the output is bytes, not str, which is what HTTP servers and most network libraries expect anyway.

Python msgspec tutorial illustration 1
5x faster than json.dumps(). Your API deserves better.

What Is msgspec and When Should You Use It?

msgspec is a C-extension library by Jim Crist-Harif that provides both serialization performance and type safety. It is designed as a faster, lower-footprint alternative to Pydantic for use cases where you need validated deserialization at high throughput.

LibraryJSON SpeedValidationMemoryType
json (stdlib)BaselineNoLowdict/list
orjson5-10x fasterNoLowdict/list
msgspec5-10x fasterYesVery lowStruct
Pydantic v22-3x fasterYes (rich)HigherBaseModel

Use msgspec when you need high-throughput JSON encoding/decoding with type safety and low memory usage — particularly in web APIs (FastAPI, Starlette), event streaming, and data pipeline code where JSON parsing is in the hot path. Use Pydantic when you need rich validators, field aliases, custom serializers, or deep ecosystem integration. Use orjson when you need maximum speed with no schema requirements.

Installation

pip install msgspec

Successfully installed msgspec-0.18.6

msgspec ships as a pre-compiled C extension for Linux, macOS, and Windows on both CPython and PyPy. Import with import msgspec for Struct definitions and import msgspec.json for JSON operations.

Defining Structs

A msgspec.Struct is a fast, memory-efficient data class. It uses Python type annotations to define its fields and generates optimized __init__, __repr__, __eq__, and encoder/decoder hooks automatically.

# struct_basics.py
import msgspec
from typing import Optional, List
from datetime import datetime

class Address(msgspec.Struct):
    street: str
    city: str
    country: str
    postal_code: str = ""          # Default value

class Order(msgspec.Struct):
    order_id: str
    customer_name: str
    items: List[str]
    total: float
    shipping_address: Address       # Nested Struct
    created_at: datetime            # datetime is supported natively
    notes: Optional[str] = None    # Optional field

# Create instances
addr = Address(street="123 Main St", city="Austin", country="US", postal_code="78701")
order = Order(
    order_id="ORD-001",
    customer_name="Alice Smith",
    items=["Widget A", "Gadget B"],
    total=89.99,
    shipping_address=addr,
    created_at=datetime(2026, 4, 30, 10, 0, 0),
)

print(order)
print()
print(f"Order ID: {order.order_id}")
print(f"City: {order.shipping_address.city}")
print(f"Notes: {order.notes}")   # None -- the default

Order(order_id='ORD-001', customer_name='Alice Smith', items=['Widget A', 'Gadget B'], total=89.99, shipping_address=Address(street='123 Main St', city='Austin', country='US', postal_code='78701'), created_at=datetime.datetime(2026, 4, 30, 10, 0), notes=None)

Order ID: ORD-001
City: Austin
Notes: None

Structs are immutable by default (frozen). They have no __dict__, which makes them significantly more memory-efficient than regular Python objects or dataclasses. Access fields as attributes. Nested Structs, lists, dicts, and Python’s standard types (datetime, UUID, Decimal) are all supported as field types.

Encoding: Struct to JSON

Use msgspec.json.encode() to serialize any Struct or supported Python value to JSON bytes:

# encoding.py
import msgspec
import msgspec.json
from datetime import datetime

class Product(msgspec.Struct):
    id: int
    name: str
    price: float
    in_stock: bool
    tags: list

product = Product(id=1, name="Widget Pro", price=29.99,
                  in_stock=True, tags=["electronics", "gadgets"])

# encode returns bytes
json_bytes = msgspec.json.encode(product)
print(json_bytes)
print(type(json_bytes))

# Decode bytes to string if needed
json_str = json_bytes.decode("utf-8")
print(json_str)

# Encode a plain Python dict (msgspec handles these too)
data = {"key": "value", "num": 42, "flag": True}
print(msgspec.json.encode(data))

# Encode a list of Structs
products = [
    Product(id=i, name=f"Product {i}", price=9.99 * i,
            in_stock=i % 2 == 0, tags=[])
    for i in range(1, 4)
]
print(msgspec.json.encode(products))

b'{"id":1,"name":"Widget Pro","price":29.99,"in_stock":true,"tags":["electronics","gadgets"]}'
<class 'bytes'>
{"id":1,"name":"Widget Pro","price":29.99,"in_stock":true,"tags":["electronics","gadgets"]}
b'{"key":"value","num":42,"flag":true}'
b'[{"id":1,...},{"id":2,...},{"id":3,...}]'

Field names in the JSON output match the Struct attribute names exactly. If you need different JSON field names (e.g., camelCase in JSON, snake_case in Python), use msgspec.Struct with the rename option or rename individual fields with msgspec.field(name="camelCaseName").

Decoding with Validation

The decode() function deserializes JSON and validates it against your type annotation simultaneously. Invalid data raises a descriptive msgspec.ValidationError:

# decoding_validation.py
import msgspec
import msgspec.json
from typing import Optional, List

class UserProfile(msgspec.Struct):
    user_id: int
    username: str
    email: str
    score: float
    tags: List[str]
    bio: Optional[str] = None

# Decode valid JSON
valid_json = b'''
{
  "user_id": 42,
  "username": "alice",
  "email": "alice@example.com",
  "score": 9.5,
  "tags": ["python", "developer"],
  "bio": "Python enthusiast"
}
'''

profile = msgspec.json.decode(valid_json, type=UserProfile)
print(f"User: {profile.username}, Score: {profile.score}")
print(f"Tags: {profile.tags}")
print()

# Decode with missing optional field -- OK
minimal = b'{"user_id":1,"username":"bob","email":"b@b.com","score":7.0,"tags":[]}'
p2 = msgspec.json.decode(minimal, type=UserProfile)
print(f"Bio (optional): {p2.bio}")   # None
print()

# Decode with wrong type -- raises ValidationError
try:
    bad = b'{"user_id":"not_an_int","username":"x","email":"x@x.com","score":1.0,"tags":[]}'
    msgspec.json.decode(bad, type=UserProfile)
except msgspec.ValidationError as e:
    print(f"Validation error: {e}")

# Decode with missing required field -- raises ValidationError
try:
    missing = b'{"user_id":1,"username":"alice"}'
    msgspec.json.decode(missing, type=UserProfile)
except msgspec.ValidationError as e:
    print(f"Missing field error: {e}")

User: alice, Score: 9.5
Tags: ['python', 'developer']

Bio (optional): None

Validation error: Expected `int`, got `str` - at `$.user_id`
Missing field error: Object missing required field `email` - at `$`

The validation error messages include a JSON path ($.user_id) that tells you exactly which field failed and why. This replaces the typical pattern of calling json.loads() and then manually checking types — msgspec does both in one step, at C speed. The path notation ($ for the root, $.field for a specific field, $.list[0] for a list element) makes debugging API input validation errors straightforward.

Python msgspec tutorial illustration 2
ValidationError at $.user_id — caught at the door, not in production.

Reusable Encoder and Decoder Objects

For high-throughput code, create Encoder and Decoder objects once and reuse them. This avoids per-call setup overhead and enables encoder customization:

# encoder_decoder.py
import msgspec
import msgspec.json

class Event(msgspec.Struct):
    event_type: str
    payload: dict
    timestamp: float

# Create once at module level -- reuse for every request
encoder = msgspec.json.Encoder()
decoder = msgspec.json.Decoder(Event)

# Encode
event = Event(event_type="user_login", payload={"user_id": 42}, timestamp=1714483200.0)
data = encoder.encode(event)
print(data)

# Decode
raw = b'{"event_type":"purchase","payload":{"order_id":"123"},"timestamp":1714483260.0}'
decoded = decoder.decode(raw)
print(decoded)
print(f"Type: {decoded.event_type}, Order: {decoded.payload.get('order_id')}")

b'{"event_type":"user_login","payload":{"user_id":42},"timestamp":1714483200.0}'
Event(event_type='purchase', payload={'order_id': '123'}, timestamp=1714483260.0)
Type: purchase, Order: 123

Reusable encoder/decoder objects are the pattern to use in web servers (FastAPI, Flask) where the same type is encoded or decoded on every request. Create them at module level (outside of request handlers) so the per-type setup cost is paid once at startup, not on every request.

Real-Life Example: FastAPI Response Serialization

Here is how to use msgspec for high-performance JSON responses in a FastAPI application:

# fastapi_msgspec.py
# Install: pip install fastapi uvicorn msgspec
from fastapi import FastAPI
from fastapi.responses import Response
import msgspec
import msgspec.json
from typing import List
from datetime import datetime

app = FastAPI()

class Product(msgspec.Struct):
    id: int
    name: str
    price: float
    in_stock: bool
    updated_at: datetime

# Prebuilt encoder -- created once at module load time
encoder = msgspec.json.Encoder()

# Sample data (in a real app, this comes from a database)
PRODUCTS = [
    Product(id=i, name=f"Product {i}", price=round(9.99 * i, 2),
            in_stock=i % 3 != 0, updated_at=datetime(2026, 4, 30))
    for i in range(1, 101)
]

@app.get("/products")
def list_products() -> Response:
    """Return all products as JSON -- using msgspec for fast encoding."""
    return Response(
        content=encoder.encode(PRODUCTS),
        media_type="application/json"
    )

@app.get("/products/{product_id}")
def get_product(product_id: int) -> Response:
    """Return a single product by ID."""
    product = next((p for p in PRODUCTS if p.id == product_id), None)
    if product is None:
        return Response(content=b'{"error":"not found"}',
                        media_type="application/json", status_code=404)
    return Response(
        content=encoder.encode(product),
        media_type="application/json"
    )

if __name__ == "__main__":
    import uvicorn
    uvicorn.run(app, host="0.0.0.0", port=8000)

# Run with: python fastapi_msgspec.py
# GET /products returns 100 products as JSON
# GET /products/1 returns a single product

By bypassing FastAPI’s built-in JSON serialization (which uses the stdlib json module via Pydantic) and using a msgspec Encoder directly, you get 5-10x faster serialization for response bodies. The key pattern: return fastapi.responses.Response with pre-encoded bytes instead of returning a Python dict and letting FastAPI encode it. This is the approach used by high-traffic Python APIs to shave milliseconds off every response.

Python msgspec tutorial illustration 3
5x faster serialization on every request. At 10,000 req/s, that’s 50,000 ms saved per second.

Frequently Asked Questions

When should I use msgspec instead of Pydantic?

Use msgspec when performance is the primary concern and you do not need Pydantic’s ecosystem features: validators on individual fields, aliases, computed fields, custom serializers, or deep framework integration (Django, SQLAlchemy). msgspec Structs are faster to create, use less memory, and encode/decode faster than Pydantic models. Use Pydantic when you need its rich validation API or when your framework requires it (FastAPI’s dependency injection uses Pydantic models natively).

What is MessagePack and when should I use it instead of JSON?

MessagePack is a binary serialization format that is more compact and faster to parse than JSON. Use msgspec.msgpack.encode() and msgspec.msgpack.decode() for internal service-to-service communication where both sides speak Python. MessagePack is about 30% smaller than equivalent JSON for typical data and faster to encode/decode. Stick with JSON when the data needs to be human-readable, logged, or consumed by a non-msgspec client.

How do I make a Struct mutable?

Pass frozen=False to the Struct class: class MyStruct(msgspec.Struct, frozen=False). By default, Structs are frozen (immutable). Mutable Structs allow field assignment (obj.field = value) and can be used where you need to update individual fields after construction. Frozen Structs are hashable (can be used as dict keys or in sets); mutable Structs are not.

Does msgspec support Struct inheritance?

Yes, with limitations. A Struct subclass inherits all parent fields. However, a subclass cannot override parent fields or change their types. Struct inheritance is useful for adding fields to a base type without redefining everything, but it is not as flexible as Python class inheritance. For polymorphic types, use typing.Union with a tag field to distinguish subtypes during decoding.

msgspec encodes to bytes, but I need a str. How do I convert?

Call .decode() on the bytes: json_str = msgspec.json.encode(obj).decode("utf-8"). This adds one small allocation, but the total is still faster than json.dumps(). Most HTTP servers (WSGI/ASGI) accept bytes directly in the response body, so the conversion is often unnecessary in practice.

Conclusion

msgspec gives you the best of both worlds: the speed of a C-extension JSON library and the type safety of a schema validation library. You have seen how to define Struct classes with type annotations, encode and decode JSON with automatic validation, use reusable Encoder/Decoder objects for high-throughput code, handle optional and nested fields, and wire msgspec into FastAPI for fast API responses. The official documentation at jcristharif.com/msgspec covers advanced topics including custom hooks, YAML/TOML support, and the full type system.