Integrations

This article describes how adaptix works with other packages and systems.

Supported model kinds

Models are classes that have a predefined set of fields. adaptix processes models in the same, consistent way.

Models that are supported out of the box:

Arbitrary types also are supported to be loaded by introspection of __init__ method, but it can not be dumped.

You do not need to do anything to enable support for models from a third-party library. Everything just works. But you can install adaptix with certain extras to ensure version compatibility.

Known peculiarities and limitations

dataclass

  • Signature of custom __init__ method must be same as signature of generated by @dataclass, because there is no way to distinguish them.

TypedDict

  • Due to the way Python works with annotations, there is a bug, when field annotation of TypedDict is stringified or from __future__ import annotations is placed in file Required and NotRequired specifiers is ignored when required_keys and optional_keys is calculated. adaptix takes this into account and processes it properly.

__init__ introspection or using constructor

  • Fields of unpacked typed dict (**kwargs: Unpack[YourTypedDict]) cannot collide with parameters of function.

sqlalchemy

  • Only mapping to Table is supported, implementations for FromClause instances such as Subquery and Join are not provided.

  • dataclass and attrs mapped by sqlalchemy are not supported for introspection.

  • It does not support registering the order of mapped fields by design, so you should use manual mapping to list instead of automatic as_list=True.

  • Relationships with custom collection_class are not supported.

  • All input fields of foreign keys and relationships are considered as optional due to a user can pass only relationship instances or only foreign key values.

pydantic

  • Custom __init__ function must have only one parameter accepting arbitrary keyword arguments (like **kwargs or **data).

  • There are 3 categories of fields: regular fields, computed fields (marked properties), and private attributes. Pydantic tracks order inside one category but does not track between categories. Also, pydantic does not keep the right order inside private attributes.

    Therefore, during the dumping of fields, regular fields will come first, followed by computed fields, and then private attributes. You can use manual mapping to list instead of automatic as_list=True to control the order.

  • Fields with constraints defined by parameters (like f1: int = Field(gt=1, ge=10)) are translated to Annotated with corresponding metadata. Metadata is generated by Pydantic and consists of objects from annotated_types package (like Annotated[int, Gt(gt=1), Ge(ge=10)]).

  • Parametrized generic pydantic models do not expose common type hints dunders that prevents appropriate type hints introspection. This leads to incorrect generics resolving in some tricky cases.

    Also, there are some bugs in generic resolving inside pydantic itself.

  • Pydantic does not support variadic generics.

  • pydantic.dataclasses is not supported.

  • pydantic.v1 is not supported.

Working with Pydantic

By default, any pydantic model is loaded and dumped like any other model. For example, any aliases or config parameters defined inside the model are ignored. You can override this behavior to use a native pydantic validation/serialization mechanism.

from adaptix import Retort
from adaptix.integrations.pydantic import native_pydantic
from pydantic import BaseModel, Field


class Book(BaseModel):
    title: str = Field(alias="name")
    price: int


data = {
    "name": "Fahrenheit 451",
    "price": 100,
}

retort = Retort(
    recipe=[
        native_pydantic(Book, to_python={"by_alias": True}),
    ],
)

book = retort.load(data, Book)
assert book == Book(name="Fahrenheit 451", price=100)
assert retort.dump(book) == data

Working with msgspec

By default, any msgspec Struct is loaded, dumped and converted like any other model. If your code uses specific options for to_builtins or convert functions, you can specify them with native msgspec mechanism defined in Retort as shown in example.

import datetime

from adaptix import Retort
from adaptix.integrations.msgspec import native_msgspec
from msgspec import Struct


class Music(Struct):
    released: datetime.date
    composition: str


data = {
    "released": datetime.date(2007,1,20),
    "composition": "Espacio de silencio",
}

retort = Retort(
    recipe=[
        native_msgspec(Music, to_builtins={"builtin_types": [datetime.date]}),
    ],
)

assert data == retort.dump(
    Music(
        datetime.date(2007, 1, 20),
        "Espacio de silencio",
    ),
)

SQLAlchemy JSON

You can use adaptix to store structured JSON data inside a relational database. SQLAlchemy will automatically map JSON to your model using adaptix.

Let’s see how you can define database schema.

from dataclasses import dataclass
from typing import Literal

from adaptix import Retort
from adaptix.integrations.sqlalchemy import AdaptixJSON
from sqlalchemy.orm import DeclarativeBase, Mapped, mapped_column


@dataclass
class UserCreated:
    id: int
    name: str

    tag: Literal["user_created"] = "user_created"


@dataclass
class UserChanged:
    id: int
    name: str

    tag: Literal["user_changed"] = "user_changed"


AnyAuditLog = UserCreated | UserChanged


class Base(DeclarativeBase):
    pass


db_retort = Retort()


class AuditLogRecord(Base):
    __tablename__ = "audit_logs"

    id: Mapped[int] = mapped_column(primary_key=True)
    data: Mapped[AnyAuditLog] = mapped_column(AdaptixJSON(db_retort, AnyAuditLog))

The constructor of AdaptixJSON takes two parameters: retort and type of data. Also, you can pass a custom JSON column type via impl keyword parameter.

Basic usage

You can pass your model to any place where SQLAlchemy expects an instance of data.

session.add(
    AuditLogRecord(
        data=UserCreated(id=1, name="Sam"),
    ),
)

session.execute(
    insert(AuditLogRecord)
    .values(
        data=UserChanged(id=1, name="Leo"),
    ),
)
session.commit()

assert session.get(AuditLogRecord, 1).data == UserCreated(id=1, name="Sam")
assert session.get(AuditLogRecord, 2).data == UserChanged(id=1, name="Leo")

Querying and filtering

AdaptixJSON is TypeDecorator of JSON type, so you can use any method of JSON type to produce query.

session.add(
    AuditLogRecord(
        data=UserCreated(
            id=1,
            name="Sam",
        ),
    ),
)
session.commit()

record = session.scalar(
    select(AuditLogRecord)
    .where(AuditLogRecord.data["id"].as_integer() == 1),
)
assert record.data == UserCreated(id=1, name="Sam")

Caution

name_mapping is not applied to the query builder. Your query will use json representation of model.

Mutation tracking

SQLAlchemy flushes objects only if some are marked as modified (dirty). The instance becomes dirty when __setattr__ is invoked. So, SQLAlchemy cannot track the mutation of the object associated with the attribute.

with session_factory() as session:
    session.add(
        AuditLogRecord(
            data=UserCreated(
                id=1,
                name="Example",
            ),
        ),
    )
    session.commit()

with session_factory() as session:
    record = session.get(AuditLogRecord, 1)
    record.data.name = "Example2"
    session.commit()

with session_factory() as session:
    record = session.get(AuditLogRecord, 1)
    assert record.data.name == "Example"  # (!) mutation tracking does not work (!)
Workaround for mutation tracking

  1. Modify the entire object graph to track mutation at any edge. See for details mutable extension.

  2. Save a copy of the dataclass at the ‘after_attach’ event and compares it with an actual object at ‘before_flush’. If it differs, mark the instance as dirty via flag_modified. This will work only if session.flush() is called directly. Other methods may skip calling flush (and invoking its events) if there are no dirty objects.

  3. Call flag_modified directly after dataclass mutation.

Finally, it is not recommended to mutate a model that will be persistent to JSON, because more likely it means that you should not store JSON in RDBMS.

Redefining none_as_null parameter

To override the parameters of JSON type itself, you can pass a custom SQLAlchemy type.

from sqlalchemy import JSON

class AuditLogRecord(Base):
    __tablename__ = "audit_logs2"

    id: Mapped[int] = mapped_column(primary_key=True)
    data: Mapped[AnyAuditLog] = mapped_column(
        AdaptixJSON(db_retort, AnyAuditLog, impl=JSON(none_as_null=True)),
    )