Extended usage¶

This section continues the tutorial to illuminate some more complex topics.

Generic classes¶

Generic classes are supported out of the box.

from dataclasses import dataclass
from typing import Generic, TypeVar

from adaptix import Retort

T = TypeVar("T")


@dataclass
class MinMax(Generic[T]):
    min: T | None = None
    max: T | None = None


retort = Retort()

data = {"min": 10, "max": 20}
min_max = retort.load(data, MinMax[int])
assert min_max == MinMax(min=10, max=20)
assert retort.dump(min_max, MinMax[int]) == data

If a generic class is not parametrized, Python specification requires to assume Any for each position. adaptix acts slightly differently, it derives implicit parameters based on TypeVar properties.

TypeVar	Derived implicit parameter
`T = TypeVar('T')`	`Any`
`B = TypeVar('B', bound=Book)`	`Book`
`C = TypeVar('C', str, bytes)`	`Union[str, bytes]`

You should always pass concrete type to the second argument Retort.dump method. There is no way to determine the type parameter of an object at runtime due to type erasure. If you pass non-parametrized generic, retort will raise error.

Recursive data types¶

These types could be loaded and dumped without additional configuration.

from dataclasses import dataclass
from typing import List

from adaptix import Retort


@dataclass
class ItemCategory:
    id: int
    name: str
    sub_categories: List["ItemCategory"]


retort = Retort()

data = {
    "id": 1,
    "name": "literature",
    "sub_categories": [
        {
            "id": 2,
            "name": "novel",
            "sub_categories": [],
        },
    ],
}
item_category = retort.load(data, ItemCategory)
assert item_category == ItemCategory(
    id=1,
    name="literature",
    sub_categories=[
        ItemCategory(
            id=2,
            name="novel",
            sub_categories=[],
        ),
    ],
)
assert retort.dump(item_category) == data

But it does not work with cyclic-referenced objects like

item_category.sub_categories.append(item_category)

Dealing with `if TYPE_CHECKING`¶

Sometimes you want to split interdependent models into several files. This results in some imports being visible only to type checkers. Analysis of such type hints is not available at runtime.

Let’s imagine that we have two files:

File chat.py¶

from dataclasses import dataclass
from typing import TYPE_CHECKING, List

if TYPE_CHECKING:
    from .message import Message


@dataclass
class Chat:
    id: int
    name: str
    messages: List["Message"]

File message.py¶

from dataclasses import dataclass
from typing import TYPE_CHECKING

if TYPE_CHECKING:
    from .chat import Chat


@dataclass
class Message:
    id: int
    text: str
    chat: "Chat"

If you try to get type hints at runtime, you will fail:

from typing import get_type_hints

from .chat import Chat
from .message import Message

try:
    get_type_hints(Chat)
except NameError as e:
    assert str(e) == "name 'Message' is not defined"


try:
    get_type_hints(Message)
except NameError as e:
    assert str(e) == "name 'Chat' is not defined"

At runtime, these imports are not executed, so the builtin analysis function can not resolve forward refs.

adaptix can overcome this via type_tools.exec_type_checking. It extracts code fragments defined under if TYPE_CHECKING and if typing.TYPE_CHECKING constructs and then executes them in the context of module. As a result, the module namespace is filled with missing names, and any introspection function can acquire types.

You should call exec_type_checking after all required modules can be imported. Usually, it must be at main module.

File main.py¶

from typing import List, get_type_hints

from adaptix.type_tools import exec_type_checking

from . import chat, message

# You pass the module object
exec_type_checking(chat)
exec_type_checking(message)

# After these types can be extracted
assert get_type_hints(chat.Chat) == {
    "id": int,
    "name": str,
    "messages": List[message.Message],
}
assert get_type_hints(chat.Message) == {
    "id": int,
    "text": str,
    "chat": chat.Chat,
}

Detecting absence of a field¶

Sometimes you need to detect that user does not pass any value to the field. You can use None for this, but that makes it impossible to distinguish from a user-provided value.

To solve this, you can use special types called “sentinels”. Since they cannot be represented in the outside world, they provide a reliable way to determine when a field value was not explicitly set by the user.

Omitted is the builtin sentinel type. Let’s examine how it can be used:

from dataclasses import dataclass

from adaptix import Omittable, Omitted, Retort, name_mapping
from adaptix.load_error import AggregateLoadError, TypeLoadError


@dataclass
class PatchBook:
    id: int
    title: Omittable[str] = Omitted()
    sub_title: Omittable[str | None] = Omitted()


retort = Retort(
    recipe=[
        name_mapping(omit_default=True),
    ],
)

Enabling omit_default policy is required here. It allow to skip this value at dumping. All missing fields are filled by default value (Omitted), and this fields are omitted at dumping.

data = {"id": 435}
patch_book = retort.load(data, PatchBook)
assert patch_book == PatchBook(
    id=435,
    title=Omitted(),
    sub_title=Omitted(),
)
assert retort.dump(patch_book) == data

None value is accepted as usual for None-able fields:

data_with_none = {"id": 435, "sub_title": None}
patch_book = retort.load(data_with_none, PatchBook)
assert patch_book == PatchBook(
    id=435,
    title=Omitted(),
    sub_title=None,
)
assert retort.dump(patch_book) == data_with_none

But None value is forbidden for fields without None type:

data_with_none = {"id": 435, "title": None}
try:
    patch_book = retort.load(data_with_none, PatchBook)
except AggregateLoadError as e:
    assert len(e.exceptions) == 1
    assert isinstance(e.exceptions[0], TypeLoadError)
    assert e.exceptions[0].expected_type is str

Also, you can create own sentinel types via as_sentinel.

If None is invalid value for every field, you can treat None as sentinel.

from dataclasses import dataclass
from typing import Union

from adaptix import P, Retort, as_sentinel, name_mapping
from adaptix.load_error import AggregateLoadError, TypeLoadError


@dataclass
class PatchBook:
    id: int
    title: str | None = None
    sub_title: str | None = None


retort = Retort(
    recipe=[
        name_mapping(omit_default=True),
        as_sentinel(P[PatchBook][Union][None]),
    ],
)

You can also use NotRequired fields of TypedDict for this, but TypedDict itself is quite limited compared to other models.

from typing import NotRequired, TypedDict

from adaptix import Retort


class PatchBook(TypedDict):
    id: int
    title: NotRequired[str]
    sub_title: NotRequired[str]


retort = Retort()

data = {"id": 435}
patch_book = retort.load(data, PatchBook)
assert patch_book == PatchBook(id=435)
assert retort.dump(patch_book, PatchBook) == data

Name mapping¶

The name mapping mechanism allows precise control outer representation of a model.

It is configured entirely via name_mapping.

The first argument of this function is a predicate, which selects affected classes (see Predicate system for detail). If it is omitted, rules will be applied to all models.

Mutating field name¶

There are several ways to change the name of a field for loading and dumping.

Field renaming¶

Sometimes you have JSON with keys that leave much to be desired. For example, they might be invalid Python identifiers or just have unclear meanings. The simplest way to fix it is to use name_mapping.map to rename it.

from dataclasses import dataclass
from datetime import datetime, timezone

from adaptix import Retort, name_mapping


@dataclass
class Event:
    name: str
    timestamp: datetime


retort = Retort(
    recipe=[
        name_mapping(
            Event,
            map={
                "timestamp": "ts",
            },
        ),
    ],
)

data = {
    "name": "SystemStart",
    "ts": "2023-05-14T00:06:33+00:00",
}
event = retort.load(data, Event)
assert event == Event(
    name="SystemStart",
    timestamp=datetime(2023, 5, 14, 0, 6, 33, tzinfo=timezone.utc),
)
assert retort.dump(event) == data

The keys of map refers to the field name at model definition, and values contain a new field name.

Fields absent in map are not translated and used with their original names.

There are more complex and more powerful use cases of map, which will be described at Advanced mapping.

Name style¶

Sometimes JSON keys are quite normal but do not fit PEP8 recommendations of variable naming. You can rename each field individually, but library can automatically translate such names.

from dataclasses import dataclass

from adaptix import NameStyle, Retort, name_mapping


@dataclass
class Person:
    first_name: str
    last_name: str


retort = Retort(
    recipe=[
        name_mapping(
            Person,
            name_style=NameStyle.CAMEL,
        ),
    ],
)

data = {
    "firstName": "Richard",
    "lastName": "Stallman",
}
event = retort.load(data, Person)
assert event == Person(first_name="Richard", last_name="Stallman")
assert retort.dump(event) == data

See NameStyle for a list of all available target styles.

You cannot convert names that do not follow snake_case style. name_mapping.map takes precedence over name_mapping.name_style, so you can use it to rename fields that do not follow snake_case or override automatic style adjusting.

Stripping underscore¶

Sometimes API uses reserved Python keywords therefore it can not be used as a field name. Usually, it is solved by adding a trailing underscore to the field name (e.g. from_ or import_). Retort trims trailing underscore automatically.

from dataclasses import dataclass

from adaptix import Retort


@dataclass
class Interval:
    from_: int
    to_: int


retort = Retort()

data = {
    "from": 10,
    "to": 20,
}
event = retort.load(data, Interval)
assert event == Interval(from_=10, to_=20)
assert retort.dump(event) == data

If this behavior is unwanted, you can disable this feature by setting trim_trailing_underscore=False

from dataclasses import dataclass

from adaptix import Retort, name_mapping


@dataclass
class Interval:
    from_: int
    to_: int


retort = Retort(
    recipe=[
        name_mapping(
            Interval,
            trim_trailing_underscore=False,
        ),
    ],
)

data = {
    "from_": 10,
    "to_": 20,
}
event = retort.load(data, Interval)
assert event == Interval(from_=10, to_=20)
assert retort.dump(event) == data

name_mapping.map is prioritized over name_mapping.trim_trailing_underscore.

Fields filtering¶

You can select which fields will be loaded or dumped. Two parameters that can be used for these: name_mapping.skip and name_mapping.only

from dataclasses import dataclass

from adaptix import ProviderNotFoundError, Retort, name_mapping


@dataclass
class User:
    id: int
    name: str
    password_hash: str


retort = Retort(
    recipe=[
        name_mapping(
            User,
            skip=["password_hash"],
        ),
    ],
)


user = User(
    id=52,
    name="Ken Thompson",
    password_hash="ZghOT0eRm4U9s",
)
data = {
    "id": 52,
    "name": "Ken Thompson",
}
assert retort.dump(user) == data

try:
    retort.get_loader(User)
except ProviderNotFoundError:
    pass

Excluding the required field makes it impossible to create a loader, but the dumper will work properly.

Both parameters take predicate or iterable of predicates, so you can use all features of Predicate system. For example, you can filter fields based on their type.

from dataclasses import dataclass

from adaptix import Retort, dumper, loader, name_mapping


class HiddenStr(str):
    def __repr__(self):
        return "'<hidden>'"


@dataclass
class User:
    id: int
    name: str
    password_hash: HiddenStr


retort = Retort(
    recipe=[
        loader(HiddenStr, HiddenStr),
        dumper(HiddenStr, str),
    ],
)
skipping_retort = retort.extend(
    recipe=[
        name_mapping(
            User,
            skip=HiddenStr,
        ),
    ],
)

user = User(
    id=52,
    name="Ken Thompson",
    password_hash=HiddenStr("ZghOT0eRm4U9s"),
)
data = {
    "id": 52,
    "name": "Ken Thompson",
}
data_with_password_hash = {
    **data,
    "password_hash": "ZghOT0eRm4U9s",
}
assert repr(user) == "User(id=52, name='Ken Thompson', password_hash='<hidden>')"
assert retort.dump(user) == data_with_password_hash
assert retort.load(data_with_password_hash, User) == user
assert skipping_retort.dump(user) == data

Omit default¶

If you have defaults for some fields, it could be unnecessary to store them in dumped representation. You can omit them when serializing a name_mapping.omit_default parameter. Values that are equal to default, will be stripped from the resulting dict.

from dataclasses import dataclass, field

from adaptix import Retort, name_mapping


@dataclass
class Book:
    title: str
    sub_title: str | None = None
    authors: list[str] = field(default_factory=list)


retort = Retort(
    recipe=[
        name_mapping(
            Book,
            omit_default=True,
        ),
    ],
)

book = Book(title="Fahrenheit 451")
assert retort.dump(book) == {"title": "Fahrenheit 451"}

By default, omit_default is disabled, you can set it to True which will affect all fields. Also, you can pass any predicate or iterable of predicate to apply the rule only to selected fields.

from dataclasses import dataclass, field

from adaptix import Retort, name_mapping


@dataclass
class Book:
    title: str
    sub_title: str | None = None
    authors: list[str] = field(default_factory=list)


retort = Retort(
    recipe=[
        name_mapping(
            Book,
            omit_default="authors",
        ),
    ],
)

book = Book(title="Fahrenheit 451")
assert retort.dump(book) == {"title": "Fahrenheit 451", "sub_title": None}

Unknown fields processing¶

Unknown fields are the keys of mapping that do not map to any known field.

By default, all extra data that is absent in the target structure are ignored. You can change this behavior via name_mapping.extra_in and name_mapping.extra_out parameters.

Field renaming does not affect on unknown fields, collected unknown fields will have original names.

On loading¶

Parameter name_mapping.extra_in controls policy how extra data is saved.

`ExtraSkip`¶

Default behavior. All extra data is ignored.

from dataclasses import dataclass

from adaptix import Retort


@dataclass
class Book:
    title: str
    price: int


data = {
    "title": "Fahrenheit 451",
    "price": 100,
    "unknown1": 1,
    "unknown2": 2,
}

retort = Retort()

book = retort.load(data, Book)
assert book == Book(title="Fahrenheit 451", price=100)

`ExtraForbid`¶

This policy raises load_error.ExtraFieldsLoadError in case of any unknown field is found.

from dataclasses import dataclass

from adaptix import ExtraForbid, Retort, name_mapping
from adaptix.load_error import AggregateLoadError, ExtraFieldsLoadError


@dataclass
class Book:
    title: str
    price: int


data = {
    "title": "Fahrenheit 451",
    "price": 100,
    "unknown1": 1,
    "unknown2": 2,
}

retort = Retort(
    recipe=[
        name_mapping(Book, extra_in=ExtraForbid()),
    ],
)

try:
    retort.load(data, Book)
except AggregateLoadError as e:
    assert len(e.exceptions) == 1
    assert isinstance(e.exceptions[0], ExtraFieldsLoadError)
    assert set(e.exceptions[0].fields) == {"unknown1", "unknown2"}

Non-guaranteed behavior

Order of fields inside load_error.ExtraFieldsLoadError is not guaranteed and can be unstable between runs.

`ExtraKwargs`¶

Extra data are passed as additional keyword arguments.

from adaptix import ExtraKwargs, Retort, name_mapping


class Book:
    def __init__(self, title: str, price: int, **kwargs):
        self.title = title
        self.price = price
        self.kwargs = kwargs

    def __eq__(self, other):
        return (
            self.title == other.title
            and self.price == other.price
            and self.kwargs == other.kwargs
        )


data = {
    "title": "Fahrenheit 451",
    "price": 100,
    "unknown1": 1,
    "unknown2": 2,
}

retort = Retort(
    recipe=[
        name_mapping(Book, extra_in=ExtraKwargs()),
    ],
)

book = retort.load(data, Book)
assert book == Book(title="Fahrenheit 451", price=100, unknown1=1, unknown2=2)

This policy has significant flaws by design and, generally, should not be used.

All extra fields are passed as additional keywords arguments without any conversion, specified type of **kwargs is ignored.

If an unknown field collides with the original field name, TypeError will be raised, treated as an unexpected error.

from adaptix import ExtraKwargs, Retort, name_mapping


class Book:
    def __init__(self, title: str, price: int, **kwargs):
        self.title = title
        self.price = price
        self.kwargs = kwargs

    def __eq__(self, other):
        return (
            self.title == other.title
            and self.price == other.price
            and self.kwargs == other.kwargs
        )


data = {
    "name": "Fahrenheit 451",
    "price": 100,
    "title": "Celsius 232.778",
}

retort = Retort(
    recipe=[
        name_mapping(Book, map={"title": "name"}),
        name_mapping(Book, extra_in=ExtraKwargs()),
    ],
)

try:
    retort.load(data, Book)
except TypeError as e:
    assert str(e).endswith("__init__() got multiple values for argument 'title'")

The following strategy one has no such problems.

Field id¶

You can pass the string with field name. Loader of corresponding field will receive mapping with unknown data.

from collections.abc import Mapping
from dataclasses import dataclass
from typing import Any

from adaptix import Retort, name_mapping


@dataclass
class Book:
    title: str
    price: int
    extra: Mapping[str, Any]


data = {
    "title": "Fahrenheit 451",
    "price": 100,
    "unknown1": 1,
    "unknown2": 2,
}

retort = Retort(
    recipe=[
        name_mapping(Book, extra_in="extra"),
    ],
)

book = retort.load(data, Book)
assert book == Book(
    title="Fahrenheit 451",
    price=100,
    extra={
        "unknown1": 1,
        "unknown2": 2,
    },
)

Also you can pass Iterable[str]. Each field loader will receive same mapping of unknown data.

Saturator function¶

There is a way to use a custom mechanism of unknown field saving.

You can pass a callable taking created model and mapping of unknown data named ‘saturator’. Precise type hint is Callable[[T, Mapping[str, Any]], None]. This callable can mutate the model to inject unknown data as you want.

from collections.abc import Mapping
from dataclasses import dataclass
from typing import Any

from adaptix import Retort, name_mapping


@dataclass
class Book:
    title: str
    price: int


data = {
    "title": "Fahrenheit 451",
    "price": 100,
    "unknown1": 1,
    "unknown2": 2,
}


def attr_saturator(model: Book, extra_data: Mapping[str, Any]) -> None:
    for key, value in extra_data.items():
        setattr(model, key, value)


retort = Retort(
    recipe=[
        name_mapping(Book, extra_in=attr_saturator),
    ],
)

book = retort.load(data, Book)
assert book == Book(title="Fahrenheit 451", price=100)
assert book.unknown1 == 1  # type: ignore[attr-defined]
assert book.unknown2 == 2  # type: ignore[attr-defined]

On dumping¶

Parameter name_mapping.extra_in controls policy how extra data is extracted.

`ExtraSkip`¶

Default behavior. All extra data is ignored.

from collections.abc import Mapping
from dataclasses import dataclass
from typing import Any

from adaptix import Retort, name_mapping


@dataclass
class Book:
    title: str
    price: int
    extra: Mapping[str, Any]


data = {
    "title": "Fahrenheit 451",
    "price": 100,
    "unknown1": 1,
    "unknown2": 2,
}

retort = Retort(
    recipe=[
        name_mapping(Book, extra_in="extra"),
    ],
)

book = retort.load(data, Book)
assert book == Book(
    title="Fahrenheit 451",
    price=100,
    extra={
        "unknown1": 1,
        "unknown2": 2,
    },
)
assert retort.dump(book) == {
    "title": "Fahrenheit 451",
    "price": 100,
    "extra": {  # `extra` is treated as common field
        "unknown1": 1,
        "unknown2": 2,
    },
}

You can skip extra from dumping. See Fields filtering for detail.

Field id¶

You can pass the string with field name. Dumper of this field must return a mapping that will be merged with dict of dumped representation.

from collections.abc import Mapping
from dataclasses import dataclass
from typing import Any

from adaptix import Retort, name_mapping


@dataclass
class Book:
    title: str
    price: int
    extra: Mapping[str, Any]


data = {
    "title": "Fahrenheit 451",
    "price": 100,
    "unknown1": 1,
    "unknown2": 2,
}

retort = Retort(
    recipe=[
        name_mapping(Book, extra_in="extra", extra_out="extra"),
    ],
)

book = retort.load(data, Book)
assert book == Book(
    title="Fahrenheit 451",
    price=100,
    extra={
        "unknown1": 1,
        "unknown2": 2,
    },
)
assert retort.dump(book) == data

Non-guaranteed behavior

Output mapping keys have not collide with keys of dumped model. Otherwise the result is not guaranteed.

You can pass several field ids (Iterable[str]). The output mapping will be merged.

from collections.abc import Mapping
from dataclasses import dataclass
from typing import Any

from adaptix import Retort, name_mapping


@dataclass
class Book:
    title: str
    price: int
    extra1: Mapping[str, Any]
    extra2: Mapping[str, Any]


retort = Retort(
    recipe=[
        name_mapping(Book, extra_out=["extra1", "extra2"]),
    ],
)

book = Book(
    title="Fahrenheit 451",
    price=100,
    extra1={
        "unknown1": 1,
        "unknown2": 2,
    },
    extra2={
        "unknown3": 3,
        "unknown4": 4,
    },
)
assert retort.dump(book) == {
    "title": "Fahrenheit 451",
    "price": 100,
    "unknown1": 1,
    "unknown2": 2,
    "unknown3": 3,
    "unknown4": 4,
}

Non-guaranteed behavior

Priority of output mapping is not guaranteed.

Extractor function¶

There is way to take out extra data from via custom function called ‘extractor’. A callable must taking model and produce mapping of extra fields. Precise type hint is Callable[[T], Mapping[str, Any]].

import dataclasses
from collections.abc import Mapping
from dataclasses import dataclass
from typing import Any

from adaptix import Retort, name_mapping


@dataclass
class Book:
    title: str
    price: int


data = {
    "title": "Fahrenheit 451",
    "price": 100,
    "unknown1": 1,
    "unknown2": 2,
}


def attr_saturator(model: Book, extra_data: Mapping[str, Any]) -> None:
    for key, value in extra_data.items():
        setattr(model, key, value)


book_fields = {fld.name for fld in dataclasses.fields(Book)}


def attr_extractor(model: Book) -> Mapping[str, Any]:
    return {
        key: value
        for key, value in vars(model).items()
        if key not in book_fields
    }


retort = Retort(
    recipe=[
        name_mapping(Book, extra_in=attr_saturator, extra_out=attr_extractor),
    ],
)

book = retort.load(data, Book)
assert retort.dump(book) == data

Non-guaranteed behavior

Output mapping keys have not collide with keys of dumped model. Otherwise the result is not guaranteed.

Mapping to list¶

Some APIs store structures as lists or arrays rather than dict for optimization purposes. For example, Binance uses it to represent historical market data.

There is name_mapping.as_list that converts the model to a list. Position at the list is determined by order of field definition.

from dataclasses import dataclass
from datetime import datetime, timezone

from adaptix import Retort, name_mapping


@dataclass
class Action:
    user_id: int
    kind: str
    timestamp: datetime


retort = Retort(
    recipe=[
        name_mapping(
            Action,
            as_list=True,
        ),
    ],
)


action = Action(
    user_id=23,
    kind="click",
    timestamp=datetime(2023, 5, 20, 15, 58, 23, 410366, tzinfo=timezone.utc),
)
data = [
    23,
    "click",
    "2023-05-20T15:58:23.410366+00:00",
]
assert retort.dump(action) == data
assert retort.load(data, Action) == action

You can override the order of fields using name_mapping.map parameter.

from dataclasses import dataclass
from datetime import datetime, timezone

from adaptix import Retort, name_mapping


@dataclass
class Action:
    user_id: int
    kind: str
    timestamp: datetime


retort = Retort(
    recipe=[
        name_mapping(
            Action,
            map={
                "user_id": 1,
                "kind": 0,
            },
            as_list=True,
        ),
    ],
)


action = Action(
    user_id=23,
    kind="click",
    timestamp=datetime(2023, 5, 20, 15, 58, 23, 410366, tzinfo=timezone.utc),
)
data = [
    "click",
    23,
    "2023-05-20T15:58:23.410366+00:00",
]
assert retort.dump(action) == data
assert retort.load(data, Action) == action

Also, you can map the model to list via name_mapping.map without using name_mapping.as_list, if you assign every field to their position on the list.

Only ExtraSkip and ExtraForbid is could be used with mapping to list.

Structure flattening¶

Too complex hierarchy of structures in API could be fixed via map parameter. Earlier, you used it to rename fields, but also you can use it to map a name to a nested value by specifying a path to it. Integers in the path are treated as list indices, strings - as dict keys.

from dataclasses import dataclass

from adaptix import Retort, name_mapping


@dataclass
class Book:
    title: str
    price: int
    author: str


retort = Retort(
    recipe=[
        name_mapping(
            Book,
            map={
                "author": ["author", "name"],
                "title": ["book", "title"],
                "price": ["book", "price"],
            },
        ),
    ],
)

data = {
    "book": {
        "title": "Fahrenheit 451",
        "price": 100,
    },
    "author": {
        "name": "Ray Bradbury",
    },
}
book = retort.load(data, Book)
assert book == Book(
    title="Fahrenheit 451",
    price=100,
    author="Ray Bradbury",
)
assert retort.dump(book) == data

This snippet could be reduced.

Ellipsis (...) inside path is replaced by original field name after automatic conversions.
Dict could be replaced with a list of pairs. The first item of the pair is predicate (see Predicate system for detail), the second is the mapping result (path in this case).

from dataclasses import dataclass

from adaptix import Retort, name_mapping


@dataclass
class Book:
    title: str
    price: int
    author: str


retort = Retort(
    recipe=[
        name_mapping(
            Book,
            map=[
                ("author", (..., "name")),
                ("title|price", ("book", ...)),
            ],
        ),
    ],
)

data = {
    "book": {
        "title": "Fahrenheit 451",
        "price": 100,
    },
    "author": {
        "name": "Ray Bradbury",
    },
}
book = retort.load(data, Book)
assert book == Book(
    title="Fahrenheit 451",
    price=100,
    author="Ray Bradbury",
)
assert retort.dump(book) == data

Chaining (partial overriding)¶

Result name_mapping is computed by merging all parameters of matched name_mapping.

from dataclasses import dataclass
from typing import Any

from adaptix import NameStyle, Retort, name_mapping


@dataclass
class Person:
    first_name: str
    last_name: str
    extra: dict[str, Any]


@dataclass
class Book:
    title: str
    author: Person


retort = Retort(
    recipe=[
        name_mapping(Person, name_style=NameStyle.CAMEL),
        name_mapping("author", extra_in="extra", extra_out="extra"),
    ],
)

data = {
    "title": "Lord of Light",
    "author": {
        "firstName": "Roger",
        "lastName": "Zelazny",
        "unknown_field": 1995,
    },
}
book = retort.load(data, Book)
assert book == Book(
    title="Lord of Light",
    author=Person(
        first_name="Roger",
        last_name="Zelazny",
        extra={"unknown_field": 1995},
    ),
)
assert retort.dump(book) == data

The first provider override parameters of next providers.

from dataclasses import dataclass
from typing import Any

from adaptix import NameStyle, Retort, name_mapping


@dataclass
class Person:
    first_name: str
    last_name: str
    extra: dict[str, Any]


@dataclass
class Book:
    title: str
    author: Person


retort = Retort(
    recipe=[
        name_mapping(Person, name_style=NameStyle.UPPER_SNAKE),
        name_mapping(Person, name_style=NameStyle.CAMEL),
        name_mapping("author", extra_in="extra", extra_out="extra"),
    ],
)

data = {
    "title": "Lord of Light",
    "author": {
        "FIRST_NAME": "Roger",
        "LAST_NAME": "Zelazny",
        "UNKNOWN_FIELD": 1995,
    },
}
book = retort.load(data, Book)
assert book == Book(
    title="Lord of Light",
    author=Person(
        first_name="Roger",
        last_name="Zelazny",
        extra={"UNKNOWN_FIELD": 1995},
    ),
)
assert retort.dump(book) == data

Private fields dumping¶

By default, adaptix skips private fields (any field starting with underscore) at dumping.

from adaptix import Retort
from pydantic import BaseModel


class Book(BaseModel):
    title: str
    price: int
    _private: int

    def __init__(self, **kwargs):
        super().__init__(**kwargs)
        self._private = 1


retort = Retort()
book = Book(title="Fahrenheit 451", price=100)
assert retort.dump(book) == {
    "title": "Fahrenheit 451",
    "price": 100,
}

You can include this fields by setting alias.

from adaptix import Retort, name_mapping
from pydantic import BaseModel


class Book(BaseModel):
    title: str
    price: int
    _private: int

    def __init__(self, **kwargs):
        super().__init__(**kwargs)
        self._private = 1


retort = Retort(
    recipe=[
        name_mapping(Book, map={"_private": "private_field"}),
    ],
)
book = Book(title="Fahrenheit 451", price=100)
assert retort.dump(book) == {
    "title": "Fahrenheit 451",
    "price": 100,
    "private_field": 1,
}

Alias can be equal to field name (field id) and field will be included.

Advanced mapping¶

Let’s figure it out with all features of name_mapping.map.

name_mapping.map can take data in two forms:

collections.abc.Mapping with keys of field ids and values with mapping result
Iterable of pairs (tuple of two elements) or providers or mapping described above. Provider interface for mapping currently is unstable and would not be described at this article. If you pass a tuple of two elements, the first item must be predicate (see Predicate system for detail), and the second item must be mapping result or function returning mapping result.

If you use mapping all keys must be field_id (e.g. valid python identifiers), so regexes like a|b is not allowed.

The mapping result is union of 5 types:

String of external field name
Integer indicating index inside output sequence
Ellipsis (...) that will be replaced with the key after builtin conversions by name_mapping.trim_trailing_underscore, name_mapping.name_style and name_mapping.as_list.
Iterable of string, integer or ellipsis, aka Structure flattening
None that means skipped field. name_mapping.map is applied after name_mapping.only. So the field will be skipped despite the match by name_mapping.only.

Name mapping reuses concepts of recipe inside retort and also implements chain-of-responsibility design pattern.

Only the first element matched by its predicate is used to determine the mapping result.

The callable producing mapping result must take two parameters: the shape of the model and the field. Types of these parameters currently are internal. You can find an exact definition in the source code but it could change in the future.

Example of using advanced techniques:

import re
from collections.abc import Iterable, Sequence
from dataclasses import dataclass

from adaptix import P, Retort, name_mapping


@dataclass
class Document:
    key: str

    redirects: list[str]
    edition_keys: list[str]
    lcc_list: list[str]


def create_plural_stripper(
    *,
    exclude: Sequence[str] = (),
    suffixes: Iterable[str] = ("s", "_list"),
):
    pattern = "^(.*)(" + "|".join(suffixes) + ")$"

    def plural_stripper(shape, fld):
        return re.sub(pattern, lambda m: m[1], fld.id)

    return (
        P[pattern] & ~P[tuple(exclude)],
        plural_stripper,
    )


retort = Retort(
    recipe=[
        name_mapping(
            Document,
            map=[
                {"key": "name"},
                create_plural_stripper(exclude=["redirects"]),
            ],
        ),
    ],
)
data = {
    "name": "The Lord of the Rings",
    "redirects": ["1234"],
    "edition_key": ["423", "4235"],
    "lcc": ["675", "345"],
}
document = retort.load(data, Document)
assert document == Document(
    key="The Lord of the Rings",
    redirects=["1234"],
    edition_keys=["423", "4235"],
    lcc_list=["675", "345"],
)
assert retort.dump(document) == data

Some XML APIs or APIs derived from XML do not use plural forms for repeated fields. So you need to strip the plural form at external representation.

The first item of name_mapping.map is dict that renames individual field. The second item is a tuple created by a function. The function constructs appropriate regex to match fields and trim plural suffixes.

The merging of map is different from other parameters. A new map does not replace others. The new iterable is concatenated to the previous.

Extended usage¶

Generic classes¶

Recursive data types¶

Dealing with if TYPE_CHECKING¶

Detecting absence of a field¶

Name mapping¶

Mutating field name¶

Field renaming¶

Name style¶

Stripping underscore¶

Fields filtering¶

Omit default¶

Unknown fields processing¶

On loading¶

ExtraSkip¶

ExtraForbid¶

ExtraKwargs¶

Field id¶

Saturator function¶

On dumping¶

ExtraSkip¶

Field id¶

Extractor function¶

Mapping to list¶

Structure flattening¶

Chaining (partial overriding)¶

Private fields dumping¶

Advanced mapping¶

Dealing with `if TYPE_CHECKING`¶

`ExtraSkip`¶

`ExtraForbid`¶

`ExtraKwargs`¶

`ExtraSkip`¶