Skip to content

Add struct class-level options #39

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
jcrist merged 6 commits into from
Jul 3, 2021
Merged

Add struct class-level options #39

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
6 commits
Select commit Hold shift + click to select a range
f18370f
Add initial support for struct class-level options
jcrist Jul 3, 2021
87d6613
Implement `immutable` option
jcrist Jul 3, 2021
c87d540
Add `asarray` option for encoding structs as arrays
jcrist Jul 3, 2021
a3212c4
Support decoding structs from arrays
jcrist Jul 3, 2021
e5eea16
Add tests for typed struct asarray decoding
jcrist Jul 3, 2021
a82f6bb
Add docs
jcrist Jul 3, 2021
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
64 changes: 54 additions & 10 deletions docs/source/index.rst
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -202,19 +202,22 @@ acceptible:
- `list` / `typing.List`
- `dict` / `typing.Dict`
- `set` / `typing.Set`
- `datetime.datetime`
- `typing.Any`
- `typing.Optional`
- `msgspec.Ext`
- `enum.Enum` derived types
- `enum.IntEnum` derived types
- `msgspec.Struct` derived types
- Custom types (provided a valid ``ext_hook`` or ``dec_hook`` callback on the Decoder)


Structs
~~~~~~~

``msgspec`` can serialize many builtin types, but unlike protocols like
`pickle`_/`quickle`_, it can't serialize arbitrary user classes. Two
user-defined types are supported:
`pickle`_/`quickle`_, it can't serialize arbitrary user classes by default. Two
user-defined types are supported though:

- `Struct`
- `enum.Enum`
Expand Down Expand Up @@ -254,11 +257,31 @@ annotations:
>>> ron == harry
False

It is forbidden to override ``__init__``/``__new__`` in a struct definition,
but other methods can be overridden or added as needed. The struct fields are
available via the ``__struct_fields__`` attribute (a tuple of the fields in
argument order ) if you need them. Here we add a method for converting a struct
to a dict.
If needed, a ``__hash__`` method can also be generated by specifying
``immutable=True`` when defining the struct. Note that this disables modifying
field values after initialization.

.. code-block:: python

>>> class Point(msgspec.Struct, immutable=True):
... """This struct is immutable & hashable"""
... x: float
... y: float
...
>>> p = Point(1.0, 2.0)
>>> {p: 1} # immutable structs are hashable, and can be keys in dicts
{Point(1.0, 2.0): 1}
>>> p.x = 2.0 # immutable structs cannot be modified after creation
Traceback (most recent call last):
...
TypeError: immutable type: 'Point'

Note that it is forbidden to override ``__init__``/``__new__`` in a struct
definition, but other methods can be overridden or added as needed.

The struct fields are available via the ``__struct_fields__`` attribute (a
tuple of the fields in argument order ) if you need them. Here we add a method
for converting a struct to a dict.

.. code-block:: python

Expand Down Expand Up @@ -295,6 +318,25 @@ deserialization, it also can improve performance. Depending on the schema,
deserializing a message into a `Struct` can be *roughly twice as fast* as
deserializing it into a `dict`.

If you need higher performance (at the cost of more inscrutable message
encoding), you can set ``asarray=True`` on a struct definition. Structs with
this option enabled are encoded/decoded as MessagePack ``array`` types (rather
than ``map`` types), removing the field names from the serialized message. This
can provide another ~2x speedup for decoding (and ~1.5x speedup for encoding).

.. code-block:: python

>>> class ArrayBasedStruct(msgspec.Struct, asarray=True):
... """This struct is serialized as a MessagePack array type
... (instead of a map type). This means no field names are sent
... as part of the message, speeding up encoding/decoding."""
... my_first_field: str
... my_second_field: int
...
>>> x = ArrayBasedStruct("some string", 2)
>>> msgspec.encode(x)
b'\x92\xabsome string\x02'

.. _extensions:

Extensions
Expand Down Expand Up @@ -436,9 +478,11 @@ mismatched versions.
For schema evolution to work smoothly, you need to follow a few guidelines:

1. Any new fields on a `Struct` must specify default values.
2. Don't change the type annotations for existing messages or fields
3. Don't change the type codes or implementations for any defined
:ref:`Extensions`
2. Structs with ``asarray=True`` must not reorder fields, and any new fields
must be appended to the end (and have defaults).
3. Don't change the type annotations for existing messages or fields.
4. Don't change the type codes or implementations for any defined
:ref:`Extensions`.

For example, suppose we wanted to add a new ``email`` field to our ``Person``
struct. To do so, we add it at the end of the definition, with a default value.
Expand Down
Loading