Skip to content

dataclass support - #218

Merged
jcrist merged 7 commits into
mainfrom
dataclasses
Dec 2, 2022
Merged

dataclass support#218
jcrist merged 7 commits into
mainfrom
dataclasses

Conversation

@jcrist

@jcrist jcrist commented Dec 1, 2022

Copy link
Copy Markdown
Member

This PR adds encoding/decoding support for dataclasses, mirroring our support for typing.NamedTuple/collections.namedtuple/typing.TypedDict. Dataclasses are encoded to/decoded from mappings containing all attributes assigned to the object (union of __dict__ and __slots__).

I do not intend to add support for additional configuration (e.g. renaming fields using camelCase) - if the user wants more customization they should switch to using structs. Structs support all the common features of dataclasses, but are more performant to create/use/encode/decode, and are more customizable. The intent of adding dataclass support is:

  • Compatibility with what orjson is capable of encoding efficiently. Initial benchmarks show the implementation here encodes measurably faster than the one in orjson, especially if slots=True is used, see below.
  • More general support for common python object types.

Todo:

  • JSON encoding support
  • MessagePack encoding support
  • JSON decoding support
  • MessagePack decoding support
  • Docs

Fixes #57.

@jcrist

jcrist commented Dec 1, 2022

Copy link
Copy Markdown
Member Author

A simple encoding benchmark:

In [1]: import msgspec, orjson

In [2]: from dataclasses import dataclass

In [3]: enc = msgspec.json.Encoder()

In [4]: @dataclass
   ...: class NoSlots:
   ...:     field_one: int
   ...:     field_two: int
   ...: 

In [5]: @dataclass(slots=True)
   ...: class Slots:
   ...:     field_one: int
   ...:     field_two: int
   ...: 

In [6]: no_slots = [NoSlots(i - 1, i + 1) for i in range(10000)]

In [7]: with_slots = [Slots(i - 1, i + 1) for i in range(10000)]

In [8]: %timeit enc.encode(no_slots)  # msgspec, no slots
561 µs ± 2.04 µs per loop (mean ± std. dev. of 7 runs, 1,000 loops each)

In [9]: %timeit orjson.dumps(no_slots)  # orjson, no slots
834 µs ± 1.69 µs per loop (mean ± std. dev. of 7 runs, 1,000 loops each)

In [10]: %timeit enc.encode(with_slots)  # msgspec, with slots
779 µs ± 20 µs per loop (mean ± std. dev. of 7 runs, 1,000 loops each)

In [11]: %timeit orjson.dumps(with_slots)  # orjson, with slots
3.71 ms ± 90.1 µs per loop (mean ± std. dev. of 7 runs, 100 loops each)

In [12]: class Struct(msgspec.Struct):
    ...:     field_one: int
    ...:     field_two: int
    ...: 

In [13]: structs = [Struct(i - 1, i + 1) for i in range(10000)]

In [14]: %timeit enc.encode(structs)  # msgspec structs
356 µs ± 307 ns per loop (mean ± std. dev. of 7 runs, 1,000 loops each)

For these type definitions on my machine:

  • msgspec encodes dataclasses to JSON 1.5x faster than orjson
  • msgspec encodes dataclasses with slots=True to JSON 5x faster than orjson
  • dataclasses with slots=True are slower to encode than slots=False. This has to do with object layouts and what information is efficiently accessible on the type definition.
  • msgspec encodes Struct types 1.5x faster than dataclasses

@jcrist jcrist changed the title [WIP] dataclass support dataclass support Dec 2, 2022
@jcrist
jcrist merged commit c3b5263 into main Dec 2, 2022
@jcrist
jcrist deleted the dataclasses branch December 2, 2022 02:03
@jcrist jcrist mentioned this pull request Dec 2, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Support for dataclasses

1 participant