Skip to content

Use dict str cache for msgpack decoder - #121

Merged
jcrist merged 1 commit into
mainfrom
msgpack-str-cache
Jun 17, 2022
Merged

Use dict str cache for msgpack decoder#121
jcrist merged 1 commit into
mainfrom
msgpack-str-cache

Conversation

@jcrist

@jcrist jcrist commented Jun 17, 2022

Copy link
Copy Markdown
Member

Previously we added a cache for short ascii string keys when decoding
dicts using JSON. We now expand this optimization to the msgpack decoder
as well.

Previously we added a cache for short ascii string keys when decoding
dicts using JSON. We now expand this optimization to the msgpack decoder
as well.
@jcrist

jcrist commented Jun 17, 2022

Copy link
Copy Markdown
Member Author

With this change, the standard twitter.json dataset loads ~30% faster when using the untyped decoder. We still recommend using Struct types when data has a known schema, but this optimization dramatically speeds up decoding when loading structured data (dicts have a common, repeated structure) but no schema is provided.

@jcrist
jcrist merged commit a941785 into main Jun 17, 2022
@jcrist
jcrist deleted the msgpack-str-cache branch June 17, 2022 19:38
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant