Encode UUID as FixedLenByteArray in parquet_derive #5773
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
conradludgate
force-pushed
the
fix-uuid-derive
branch
from
73e5dc1
to
8de9de7
Compare
tustvold
changed the title
tustvold
changed the title
tustvold
reviewed
May 16, 2024
5 tasks
tustvold
reviewed
May 16, 2024
|
Thank you |
conradludgate
added a commit
to neondatabase/neon
that referenced
this pull request
May 19, 2024
## Summary of changes Updates the parquet lib. one change left that we need is in an open PR against upstream, hopefully we can remove the git dependency by 52.0.0 apache/arrow-rs#5773 I'm not sure why the parquet files got a little bit bigger. I tested them and they still open fine. 🤷 side effect of the update, chrono updated and added yet another deprecation warning (hence why the safekeepers change)
a-masterov
pushed a commit
to neondatabase/neon
that referenced
this pull request
May 20, 2024
## Summary of changes Updates the parquet lib. one change left that we need is in an open PR against upstream, hopefully we can remove the git dependency by 52.0.0 apache/arrow-rs#5773 I'm not sure why the parquet files got a little bit bigger. I tested them and they still open fine. 🤷 side effect of the update, chrono updated and added yet another deprecation warning (hence why the safekeepers change)
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Which issue does this PR close?
Closes #5254.
Rationale for this change
UUIDs are specified to be FixedByteArray(16), not ByteArray.
What changes are included in this PR?
Adds support for defining type builders with a length field. Fixes mapping for uuid to encoded type.
Are there any user-facing changes?
This is likely a breaking change as it is no longer compatible with prior parquet encodings generated with this lib.