Change Footer version handling, Make compression dynamic #1060
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Change Footer version handling Simplify version handling by switching to JSON instead of binary serialization. fixes quickwit-oss#1058 Make compression dynamic Instead of choosing the compression during compile time via a feature flag, you can now have multiple compression algorithms enabled and decide during runtime which one to choose via IndexSettings. Changing the compression algorithm on an index is also supported. The information which algorithm was used in the doc store is stored in the DocStoreFooter. The default is the lz4 block format. fixes quickwit-oss#904
Handle merging of different compressors Fix feature flag names Add doc store test for all compressors
fulmicoton
reviewed
May 25, 2021
fulmicoton
reviewed
May 25, 2021
fulmicoton
reviewed
May 25, 2021
fulmicoton
reviewed
May 25, 2021
fulmicoton
reviewed
May 25, 2021
fulmicoton
reviewed
May 25, 2021
fulmicoton
reviewed
May 25, 2021
fulmicoton
reviewed
May 25, 2021
fulmicoton
requested changes
May 25, 2021
|
Should be fine now, except removing the lz-frame stuff Edit also removed the lz4 frame stuff |
use docstore_compression default fallback in order to parse old versions of meta.json
rename lz4-block-compression to lz4-compression
fulmicoton
reviewed
May 27, 2021
fulmicoton
reviewed
May 27, 2021
fulmicoton
reviewed
May 27, 2021
fulmicoton
requested changes
May 27, 2021
This was referenced Feb 18, 2022
Closed
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Change Footer version handling
Simplify version handling by switching to JSON instead of binary serialization.
fixes #1058
Make compression dynamic
Instead of choosing the compression during compile time via a feature flag, you can now have multiple compression algorithms enabled and decide during runtime which one to choose via IndexSettings. Changing the compression algorithm on an index is also supported. The information which algorithm was used in the doc store is stored in the DocStoreFooter. The default is the lz4 block format.
fixes #904