Skip to content

Commit

Permalink
WIP: S2++
Browse files Browse the repository at this point in the history
If the first bytes of a block is `0x40, 0x00` (repeat, length 4), this indicates that all [Copy with 4-byte offset (11)](https://github.com/google/snappy/blob/main/format_description.txt#L106) are all 3 bytes instead for the remainder of the block.

There can be no literals before this tag and no repeats before a match as specified above.
This will only trigger on this exact tag.

> These are like the copies with 2-byte offsets (see previous subsection),
> except that the offset is stored as a 24-bit integer instead of a
> 16-bit integer (and thus will occupy three bytes).

When in this mode the maximum backreference offset is 16777215.

This *cannot* be combined with dictionaries.
  • Loading branch information
klauspost committed Aug 7, 2023
1 parent c1dcc38 commit e1bd1af
Showing 1 changed file with 15 additions and 0 deletions.
15 changes: 15 additions & 0 deletions s2/README.md
Expand Up @@ -1047,6 +1047,21 @@ The first copy of a block cannot be a repeat offset and the offset is reset on e

Default streaming block size is 1MB.

## S2++

If the first bytes of a block is `0x40, 0x00` (repeat, length 4), this indicates that all [Copy with 4-byte offset (11)](https://github.com/google/snappy/blob/main/format_description.txt#L106) are all 3 bytes instead for the remainder of the block.

There can be no literals before this tag and no repeats before a match as specified above.
This will only trigger on this exact tag.

> These are like the copies with 2-byte offsets (see previous subsection),
> except that the offset is stored as a 24-bit integer instead of a
> 16-bit integer (and thus will occupy three bytes).
When in this mode the maximum backreference offset is 16777215.

This *cannot* be combined with dictionaries.

# Dictionary Encoding

Adding dictionaries allow providing a custom dictionary that will serve as lookup in the beginning of blocks.
Expand Down

0 comments on commit e1bd1af

Please sign in to comment.