Get CRC Hash from S2 Header #782
-
|
Hi, Would it be possible to add a method to extract the CRC hash from the S2 header of an S2 compressed block? Thanks. |
Beta Was this translation helpful? Give feedback.
Replies: 1 comment 1 reply
-
|
For S2 (and Snappy) a block doesn't have any checksums or content validation. On a Stream, each block is written with a checksum, that is a CRC32C, that is shifted: https://github.com/klauspost/compress/blob/master/s2/s2.go#L112-L117 This means that checksums here only represents each block, and you cannot get the total stream CRC. If you just want a checksum and not a specific one, you could read read the stream and combine the checksums. The framing format is the same as Snappy framing format, except S2 has a different stream identifier. |
Beta Was this translation helpful? Give feedback.
-
|
That helps, thanks for the explanation. |
Beta Was this translation helpful? Give feedback.
For S2 (and Snappy) a block doesn't have any checksums or content validation.
On a Stream, each block is written with a checksum, that is a CRC32C, that is shifted: https://github.com/klauspost/compress/blob/master/s2/s2.go#L112-L117
This means that checksums here only represents each block, and you cannot get the total stream CRC.
If you just want a checksum and not a specific one, you could read read the stream and combine the checksums.
The framing format is the same as Snappy framing format, except S2 has a different stream identifier.