enhance: add supposer for zstd skippable frames #39
Conversation
| let magic = u32::from_le_bytes(buf[0..4].try_into().unwrap()) as usize; | ||
| if magic & ZSTD_SKIP_MASK == ZSTD_SKIP_START { | ||
| let data_len = u32::from_le_bytes(buf[4..8].try_into().unwrap()) as usize; |
There was a problem hiding this comment.
Converting it into a u32 and then casting it to usize feels very fragile to me, and would definitely break on 16 bit systems (which I have zero knowledge in, but infer could probably run perfectly on them otherwise). It also adds the ambiguity of what happens on big endian platforms. I think we should compare slices directly.
There was a problem hiding this comment.
👋 Sorry been distracted with a lot of other commitments.
It's been a while since I looked at this (and worked with Rust) so my context might be rusty, and I might be missing something, but wanted to try and return to this slowly...
According to the docs the magic number is 4 bytes and little endian, the frame size is 4 bytes, little-endian format, unsigned 32-bits as well. So it would seem this is the required data layout regardless of the underlying architecture, and we need to be able to load and inspect the size as 32 bit unsigned int in order to fully support the possible data size? So perhaps this should be behind feature flag / conditional compilation? Are there other work arounds?
There was a problem hiding this comment.
The solution would be to size-check when converting u32->usize.
impl TryFrom<u32> for usize does exactly what you want, it never fails on 32 and 64-bit machines, but might fail on 16-bit ones.
I see that you've been busy (btw, ghz is pretty cool!) and this PR was hanging for long, so I re-made the PR with the fixes needed, I hope you don't mind.
|
@bojand I believe you can close this now |
See h2non/filetype#100