Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: decode_in_slice() #83

Open
wants to merge 1 commit into
base: main
Choose a base branch
from

Conversation

jarkkojs
Copy link

@jarkkojs jarkkojs commented Dec 2, 2023

For memory constrained enviroments and network protocols being able to use memory without extra copies is a benefit. E.g. in my ZMODEM2 crate I need to do this for hex messages:

if encoding == Encoding::ZHEX {
    let mut tmp = [0; (FrameHeader::unescaped_size(Encoding::ZHEX) - 1) / 2];
    match hex::decode_to_slice(out, &mut tmp) {
        Ok(x) => x,
        _ => {
            log::error!("from_hex error");
            return Err(ErrorKind::InvalidData.into());
        }
    }
    out.clear();
    out.extend_from_slice(&tmp);
}

It is good example since the protocol frames can come also in binary format in addition to hex. It would be nice to do this preprocessing within the same buffer as where the data landed to make a transparent zerocopy flow.

Link: https://github.com/jarkkojs/zmodem2/

For memory constrained enviroments and network protocols being able to use
memory without extra copies is a benefit. E.g. in my ZMODEM2 crate I need
to do this for hex messages:

```
if encoding == Encoding::ZHEX {
    let mut tmp = [0; (FrameHeader::unescaped_size(Encoding::ZHEX) - 1) / 2];
    match hex::decode_to_slice(out, &mut tmp) {
        Ok(x) => x,
        _ => {
            log::error!("from_hex error");
            return Err(ErrorKind::InvalidData.into());
        }
    }
    out.clear();
    out.extend_from_slice(&tmp);
}
```

It is good example since the protocol frames can come also in binary format
in addition to hex. It would be nice to do this preprocessing within the
same buffer as where the data landed to make a transparent zerocopy flow.

Link: https://github.com/jarkkojs/zmodem2/
Signed-off-by: Jarkko Sakkinen <jarkko.sakkinen@iki.fi>
@jarkkojs
Copy link
Author

jarkkojs commented Dec 2, 2023

Asymmetry is actually cool here. There's much less use for encode counter-part as you don't end up doing extra copies with current functions, as you build your buffer from zero. So I'd consider this without encode_in_slice. It is also less nice in the sense that it needs more space than the original.

@jarkkojs
Copy link
Author

jarkkojs commented Dec 2, 2023

I adapted it to my zmodem2 crate:

        if encoding == Encoding::ZHEX {
            hex::decode_in_slice(&mut out).or::<io::Error>(Err(ErrorKind::InvalidData.into()))?;
            out.truncate(out.len() / 2);
        }

I'd like to make this scale from microcontroller to xeon so thus I'm proactively interested on saving in this case 7 byte of stack space.

Relative cost of the copy is +50%. For the record, I use tinyvec by Google for the stack based vector (just for the context of the example).

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants