Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Enabling zero copy deserialization (e.g. splitting a BytesMut based on on a &[u8] owned by that BytesMut). #485

Open
hgomersall opened this issue Feb 19, 2021 · 0 comments

Comments

@hgomersall
Copy link

Some serde Deserializers can do zero-copy &[u8] -> &[u8] conversion. This is obviously quite useful.

Unfortunately, there seems currently not to be a way in which a deserializer can zero copy from/into a BytesMut (for obvious reasons - it might be possible with specialization, but I haven't thought to carefully about it...).

The problem then is that you lose the benefits of having your data encapsulated in a BytesMut object, like passing to a downstream async block for further manipulation.

It's possible to split a BytesMut based on an extracted zero-copied slice fairly safely as follows:

// buf is the source `BytesMut`
// buf_slice is a slice from it (returned from e.g. the deserializer)
let size = buf_slice.len();
// The unsafe is just because `offset_from` is unsafe (mostly fairly benignly)
let offset = unsafe {
    buf_slice.as_ptr().offset_from(buf.as_ptr()) as usize
};
let mut buf = buf.split_off(offset);
let _ = buf.split_off(size);

Link to rust playground.

It would be good if this was somehow better supported. It might not be possible easily, but I thought it worth raising for discussion since it seems to be a missing piece.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant