Decoding out of image::io::Reader directly into user-controlled buffer

[jonhoo]

I would like to be able to decode from an io::Reader into a buffer I control.

My specific use case for this functionality is to avoid an allocation + copy on the critical path of a very hot loop that has to extract the raw image bytes from an image elsewhere in memory.

This is more generally applicable to anyone who may want to eliminate an extra copy in their applications.

Draft

I think most of the functionality already exists, it's just private in the form of free_functions::load_inner:

https://github.com/image-rs/image/blob/2b513ae9a6ac306e752914f562e7d408f096ba3f/src/io/free_functions.rs#L900-L111

plus a variant of decoder_to_vec:

image/src/image.rs

Lines 708 to 722 in 2b513ae

	pub(crate) fn decoder_to_vec<'a, T>(decoder: impl ImageDecoder<'a>) -> ImageResult<Vec<T>>
	where
	T: crate::traits::Primitive + bytemuck::Pod,
	{
	let total_bytes = usize::try_from(decoder.total_bytes());
	if total_bytes.is_err() || total_bytes.unwrap() > isize::max_value() as usize {
	return Err(ImageError::Limits(LimitError::from_kind(
	LimitErrorKind::InsufficientMemory,
	)));
	}
	let mut buf = vec![num_traits::Zero::zero(); total_bytes.unwrap() / std::mem::size_of::<T>()];
	decoder.read_image(bytemuck::cast_slice_mut(buf.as_mut_slice()))?;
	Ok(buf)
	}

that re-uses an existing Vec instead of allocating a new one. The latter is easy enough to replicate, and so probably doesn't need to be provided by the image crate (though it'd be a nice-to-have).

The easiest way to provide this functionality is likely to add a method to io::Reader that exposes the ImageDecoder. Might be tricky given the Visitor pattern, but hopefully doable. Perhaps via a provided callback function?

I'll tack on to the end here that we currently use a Cursor over a &[u8] to construct an io::Reader, which feels a little odd. Is there a nicer way to get with_guessed_format without having to go via an I/O trait when you have the image in a buffer already, or is this the intended way?

[fintelia]

This is actually being addressed in the 0.25 release! The ImageDecoder trait is being changed to be object safe, at which point we can add a io::Reader::into_decoder method:

let decoder = Reader::open("path/to/image.png")?.into_decoder()?;
let size = decoder.total_bytes();
decoder.read_image(&mut buf[..size])?

I'll tack on to the end here that we currently use a Cursor over a &[u8] to construct an io::Reader, which feels a little odd. Is there a nicer way to get with_guessed_format without having to go via an I/O trait when you have the image in a buffer already, or is this the intended way?

If you only need to know the format, you can call guess_format which directly takes a slice of bytes. But if you're going to be decoding the image anyway, you already need to have a Cursor<&[u8]> so might as well use it for guessing the format. And with_guessed_format is pretty simple: it reads the first 16-bytes, looks them up in a table of magic bytes for different formats, and then seeks back to the start.

[jonhoo]

Amazing! Do you have a rough ETA for 0.25 (like, is it a few weeks away, a few months, or a few years)?

For the PS, I was more wondering whether there was a way to decode something that is in a &[u8] without a Cursor in the first place (I could imagine some impls could be faster when the whole image is in memory maybe?). But if that's not the case, that's fine — just wanted to check if there was a part of the API I had missed!

[fintelia]

There's currently a 0.25.0-preview.0 release out. Assuming no glaring issues are found with it, a full 0.25.0 should be at most a few weeks away.

As far as taking advantage of the whole image being in memory, that's actually something that's been discussed in the context of the png crate (which is one of the few decoders fast enough for it to possibly matter). The strategy we're investigating there is using the methods provided by the BufRead trait. In particular, the implementation of Cursor::fill_buf returns the the entire remainder of the slice. If you are curious there's currently a draft PR and more discussion here.

[fintelia]

0.25.0 is now released!