Cap buffer sizes via ZlibStream::set_max_total_output.

[anforowicz]

PTAL? This results in quite a significant performance improvement for the noncompressed-8x8.png benchmark (bigger improvement than #427 and #428 combined - I am surprised).

[anforowicz]

Hmmm... I guess I need to look into the fuzzing failure. I suspect that IHDR can declare a smaller image size than the real image - I'll try to author a regression test that would do that.

[anforowicz]

Hmmm... I guess I need to look into the fuzzing failure. I suspect that IHDR can declare a smaller image size than the real image - I'll try to author a regression test that would do that.

This is done now. I am not sure if I need to do something special to kick-off the CI tests again?

[anforowicz]

@fintelia, can you PTAL?

[fintelia]

Haven't forgotten about this PR. But I've been wanting to finish image-rs/fdeflate#14 first so that we don't have to reserve extra space just to make fdeflate happy

Update: fdeflate:0.3.2 is now published. It removes the requirements for extra buffer space in the output buffer

[anforowicz]

@fintelia, can you PTAL again?

Thanks for the fdeflate changes. I've tweaked the PR-under-review to simplify things as allowed by the new fdeflate behavior.

FWIW, I've re-measured the impact on the non-compressed 8x8 image and still get 20+% improvement.

[fintelia]

nit: "8 image" -> "8x256 image"

[anforowicz]

Ooops. Done.

[fintelia]

Left a small comment, but otherwise looks good to me. As a followup I plan to experiment with changing ZlibStream to directly write into a caller provided buffer instead of using its own out_buffer.

[anforowicz]

Left a small comment, but otherwise looks good to me.

Ack. Thanks for reviewing!

As a followup I plan to experiment with changing ZlibStream to directly write into a caller provided buffer instead of using its own out_buffer.

Thanks for looking into this. Let me try to share a few notes, hoping that it helps (i.e. I am mostly trying to share knowledge and past experience, and definitely not trying to influence the changes one way or another - I don't claim to know which way is "right").

First of all - I note that there are interesting interdependencies between the data:

• Final image doesn't have the "filter type" byte at the front of each row, but the raw decompressed data includes such byte.
• Decompressor needs to retain 32kB for looking back.

My early attempts at a similar change can be found in anforowicz@6071020. (Although this is slightly different than what you are considering - here I am avoiding passing &mut Vec<u8> into ZlibStream's decompress and finish_compressed_chunks methods, but I still keep ZlibStream::out_buffer.) I've abandoned this attempt because:

• I was using this as a prerequisite for an avoiding extra copies into prev row, but this copying has been taken care of with a different, simpler approach - Reduce copying and allocations #422 (thanks!)
• I was getting mixed performance results (probably because I approached this wrong - I was mixing and measuring multiple related/dependent changes, which made it difficult to understand which changes had a positive/negative impact).

At the same time, removing or replacing the &mut Vec<u8> parameter from ZlibStream's methods can result in some code simplification (and therefore it may be desirable even if it is performance-neutral). For example, there are some callers of ZlibStream::decompress which pass &mut vec![] or Vec::new (read_header_info and finish_decoding in mod.rs).