Benchmark: zlib-ng vs isa-l, zlib, libdeflate, brotli

[powturbo]

TurboBench : Build or download executables and test with your own data.

Benchmark1:
TurboBench: Dynamic/Static web content compression benchmark

Benchmark 2:
turbobench silesia.tar -eigzip,0,1,2,3/zlib_ng,1,3,6,9/libdeflate,1,3,6,9,12/zlib,1,3,6,9/memcpy
Hardware: Lenovo Ideapad 5 pro - Ryzen 6600hs / (bold = pareto) MB=1.000.000

C Size	ratio%	C MB/s	D MB/s	Name
64677910	30.5	7.47	1133.66	libdeflate 12
66715898	31.5	43.04	1116.39	libdeflate 9
67511452	31.9	119.35	1127.36	libdeflate 6
67644075	31.9	15.55	483.82	zlib 9
68152563	32.2	27.79	734.94	zlib_ng 9
68228660	32.2	37.74	478.69	zlib 6
68914854	32.5	92.33	735.24	zlib_ng 6
70166917	33.1	185.35	1110.18	libdeflate 3
71068342	33.5	203.57	1085.40	libdeflate 2
72490921	34.2	138.61	694.45	zlib_ng 3
72968832	34.4	86.50	480.09	zlib 3
73505577	34.7	288.56	1075.72	libdeflate 1
75138353	35.5	271.18	1080.75	igzip 3
76571415	36.1	598.69	1047.07	igzip 2
77260023	36.5	127.47	448.95	zlib 1
78154519	36.9	615.11	1020.09	igzip 1
87551010	41.3	638.49	969.43	igzip 0
100929713	47.6	329.63	651.73	zlib_ng 1
211948544	100.0	16146.00	16117.76	memcpy

[KungFuJesus]

I assume memcpy is just raw memory bandwidth with no compression? Both libdeflate and igzip have the advantage/disadvantage of not being forks from zlib but ground up implementations (on the flip side with incompatible APIs). Not to say there's no room for improvement for zlib-ng, but that is at least a footnote to be provided here. It looks like everybody's compression speed is a bit anemic. Naturally we'd expect compression to be slower, I guess, but I do wonder what's left on the table there without sacrificing compression ratios.

[nmoinvaz]

AFAIK libdeflate requires whole-buffer and does not support streaming. So if all you look at is just speed then libdeflate will always win.

On the higher levels zlib-ng compresses twice as fast as zlib, with slight decrease in compression size which is expected.

[KungFuJesus]

Somewhat interesting is at level 6 the decompression speed is higher than level 3, but perhaps that speaks to having an inflate that is working more in inflate_fast rather than decompressing literals? I'm just guessing, I don't have profiles to really evaluate that delta, today. I've definitely wanted something that chews through literals in the main inflate loop faster for a while now.

Err well, all the implementations share that trait. I guess it really is just not having to rely on memory read bandwidth as much with those higher compression ratios.

[powturbo]

I've added a web content benchmark now.
The average web page is 84k, streaming is not relevant here.

[KungFuJesus]

Heh, the sorting by size rather than throughput is throwing me off a bit. Looks like we don't do too much worse (in terms of compression throughput) than libdeflate at level 9, albeit with some trade-offs in compression.

Better compression algorithms of course, do better. But I'm not losing sleep over that, those things aren't zlib-ng's purview. It might be worth a weekend dive into the techniques libdeflate is benefiting from for decompression.

[nmoinvaz]

Why libdeflate is faster than vanilla zlib for decompression:
https://github.com/ebiggers/libdeflate/blob/02dfa32da3ee3982c66278e714d2e21276dfb67b/lib/deflate_decompress.c#L32-L43

[KungFuJesus]

Why libdeflate is faster than vanilla zlib for decompression: https://github.com/ebiggers/libdeflate/blob/02dfa32da3ee3982c66278e714d2e21276dfb67b/lib/deflate_decompress.c#L32-L43

Word accesses rather than byte accesses when copying matches

Pretty sure we do this, at least.

Word accesses rather than byte accesses when reading input

I would hope we do this but I'll have to look at the main loop to be sure. I'm not entirely convinced we couldn't do multiple words at a time and try to decode every possible op at once.

Faster Huffman decoding combined with various DEFLATE-specific tricks

That merits a deeper dive to figure out what they're talking about.

Larger bitbuffer variable that doesn't need to be refilled as often

I think we're doing a form of this now.

On x86_64, a version of the decompression routine is compiled with BMI instructions enabled and is used automatically at runtime when supported.

100% doing this, now.

[Dead2]

I do envy the quality of docstrings that libdeflate has.

[nmoinvaz]

Additionally, libdeflate uses an extra 32KB hash table for 3-byte matches. I don't think we want to consume that much more memory.
https://github.com/ebiggers/libdeflate/blob/02dfa32da3ee3982c66278e714d2e21276dfb67b/lib/hc_matchfinder.h#L69-L74

[danielhrisca]

which one is the isa-l in the results?

[powturbo]

igzip

[powturbo]

Extended benchmark TurboBench: Dynamic/Static web content compression benchmark including zstd and memory usage.
zlib-ng memory allocation must be revised to allocate only the minimum necessary!

[Dead2]

@powturbo Can you provide some detail on how you ran the benchmarks and how memory was measured?
AFAIK, this memory usage is not possible with just zlib-ng the library, as the allocations are static and very small.
Were you using minigzip/minideflate or some other application for the benchmarks? Those might have/cause a memory leak that we are not aware of.

[powturbo]

This done by TurboBench. The allocate/free functions are intercepted and the memory usage is monitored.
Build or download the linux TurboBench from releases and type
"./turbobench -ezlib_ng,6 file".
The memory & stack usage is reported in the "file.tbb" result file.