Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add option to tune how the buffer size is allocated. #27

Open
gendx opened this issue Mar 5, 2020 · 1 comment
Open

Add option to tune how the buffer size is allocated. #27

gendx opened this issue Mar 5, 2020 · 1 comment
Labels
performance Make lzma-rs faster!

Comments

@gendx
Copy link
Owner

gendx commented Mar 5, 2020

As mentioned in #22 (comment), there is a trade-off between memory usage and speed.

I suggest adding an option to control the original size of the LZ buffer:

It remains to be seen whether there would be a performance regression between the code before #22 and using the DictSize option.

@gendx gendx added the performance Make lzma-rs faster! label Mar 5, 2020
@dragly
Copy link
Contributor

dragly commented Mar 9, 2020

I think this makes sense. It would be interesting to also see how other libraries handles this. After all, it might be that the files we are parsing are actually malformed and should have reported a smaller dictionary size.

I would be very happy with using DictSize by default if we can override it by setting InitialSize in options.

Also, it would be interesting to know why the allocation is using alloc_zeroed. Perhaps it is possible and safe to use a non-zero-initialized buffer? Our problem was that alloc_zeroed is slow in WebAssembly because of a call to memset.

chyyran added a commit to SnowflakePowered/lzma-rs that referenced this issue Aug 5, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
performance Make lzma-rs faster!
Projects
None yet
Development

No branches or pull requests

2 participants