-
Notifications
You must be signed in to change notification settings - Fork 26
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Allow passing in a preallocated buffer to LzBuffer constructors #76
Conversation
The preallocated buffer is then reused throughout subsequent calls to `decompress`
Benchmarks on
Benchmarks on this branch with default
Looks to be within a 2% difference... |
Converting this to a draft as I'm not entirely happy with having to pass ownership around where unnecessary. Even if Something like SnowflakePowered@9ea9da4 holds a mutable reference instead is probably preferable but that adds a somewhat noisy lifetime to |
Closing this for now, needs a little more work. |
Pull Request Overview
This is a small experiment building off of the raw primitives introduced in #74 to design a solution for #27.
This PR introduces the functions
LzmaDecoder::new_with_buffer
andLzma2Decoder::new_with_buffer
that allow constructing an LZMA(2) decoder with a preallocatedVec<u8>
buffer. The buffer is then reused on subsequent calls todecompress
.Lzma(2)Decoder::reset
will clear (memset
) the buffer without dropping the prior allocation although it may be resized if necessary within internal decompression routines.This allows consumers to pre-allocate a buffer upfront to mitigate the performance impact seen after #22 if necessary when using the raw decoder API.
I'm not actually sure this gives any performance benefit but I'm just putting it out here to get some thoughts.
Help Wanted
This pull request still needs proper benchmarking in a variety of cases. For my own usage in https://github.com/SnowflakePowered/chd-rs, I saw absolutely no performance difference between preallocating a buffer or using a fresh one via
Vec::new()
. There was also no difference in performance for my use case gained by reusing the buffer on eachdecompress
.Testing Strategy
This pull request was tested by...
.lzma
,.lzma2
,.xz
files).