Repository navigation
[C++] GZipCodec::Init does deflateInit and immediately throws it away #51308
Description
Describe the bug, including details regarding any error messages, version, and platform.
Performance issue in GZipCodec, no crashes/wrong results/..., but I still think it's a bug.
arrow/cpp/src/arrow/util/compression_zlib.cc
Lines 497 to 501 in cd304b2
This is wasteful: InitCompressor on line 497 calls deflateInit2, which at windowBits=15 and memLevel=8 allocates roughly 256 KB. InitDecompressor on line 501 calls EndCompressor which does deflateEnd and frees it again. It's pure waste. The only thing it gives us is some validation on the compression level.
The current code leaves the GZipCodec with a valid decompressor (which also allocates, but less) which is fine for the case where the caller wants to use the codec to do decompression, but is extra-wasteful for the case where the caller wants to compress: it allocates a compressor, frees it, allocates a decompressor, frees it, then allocates a compressor again.
I think the correct fix is firebolt-db#47, I'm happy to file it as a PR here as well.
Component(s)
C++
Activity
[-]GZipCodec::Init does deflateInit and immediately throws it away[/-][+][C++] GZipCodec::Init does deflateInit and immediately throws it away[/+]La1itchauhan commentedon Sep 22, 2026
Hi @lorenzhs, are you currently planning to open a PR for this upstream, or would you mind if I helped port the lazy initialization fix from firebolt-db#47 to apache/arrow (with you credited as co-author)? Happy to take care of the upstream PR, CI, and test coverage!
lorenzhs commentedon Sep 24, 2026
Lol, no need. I was hoping for a maintainer reaction before opening a PR, I do not need someone to click the button for me.
GH-51308: [C++] Initialize gzip codec state lazily (#51483)
pitrou commentedon Oct 6, 2026
Issue resolved by pull request 51483
#51483