[marcin-krystianc]

Rationale for this change

Follow-up to #50335.

DeltaByteArrayFLBADecoder::Decode(uint8_t*, int) decodes into a
temporary std::vector<ByteArray> and then copies the bytes into the caller's buffer,
because DeltaByteArrayDecoderImpl::GetInternal only supports ByteArray output. The
vector is a per-call allocation, and the pointer it holds for every value is discarded
immediately after the copy.

What changes are included in this PR?

Adds two output policy structs, ByteArrayOutput and DenseOutput for DeltaByteArrayFLBADecoder:

• ByteArrayOutput reproduces the previous behavior: it decodes values into a ByteArray* buffer
• DenseOutput writes each reconstructed fixed-length value directly into a caller-provided uint8_t* buffer

Adds benchmarks to test the DeltaByteArrayFLBADecoder performance.

• Before the change

Running /workspace/arrow/cpp/build/release/parquet-encoding-benchmark
Run on (20 X 2112 MHz CPU s)
CPU Caches:
  L1 Data 48 KiB (x10)
  L1 Instruction 32 KiB (x10)
  L2 Unified 1280 KiB (x10)
  L3 Unified 25600 KiB (x1)
Load Average: 2.22, 3.84, 3.64
---------------------------------------------------------------------------------------------------------------------------------------
Benchmark                                                                             Time             CPU   Iterations UserCounters...
---------------------------------------------------------------------------------------------------------------------------------------
BM_DeltaDecodingFLBA/type-length:2/batch-size:512/prefixed-percent:10              3217 ns         3215 ns       221856 bytes_per_second=303.706Mi/s compression_ratio=0.760208 items_per_second=159.229M/s
BM_DeltaDecodingFLBA/type-length:2/batch-size:512/prefixed-percent:90              3923 ns         3921 ns       185631 bytes_per_second=249.056Mi/s compression_ratio=1.38753 items_per_second=130.577M/s
BM_DeltaDecodingFLBA/type-length:2/batch-size:512/prefixed-percent:99              3929 ns         3927 ns       182369 bytes_per_second=248.649Mi/s compression_ratio=1.79649 items_per_second=130.363M/s
BM_DeltaDecodingFLBA/type-length:2/batch-size:2048/prefixed-percent:10            11335 ns        11329 ns        62995 bytes_per_second=344.803Mi/s compression_ratio=0.76148 items_per_second=180.776M/s
BM_DeltaDecodingFLBA/type-length:2/batch-size:2048/prefixed-percent:90            15514 ns        15506 ns        43892 bytes_per_second=251.918Mi/s compression_ratio=1.40466 items_per_second=132.078M/s
BM_DeltaDecodingFLBA/type-length:2/batch-size:2048/prefixed-percent:99            16525 ns        16517 ns        42482 bytes_per_second=236.502Mi/s compression_ratio=1.7747 items_per_second=123.995M/s
BM_DeltaDecodingFLBA/type-length:16/batch-size:512/prefixed-percent:10             2796 ns         2794 ns       249283 bytes_per_second=2.73034Gi/s compression_ratio=0.966836 items_per_second=183.23M/s
BM_DeltaDecodingFLBA/type-length:16/batch-size:512/prefixed-percent:90             4512 ns         4510 ns       151783 bytes_per_second=1.69171Gi/s compression_ratio=1.70489 items_per_second=113.529M/s
BM_DeltaDecodingFLBA/type-length:16/batch-size:512/prefixed-percent:99             4729 ns         4727 ns       139167 bytes_per_second=1.61411Gi/s compression_ratio=1.78203 items_per_second=108.321M/s
BM_DeltaDecodingFLBA/type-length:16/batch-size:2048/prefixed-percent:10           11151 ns        11144 ns        62274 bytes_per_second=2.73856Gi/s compression_ratio=0.966836 items_per_second=183.781M/s
BM_DeltaDecodingFLBA/type-length:16/batch-size:2048/prefixed-percent:90           17927 ns        17918 ns        39408 bytes_per_second=1.70318Gi/s compression_ratio=1.6732 items_per_second=114.298M/s
BM_DeltaDecodingFLBA/type-length:16/batch-size:2048/prefixed-percent:99           19689 ns        19678 ns        34933 bytes_per_second=1.55083Gi/s compression_ratio=1.79168 items_per_second=104.075M/s
BM_DeltaDecodingDenseFLBA/type-length:2/batch-size:512/prefixed-percent:10         3309 ns         3308 ns       211062 bytes_per_second=295.22Mi/s compression_ratio=0.760208 items_per_second=154.78M/s
BM_DeltaDecodingDenseFLBA/type-length:2/batch-size:512/prefixed-percent:90         4273 ns         4271 ns       162512 bytes_per_second=228.642Mi/s compression_ratio=1.38753 items_per_second=119.874M/s
BM_DeltaDecodingDenseFLBA/type-length:2/batch-size:512/prefixed-percent:99         4404 ns         4402 ns       157929 bytes_per_second=221.87Mi/s compression_ratio=1.79649 items_per_second=116.324M/s
BM_DeltaDecodingDenseFLBA/type-length:2/batch-size:2048/prefixed-percent:10       12811 ns        12804 ns        54299 bytes_per_second=305.071Mi/s compression_ratio=0.76148 items_per_second=159.945M/s
BM_DeltaDecodingDenseFLBA/type-length:2/batch-size:2048/prefixed-percent:90       17186 ns        17178 ns        39521 bytes_per_second=227.399Mi/s compression_ratio=1.40466 items_per_second=119.223M/s
BM_DeltaDecodingDenseFLBA/type-length:2/batch-size:2048/prefixed-percent:99       18260 ns        18250 ns        38287 bytes_per_second=214.042Mi/s compression_ratio=1.7747 items_per_second=112.219M/s
BM_DeltaDecodingDenseFLBA/type-length:16/batch-size:512/prefixed-percent:10        3218 ns         3217 ns       214108 bytes_per_second=2.37183Gi/s compression_ratio=0.966836 items_per_second=159.171M/s
BM_DeltaDecodingDenseFLBA/type-length:16/batch-size:512/prefixed-percent:90        4807 ns         4805 ns       126853 bytes_per_second=1.58781Gi/s compression_ratio=1.70489 items_per_second=106.556M/s
BM_DeltaDecodingDenseFLBA/type-length:16/batch-size:512/prefixed-percent:99        4941 ns         4939 ns       137130 bytes_per_second=1.54488Gi/s compression_ratio=1.78203 items_per_second=103.675M/s
BM_DeltaDecodingDenseFLBA/type-length:16/batch-size:2048/prefixed-percent:10      12255 ns        12249 ns        55533 bytes_per_second=2.4915Gi/s compression_ratio=0.966836 items_per_second=167.202M/s
BM_DeltaDecodingDenseFLBA/type-length:16/batch-size:2048/prefixed-percent:90      19460 ns        19338 ns        35959 bytes_per_second=1.57813Gi/s compression_ratio=1.6732 items_per_second=105.906M/s
BM_DeltaDecodingDenseFLBA/type-length:16/batch-size:2048/prefixed-percent:99      21114 ns        21103 ns        32878 bytes_per_second=1.4461Gi/s compression_ratio=1.79168 items_per_second=97.0458M/s

• After the change

Running /workspace/arrow/cpp/build/release/parquet-encoding-benchmark
Run on (20 X 2112 MHz CPU s)
CPU Caches:
  L1 Data 48 KiB (x10)
  L1 Instruction 32 KiB (x10)
  L2 Unified 1280 KiB (x10)
  L3 Unified 25600 KiB (x1)
Load Average: 1.24, 6.66, 4.25
---------------------------------------------------------------------------------------------------------------------------------------
Benchmark                                                                             Time             CPU   Iterations UserCounters...
---------------------------------------------------------------------------------------------------------------------------------------
BM_DeltaDecodingFLBA/type-length:2/batch-size:512/prefixed-percent:10              3419 ns         3418 ns       188811 bytes_per_second=285.752Mi/s compression_ratio=0.760208 items_per_second=149.816M/s
BM_DeltaDecodingFLBA/type-length:2/batch-size:512/prefixed-percent:90              3843 ns         3841 ns       174937 bytes_per_second=254.249Mi/s compression_ratio=1.38753 items_per_second=133.3M/s
BM_DeltaDecodingFLBA/type-length:2/batch-size:512/prefixed-percent:99              3730 ns         3728 ns       186091 bytes_per_second=261.966Mi/s compression_ratio=1.79649 items_per_second=137.345M/s
BM_DeltaDecodingFLBA/type-length:2/batch-size:2048/prefixed-percent:10            13447 ns        13439 ns        51135 bytes_per_second=290.661Mi/s compression_ratio=0.76148 items_per_second=152.39M/s
BM_DeltaDecodingFLBA/type-length:2/batch-size:2048/prefixed-percent:90            17300 ns        17292 ns        39339 bytes_per_second=225.901Mi/s compression_ratio=1.40466 items_per_second=118.437M/s
BM_DeltaDecodingFLBA/type-length:2/batch-size:2048/prefixed-percent:99            16789 ns        16781 ns        40393 bytes_per_second=232.784Mi/s compression_ratio=1.7747 items_per_second=122.046M/s
BM_DeltaDecodingFLBA/type-length:16/batch-size:512/prefixed-percent:10             3138 ns         3137 ns       226716 bytes_per_second=2.43218Gi/s compression_ratio=0.966836 items_per_second=163.221M/s
BM_DeltaDecodingFLBA/type-length:16/batch-size:512/prefixed-percent:90             3569 ns         3568 ns       192341 bytes_per_second=2.13857Gi/s compression_ratio=1.70489 items_per_second=143.517M/s
BM_DeltaDecodingFLBA/type-length:16/batch-size:512/prefixed-percent:99             3437 ns         3435 ns       198130 bytes_per_second=2.22082Gi/s compression_ratio=1.78203 items_per_second=149.036M/s
BM_DeltaDecodingFLBA/type-length:16/batch-size:2048/prefixed-percent:10           12079 ns        12072 ns        56551 bytes_per_second=2.52788Gi/s compression_ratio=0.966836 items_per_second=169.643M/s
BM_DeltaDecodingFLBA/type-length:16/batch-size:2048/prefixed-percent:90           14949 ns        14941 ns        43250 bytes_per_second=2.04249Gi/s compression_ratio=1.6732 items_per_second=137.069M/s
BM_DeltaDecodingFLBA/type-length:16/batch-size:2048/prefixed-percent:99           15358 ns        15350 ns        45877 bytes_per_second=1.98814Gi/s compression_ratio=1.79168 items_per_second=133.422M/s
BM_DeltaDecodingDenseFLBA/type-length:2/batch-size:512/prefixed-percent:10         3262 ns         3261 ns       212131 bytes_per_second=299.505Mi/s compression_ratio=0.760208 items_per_second=157.027M/s
BM_DeltaDecodingDenseFLBA/type-length:2/batch-size:512/prefixed-percent:90         3518 ns         3516 ns       196251 bytes_per_second=277.715Mi/s compression_ratio=1.38753 items_per_second=145.603M/s
BM_DeltaDecodingDenseFLBA/type-length:2/batch-size:512/prefixed-percent:99         3536 ns         3534 ns       199117 bytes_per_second=276.305Mi/s compression_ratio=1.79649 items_per_second=144.863M/s
BM_DeltaDecodingDenseFLBA/type-length:2/batch-size:2048/prefixed-percent:10       12662 ns        12656 ns        55254 bytes_per_second=308.658Mi/s compression_ratio=0.76148 items_per_second=161.826M/s
BM_DeltaDecodingDenseFLBA/type-length:2/batch-size:2048/prefixed-percent:90       16081 ns        16073 ns        40982 bytes_per_second=243.035Mi/s compression_ratio=1.40466 items_per_second=127.421M/s
BM_DeltaDecodingDenseFLBA/type-length:2/batch-size:2048/prefixed-percent:99       16415 ns        16407 ns        42847 bytes_per_second=238.086Mi/s compression_ratio=1.7747 items_per_second=124.825M/s
BM_DeltaDecodingDenseFLBA/type-length:16/batch-size:512/prefixed-percent:10        2969 ns         2968 ns       238205 bytes_per_second=2.57067Gi/s compression_ratio=0.966836 items_per_second=172.515M/s
BM_DeltaDecodingDenseFLBA/type-length:16/batch-size:512/prefixed-percent:90        3462 ns         3460 ns       203926 bytes_per_second=2.20507Gi/s compression_ratio=1.70489 items_per_second=147.98M/s
BM_DeltaDecodingDenseFLBA/type-length:16/batch-size:512/prefixed-percent:99        3392 ns         3390 ns       200885 bytes_per_second=2.25044Gi/s compression_ratio=1.78203 items_per_second=151.025M/s
BM_DeltaDecodingDenseFLBA/type-length:16/batch-size:2048/prefixed-percent:10      11560 ns        11554 ns        60264 bytes_per_second=2.64135Gi/s compression_ratio=0.966836 items_per_second=177.258M/s
BM_DeltaDecodingDenseFLBA/type-length:16/batch-size:2048/prefixed-percent:90      14602 ns        14512 ns        49410 bytes_per_second=2.10297Gi/s compression_ratio=1.6732 items_per_second=141.128M/s
BM_DeltaDecodingDenseFLBA/type-length:16/batch-size:2048/prefixed-percent:99      14782 ns        14775 ns        47312 bytes_per_second=2.06547Gi/s compression_ratio=1.79168 items_per_second=138.612M/s

Are these changes tested?

Yes

Are there any user-facing changes?

No

• GitHub Issue: [C++][Parquet] Make DELTA_BYTE_ARRAY value reconstruction storage-agnostic #50822

[github-actions]

⚠️ GitHub issue #50822 has been automatically assigned in GitHub to PR creator.

[adamreeve]

Thanks for this @marcin-krystianc

Looking at the benchmark results, I see that DeltaDecodingDenseFLBA is faster than it was before, and now consistently faster than the non-dense path, whereas before it was always slower. But the non-dense path results are slower in some cases than before. Is that just noise, or is it repeatable, and if so, is it possible to get back some of the performance for those rows?

I suspect it's not just noise, because it looks like the performance change is correlated with the prefix percent (10% prefixed is consistently slower, but 90% and 99% are mostly faster).

Could you also check before and after results for the BM_DeltaDecodingByteArray benchmarks?

[adamreeve]

DecodeArrow still uses this path for the FLBA type, via DecodeArrowDense. Would it be possible to use DecodeDense for reading FLBA into Arrow, or would that be a much more complicated change?

[adamreeve]

Nit: DecodeFixedLenByteArray might be a more suitable name.