Repository navigation

GH-50628: [C++][Parquet] Rework RecordReader to introduce flat optional optimization - #50629

Open

AntoinePrv wants to merge 64 commits into

apache:mainfrom

AntoinePrv:levels-bitmap

Open

GH-50628: [C++][Parquet] Rework RecordReader to introduce flat optional optimization#50629
AntoinePrv wants to merge 64 commits into
apache:mainfrom
AntoinePrv:levels-bitmap

Commits on Oct 13, 2026
- Move TypedColReaderImpl::Skip
  AntoinePrv
  committed
  40e4906
- Break TypedRecordReader inheritance on TypedColumnReader
  AntoinePrv
  committed
  2f4f24e
- Add ReadValuesBuffer
  AntoinePrv
  committed
  64236d6
- Split ColumnReaderImplBase definition and declaration
  AntoinePrv
  committed
  d7469ff
- Add DefRepLevelsDecoder
  AntoinePrv
  committed
  c1e6f63
- Explicit int32_t for LevelDecoder
  AntoinePrv
  committed
  db0d814
- Change ValuesBuffer API
  AntoinePrv
  committed
  24fedd3
- Add StackedLevelDecoder and prepare for modularity
  AntoinePrv
  committed
  d0434b1
- Remove DefRepLevelsDecoder
  AntoinePrv
  committed
  71f4c47
- Split TypedRecordReader implementation
  AntoinePrv
  committed
  bd9dab8
- First RequiredTypedRecordReader
  AntoinePrv
  committed
  2bc1bef
- Remove RecordReader protected members
  AntoinePrv
  committed
  312a1db
- Fully use RequiredRecordReader
  AntoinePrv
  committed
  72c9b40
- Rename ReadValues -> ValueSink
  AntoinePrv
  committed
  a8aad72
- Split ReserveValuesAndIsValid
  AntoinePrv
  committed
  352ce2f
- Devirtualize hooks with CRTP
  AntoinePrv
  committed
  7fa3143
- Simplify template
  AntoinePrv
  committed
  e895904
- Remove StackedLevelDecoder
  AntoinePrv
  committed
  8d894d4
- Remove mark_as_written
  AntoinePrv
  committed
  a0b6283
- Make some of ColumnReaderImplBase private
  AntoinePrv
  committed
  ae70345
- Fix horrible dictionnary handling
  AntoinePrv
  committed
  2045c45
- Simplify and rename ColumnChunkReader
  AntoinePrv
  committed
  4511714
- Add RleBitPackedToBitmapDecoder::CountUpTo
  AntoinePrv
  committed
  ea5858b
- Add BitPackedToBitmapDecoder
  AntoinePrv
  committed
  f5decb1
- Add RleBitPackedToBitmapDecoder::Advance
  AntoinePrv
  committed
  886ccd9
- Add LevelToBitmapDecoder
  AntoinePrv
  committed
  a1b2ea1
- Add ValiditySinkBuffer
  AntoinePrv
  committed
  a10e481
- Auto reserve space in sinks:
  AntoinePrv
  committed
  57225c5
- Fix integer types
  AntoinePrv
  committed
  6ae3e92
- Nit in ColumnChunkReader
  AntoinePrv
  committed
  d0b4f08
- Eagerly read dict
  AntoinePrv
  committed
  f1ce660
- Cleanups
  AntoinePrv
  committed
  7956eaa
- Add FlatOptionalTypedRecordReader
  AntoinePrv
  committed
  8eb5da6
- Add some doc
  AntoinePrv
  committed
  5b25824
- Factorize ReadDictionary
  AntoinePrv
  committed
  518d193
- Simplify read_dense_for_nullable
  AntoinePrv
  committed
  8e49856
- Factor Sinks with DataSinkBuffer
  AntoinePrv
  committed
  69e3f55
- Use LevelBufferSink in TypedRecordReader
  AntoinePrv
  committed
  e25484b
- Relax record reader level tests flat optional
  AntoinePrv
  committed
  461dde4
- Fix compiler errrors
  AntoinePrv
  committed
  c7a3f03
- Safe decoded number check and func rename
  AntoinePrv
  committed
  bee1d4d
- Explicit values_written contract
  AntoinePrv
  committed
  03ed40b
- Fix LevelDecoder::max_level type
  AntoinePrv
  committed
  52d262d
- Fix integer convertion
  AntoinePrv
  committed
  dbcad25
- Factorize level decoders
  AntoinePrv
  committed
  bd08ddd
- Fixt typos
  AntoinePrv
  committed
  00eb016
- Move level decoder utility and rename file
  AntoinePrv
  committed
  bf33ee4
- Move level decoder adapters to level_decoder.h
  AntoinePrv
  committed
  4489157
- Fix small typos
  AntoinePrv
  committed
  d02b9c8
- Fix issues in level counting
  AntoinePrv
  committed
  069c20b
- Fix extra alloc
  AntoinePrv
  committed
  27cd7c7
- Add RecordReader::Make flag
  AntoinePrv
  committed
  a69c9af
- Throw on FlatOptional def_levels
  AntoinePrv
  committed
  6e99667
- Always call RecordReader with flat optiona optimization
  AntoinePrv
  committed
  2d34f9a
- Addition flat optional test
  AntoinePrv
  committed
  a846dd2
- Detemplate kdic
  AntoinePrv
  committed
  5b24c14
- Add flat optiona bse in flba and byte array
  AntoinePrv
  committed
  aeaac95
- Ad more flba and array test
  AntoinePrv
  committed
  a2a64a1
- Make RecordReader::Make params struct public
  AntoinePrv
  committed
  2fc997a
- Run RecordReader tests with and without optimizations
  AntoinePrv
  committed
  341d941
- Rename flat_optiona_optimization as allow_levels_optimization
  AntoinePrv
  committed
  b48cb75
- Add level optim option to RowGroupReader
  AntoinePrv
  committed
  dfc539c
- Reuse validity map
  AntoinePrv
  committed
  73d7df3
- Lazily pop dictionnary
  AntoinePrv
  committed
  734ce5c

Commits

Commits on Oct 13, 2026