Repository navigation
perf(query): prune and order immutable Parquet files by event-time window #273
Description
Activity
Implemented: all three signals now derive actual signed event-time bounds from complete footer statistics. Trusted dashboard queries bind explicit overlapping files, newest first; missing bounds include files. Arbitrary SQL keeps all active files. Format 3 is unchanged. Tests cover equal ingest times with disjoint events, nanoseconds/negative instants, multiple row groups, absent statistics, qualified relations, CTE shadowing, and parameter preservation.
Implemented in ea13675; tracked in #273. Measurement report and reproduction.
just check, the full Go race suite, and native Linux ARM64 full Go tests pass. Issue closure is linked to merge of #272.
PR #272 review follow-up for
views.go: #272 (comment)Eliminate per-statement glob/footer work for unrelated retained batches and bind selected files newest first under the existing publication pin. Span metadata has start bounds; logs/metrics have ingest bounds only. Obtain actual per-signal event-time bounds without pruning by ingest time, and preserve arbitrary SQL's complete approved snapshot.
Validation: compare pruned and full scans across all three signals, late/future/out-of-order timestamps, zero/negative instants, nanosecond and partial windows, reopen, compaction/retention, and canceled readers. Measure actual files/row groups read and newest-first top-N behavior on a multi-age dataset.
Implementation is authorized in PR #272; this issue tracks the fix and its validation.