We have the high-level expectation that on tabular data (such as in the mgbench benchmarks) SuperDB's performance querying a CSUP file should meet-or-beat the performance of querying the same data in a Parquet file. However, as of super commit e39a118, mgbench bench3/q1 is almost 2x worse on CSUP than Parquet, similar to this result with a simplified query running on my Intel-based Macbook:
$ curl -sO https://brim-benchmarks.s3.us-east-2.amazonaws.com/mgbench/bench3-e39a118.csup &&
curl -sO https://brim-benchmarks.s3.us-east-2.amazonaws.com/mgbench/bench3.parquet &&
super -version &&
time super -f csv -c "SELECT count(*) AS c FROM 'bench3-e39a118.csup' WHERE log_time >= TIMESTAMP '2030-01-01 00:00:00'" &&
time super -f csv -c "SELECT count(*) AS c FROM 'bench3.parquet' WHERE log_time >= TIMESTAMP '2030-01-01 00:00:00'"
Version: v0.3.0-420-ge39a1184f
c
0
real 0m0.172s
user 0m0.802s
sys 0m0.196s
c
0
real 0m0.098s
user 0m0.115s
sys 0m0.028s
Details
Repro is with super commit e39a118.
This has a similar high-level "CSUP perf is ~2x worse than Parquet" symptom as #6632, but I used Claude to do some profiling to confirm which mgbench queries shared a root cause, and this one came up different. Its write up is available in a Gist, and its tl;dr is that CSUP pays a fixed cost every time it opens a file: for every object in the file it creates a full BSUP reader with large buffers, just to read a few hundred bytes of metadata from each. That overhead matters most on queries like bench3/q1 whose filter matches only a tiny fraction of the data (2 of the 109M rows), since pruning skips nearly all the other work and leaves that fixed cost as most of the runtime. The Gist has profiling info, points at lines of code it believes are relevant, and includes a 2-line prototype that cut bench3/q1 on CSUP by 34%.
We have the high-level expectation that on tabular data (such as in the mgbench benchmarks) SuperDB's performance querying a CSUP file should meet-or-beat the performance of querying the same data in a Parquet file. However, as of super commit e39a118, mgbench bench3/q1 is almost 2x worse on CSUP than Parquet, similar to this result with a simplified query running on my Intel-based Macbook:
Details
Repro is with super commit e39a118.
This has a similar high-level "CSUP perf is ~2x worse than Parquet" symptom as #6632, but I used Claude to do some profiling to confirm which mgbench queries shared a root cause, and this one came up different. Its write up is available in a Gist, and its tl;dr is that CSUP pays a fixed cost every time it opens a file: for every object in the file it creates a full BSUP reader with large buffers, just to read a few hundred bytes of metadata from each. That overhead matters most on queries like bench3/q1 whose filter matches only a tiny fraction of the data (2 of the 109M rows), since pruning skips nearly all the other work and leaves that fixed cost as most of the runtime. The Gist has profiling info, points at lines of code it believes are relevant, and includes a 2-line prototype that cut bench3/q1 on CSUP by 34%.