SQL benchmark configuration parity updates - #24778 2 by Omega359 · Pull Request #24803 · apache/datafusion

Omega359 · 2026-08-30T16:19:22Z

Which issue does this PR close?

Part of #21706. This PR does not close the issue.

Rationale for this change

The SQL-based TPC-H, TPC-DS, and sort TPC-H benchmarks did not consistently expose the same schemas, constraints, and loading behavior as their native Rust equivalents. These differences could affect query planning and make benchmark results less directly comparable.

What changes are included in this PR?

Align the TPC-H CSV, Parquet, and in-memory table definitions with the native benchmark's data types, nullability, and primary-key constraints.
Add explicit TPC-DS schemas and primary-key constraints while retaining the physical types used by the DataFusion benchmark Parquet dataset.
Update the sort TPC-H loader to:
- Read the complete lineitem Parquet directory rather than only lineitem.1.parquet.
- Keep the table Parquet-backed instead of materializing it.
- Declare existing l_orderkey ASC NULLS LAST ordering as metadata instead of physically sorting the data.

The sort TPC-H changes match the default behavior of the native Rust benchmark when it is run without --mem-table.

Are these changes tested?

Yes.

The following checks were run:

cargo fmt --all -- --check
cargo clippy --all-targets --all-features -- -D warnings

Sort TPC-H query 1 was also executed against the SF1 dataset in both modes:

cargo run --profile release -p datafusion-benchmarks \
  --bin benchmark_runner -- sort_tpch \
  --scale-factor 1 --sorted false --query 1 --iterations 1

cargo run --profile release -p datafusion-benchmarks \
  --bin benchmark_runner -- sort_tpch \
  --scale-factor 1 --sorted true --query 1 --iterations 1

Both modes completed successfully and returned all 6,001,215 lineitem rows.

## Are there any user-facing changes?

No public API behavior changes. The SQL benchmark definitions now more closely match the corresponding native benchmark configurations.

…ll lineitem files and declaring existing sort order as metadata instead of materializing and physically sorting the data, to match the native Rust benchmark’s default behavior.

Omega359 · 2026-08-30T16:20:25Z

+CREATE EXTERNAL TABLE lineitem
+STORED AS PARQUET
+LOCATION '${DATA_DIR:-data}/tpch_sf${BENCH_SIZE:-1}/lineitem'
+${BENCH_SORTED:-false|WITH ORDER (l_orderkey ASC NULLS LAST)| };


This change was to not use ctas (thus a mem table) but rather use parquet level sorting.

Omega359 · 2026-08-30T16:21:04Z

+    cc_gmt_offset DECIMAL(5, 2),
+    cc_tax_percentage DECIMAL(5, 2),
+    PRIMARY KEY (cc_call_center_sk)
+) STORED AS PARQUET LOCATION '${DATA_DIR:-data}/tpcds_sf${BENCH_SIZE:-1}/call_center.parquet';


Full schema required to define the primary key matching rust native code.

maybe worth a -- sql comment to this effect

Omega359 · 2026-08-30T16:21:50Z

+    n_nationkey BIGINT       NOT NULL,
+    n_name      CHAR(25)     NOT NULL,
+    n_regionkey BIGINT       NOT NULL,
+    n_comment   VARCHAR(152) NOT NULL,


These changes are primarily int -> bigint + not null matching rust native version.

Omega359 · 2026-08-30T16:22:20Z

 CREATE EXTERNAL TABLE lineitem_raw STORED AS PARQUET LOCATION '${DATA_DIR:-data}/tpch_sf${BENCH_SIZE:-1}/lineitem/lineitem.1.parquet';

-CREATE TABLE nation as SELECT * FROM nation_raw;
+CREATE TABLE nation


full schema required for primary key.

codecov-commenter · 2026-08-30T16:44:05Z

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 81.52%. Comparing base (e4cf35c) to head (572708b).

Additional details and impacted files

@@            Coverage Diff             @@
##             main   #24803      +/-   ##
==========================================
- Coverage   81.52%   81.52%   -0.01%     
==========================================
  Files        1123     1123              
  Lines      406148   406148              
  Branches   406148   406148              
==========================================
- Hits       331124   331113      -11     
- Misses      55659    55668       +9     
- Partials    19365    19367       +2

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

Omega359 · 2026-08-30T19:52:25Z

fyi @alamb

alamb

makes sense to me -- thanks @Omega359

alamb · 2026-08-31T11:38:36Z

+    cc_gmt_offset DECIMAL(5, 2),
+    cc_tax_percentage DECIMAL(5, 2),
+    PRIMARY KEY (cc_call_center_sk)
+) STORED AS PARQUET LOCATION '${DATA_DIR:-data}/tpcds_sf${BENCH_SIZE:-1}/call_center.parquet';


maybe worth a -- sql comment to this effect

Omega359 added 3 commits August 30, 2026 10:38

Align data types and nullability with native version.

f3a00fe

Align data types and nullability with native version.

f1632ff

Load sort_tpch directly as a Parquet-backed external table, reading a…

c275ee9

…ll lineitem files and declaring existing sort order as metadata instead of materializing and physically sorting the data, to match the native Rust benchmark’s default behavior.

Omega359 commented Aug 30, 2026

View reviewed changes

Omega359 marked this pull request as ready for review August 30, 2026 19:52

Merge branch 'main' into sql_benchmark_sync_config_parity_2

572708b

Omega359 changed the title ~~Sql benchmark sync config parity 2~~ SQL benchmark configuration parity updates - #24778 2 Aug 31, 2026

alamb approved these changes Aug 31, 2026

View reviewed changes

Navigation Menu

Uh oh!

SQL benchmark configuration parity updates - #24778 2 - #24803

SQL benchmark configuration parity updates - #24778 2#24803
Omega359 wants to merge 4 commits into
apache:mainfrom
Omega359:sql_benchmark_sync_config_parity_2

Omega359 commented Aug 30, 2026

Uh oh!

Omega359 Aug 30, 2026

Uh oh!

Omega359 Aug 30, 2026

Uh oh!

alamb Aug 31, 2026

Uh oh!

Omega359 Aug 30, 2026

Uh oh!

Omega359 Aug 30, 2026

Uh oh!

codecov-commenter commented Aug 30, 2026 •

edited

Loading

Uh oh!

Omega359 commented Aug 30, 2026

Uh oh!

alamb left a comment •

edited

Loading

Uh oh!

alamb Aug 31, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Uh oh!

Conversation

Omega359 commented Aug 30, 2026

Which issue does this PR close?

Rationale for this change

What changes are included in this PR?

Are these changes tested?

Uh oh!

Omega359 Aug 30, 2026

Choose a reason for hiding this comment

Uh oh!

Omega359 Aug 30, 2026

Choose a reason for hiding this comment

Uh oh!

alamb Aug 31, 2026

Choose a reason for hiding this comment

Uh oh!

Omega359 Aug 30, 2026

Choose a reason for hiding this comment

Uh oh!

Omega359 Aug 30, 2026

Choose a reason for hiding this comment

Uh oh!

codecov-commenter commented Aug 30, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

Omega359 commented Aug 30, 2026

Uh oh!

alamb left a comment • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

alamb Aug 31, 2026

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

codecov-commenter commented Aug 30, 2026 •

edited

Loading

alamb left a comment •

edited

Loading