Skip to content

chore: collapse duplicated branches that differ by one value - #24814

Open
2010YOUY01 wants to merge 2 commits into
apache:mainfrom
2010YOUY01:feat-dedup-duplicated-branches
Open

chore: collapse duplicated branches that differ by one value#24814
2010YOUY01 wants to merge 2 commits into
apache:mainfrom
2010YOUY01:feat-dedup-duplicated-branches

Conversation

@2010YOUY01

Copy link
Copy Markdown
Contributor

Which issue does this PR close?

  • Closes #.

Rationale for this change

The following code snippet demonstrates the refactor idea, I find the second version is more eye-friendly, especially when the struct gets large.

// before
if foo {
    Config { a: v1, b: v2, c: v3 }
} else {
    Config { a: v1, b: v2, c: v4 }
}

// after
let c = if foo { v3 } else { v4 };
Config { a: v1, b: v2, c }

This PR applies this idea to safe locations across the codebase.

What changes are included in this PR?

What is the testing strategy for this PR?

Are there any user-facing changes?

Several places build the same value twice in `if`/`else` arms where only
one field, argument or fragment differs. Compute the differing part in the
branch and build the value once.

- `to_proto` / `unparser::expr`: `ILikeNode`/`LikeNode` and
  `ast::Expr::ILike`/`ast::Expr::Like` were built from identical field
  lists. Serialize the operands once, then pick the node type.
- `LogicalPlanBuilder::intersect_or_except`: the two builder chains differed
  only by an inserted `.distinct()?`.
- `min_max_struct`: pick the comparator fn pointer in the `if`, then call
  `update_batch` once.
- `unparser::plan`: `extension_to_sql` was called twice; only the `query`
  argument differed, and it is just `query.as_mut()`.
- `Expr` BETWEEN display: `SchemaDisplay` and `SqlDisplay` each duplicated
  the format string per `negated`; a `"NOT "` fragment collapses both.
- `sum`: the `helper!` macro was defined twice; only the accumulator type
  differed.
- `hash_join` test helper: choose both option values in the `if`, then
  assign once.
- `graphviz::add_node`: two `writeln!`s differing only by the optional
  tooltip.

No functional change intended.
@github-actions github-actions Bot added sql SQL Planner logical-expr Logical plan and expressions common Related to common crate proto Related to proto crate functions Changes to functions implementation physical-plan Changes to the physical-plan crate labels Aug 31, 2026
@codecov-commenter

codecov-commenter commented Aug 31, 2026

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 83.09859% with 12 lines in your changes missing coverage. Please review.
✅ Project coverage is 81.58%. Comparing base (e4cf35c) to head (6730144).
⚠️ Report is 7 commits behind head on main.

Files with missing lines Patch % Lines
datafusion/expr/src/expr.rs 50.00% 6 Missing ⚠️
datafusion/expr/src/logical_plan/builder.rs 83.33% 1 Missing and 1 partial ⚠️
datafusion/proto/src/logical_plan/to_proto.rs 89.47% 0 Missing and 2 partials ⚠️
datafusion/sql/src/unparser/expr.rs 84.61% 0 Missing and 2 partials ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##             main   #24814      +/-   ##
==========================================
+ Coverage   81.52%   81.58%   +0.05%     
==========================================
  Files        1123     1123              
  Lines      406148   406590     +442     
  Branches   406148   406590     +442     
==========================================
+ Hits       331124   331701     +577     
+ Misses      55659    55453     -206     
- Partials    19365    19436      +71     

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

Comment on lines 62 to 93
}

impl GroupsAccumulator for MinMaxStructAccumulator {
fn update_batch(
&mut self,
values: &[ArrayRef],
group_indices: &[usize],
opt_filter: Option<&BooleanArray>,
total_num_groups: usize,
) -> Result<()> {
let array = &values[0];
assert_eq!(array.len(), group_indices.len());
assert_eq!(array.data_type(), &self.inner.data_type);
// apply filter if needed
let array = apply_filter_as_nulls(array, opt_filter)?;

fn struct_min(a: &StructArray, b: &StructArray) -> bool {
matches!(partial_cmp_struct(a, b), Some(Ordering::Less))
}

fn struct_max(a: &StructArray, b: &StructArray) -> bool {
matches!(partial_cmp_struct(a, b), Some(Ordering::Greater))
}

if self.is_min {
self.inner.update_batch(
array.as_struct(),
group_indices,
total_num_groups,
struct_min,
)
} else {
self.inner.update_batch(
array.as_struct(),
group_indices,
total_num_groups,
struct_max,
)
}
let cmp: fn(&StructArray, &StructArray) -> bool =
if self.is_min { struct_min } else { struct_max };

self.inner
.update_batch(array.as_struct(), group_indices, total_num_groups, cmp)
}

fn evaluate(&mut self, emit_to: EmitTo) -> Result<ArrayRef> {

@getChan getChan Aug 31, 2026

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: This refactoring coerces struct_min / struct_max to a function pointer. Since cmp is called from the per-row loop in MinMaxStructState::update_batch, this is a hot path where even small overhead can accumulate.

I compared the release LLVM IR locally: the updated version selects a function pointer and invokes it indirectly inside the loop, whereas the original version produces separate MIN/MAX loops. I also ran a small local release benchmark for a single-field STRUCT MAX aggregation; the median changed from 1.20s to 1.46s (+22%).

The original code is a little more verbose, but would you mind keeping the two branches here to preserve static dispatch?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

good catch, reverted

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

common Related to common crate functions Changes to functions implementation logical-expr Logical plan and expressions physical-plan Changes to the physical-plan crate proto Related to proto crate sql SQL Planner

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants