fix: use SUM0 in AggregateCaseToFilter D1 rewrite to avoid NULL on empty windows - #20093
fix: use SUM0 in AggregateCaseToFilter D1 rewrite to avoid NULL on empty windows#20093zhang-arvin wants to merge 1 commit into
Conversation
…pty windows The D1 rewrite in DruidAggregateCaseToFilterRule converts SUM(CASE WHEN COND THEN COL1 ELSE 0 END) to SUM(COL1) FILTER(WHERE COND), but SUM returns NULL when the filter never matches instead of the expected 0. This fix changes the D1 rewrite to use SUM0 instead of SUM, which returns 0 for empty/null input, matching the original CASE expression behavior. Fixes apache#18058, apache#17768
FrankChen021
left a comment
There was a problem hiding this comment.
| Severity | Findings |
|---|---|
| P0 | 0 |
| P1 | 1 |
| P2 | 1 |
| P3 | 0 |
| Total | 2 |
Reviewed 2 of 2 changed files.
This is an automated review by Codex GPT-5.6-Luna(max)
| RelDataType newType = typeFactory.createTypeWithNullability(call.getType(), false); | ||
| return AggregateCall.create( | ||
| call.getAggregation(), | ||
| SqlStdOperatorTable.SUM0, |
There was a problem hiding this comment.
[P1] SUM0 still executes as nullable SUM
SumZeroSqlAggregator inherits SumSqlAggregator's nullable native Long/Float/DoubleSumAggregatorFactory. With no matching filter rows, the nullable wrapper returns NULL, so native Druid queries still produce NULL for the empty/no-match cases changed to 0 in filtered_sum.iq.
| RelDataType newType = typeFactory.createTypeWithNullability(call.getType(), false); | ||
| return AggregateCall.create( | ||
| call.getAggregation(), | ||
| SqlStdOperatorTable.SUM0, |
There was a problem hiding this comment.
[P2] Rewrite changes all-null semantics
When every row matches the condition and the value is NULL, SUM(CASE WHEN condition THEN value ELSE 0 END) returns NULL, while SUM0(value) FILTER returns 0. The rewrite applies this universally; the rewrite-disabled test still documents NULL for the 7=7,null case.
Description
Fixes #18058 and #17768.
The
DruidAggregateCaseToFilterRuleD1 rewrite convertsSUM(CASE WHEN COND THEN COL1 ELSE 0 END)toSUM(COL1) FILTER(WHERE COND). However,SUMreturns NULL when the filter never matches, while the original CASE expression would return 0. This causes incorrect NULL results for queries with aggregations on empty windows.Root Cause
The D1 rewrite in
DruidAggregateCaseToFilterRule.transform()usescall.getAggregation()(which isSUM) when creating the filtered aggregate call.SUMreturns NULL for empty groups, but the originalSUM(CASE WHEN ... ELSE 0 END)returns 0 when no rows match the condition.Fix
Changed the D1 rewrite to use
SUM0(viaSqlStdOperatorTable.SUM0) instead ofSUM.SUM0returns 0 for empty/null input, which matches the expected behavior of the original CASE expression.Changes
DruidAggregateCaseToFilterRule.java: Changed the D1 case intransform()to useSUM0instead ofSUMfor the filtered aggregate call, with non-nullable result type.filtered_sum.iq: Updated expected test results to reflect the corrected behavior (0 instead of NULL for empty window and no-match cases).Behavior Changes
Key Features
Testing
Updated the
filtered_sum.iqquidem test to verify the corrected behavior for all four scenarios (empty input, no match, some match, all null values).