Skip to content

fix: Allow a correlated scalar subquery to group without aggregating #24783

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Draft
Dandandan wants to merge 1 commit into
base: main
Choose a base branch
from
+28 −7
Draft

fix: Allow a correlated scalar subquery to group without aggregating #24783

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
10 changes: 5 additions & 5 deletions datafusion/expr/src/logical_plan/invariants.rs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -362,11 +362,11 @@ fn check_aggregation_in_scalar_subquery(
inner_plan: &LogicalPlan,
agg: &Aggregate,
) -> Result<()> {
if agg.aggr_expr.is_empty() {
return plan_err!(
"Correlated scalar subquery must be aggregated to return at most one row"
);
}
// The grouping is what makes the subquery scalar, not the aggregation. An
// empty GROUP BY returns one row, and a GROUP BY on only correlated columns
// returns at most one row per set of outer values. So subqueries without
// aggregate expressions are allowed as long as the GROUP BY check below
// passes.
if !agg.group_expr.is_empty() {
let correlated_exprs = get_correlated_expressions(inner_plan)?;
let inner_subquery_cols =
Expand Down
25 changes: 23 additions & 2 deletions datafusion/sqllogictest/test_files/subquery.slt
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -446,9 +446,30 @@ SELECT t1_id, (SELECT t3_int FROM t3 WHERE t3.t3_id = t1.t1_id) as t3_int from t
44 3


#non_aggregated_correlated_scalar_subquery
statement error DataFusion error: Invalid \(non-executable\) plan after Analyzer\ncaused by\nError during planning: Correlated scalar subquery must be aggregated to return at most one row
# A GROUP BY on only correlated columns returns at most one row per outer row,
# so no aggregate function is needed to make the subquery scalar.
#grouped_correlated_scalar_subquery_without_aggregate
query II rowsort
SELECT t1_id, (SELECT t2_int FROM t2 WHERE t2.t2_int = t1_int group by t2_int) as t2_int from t1
----
11 1
22 NULL
33 3
44 NULL

query TT
explain SELECT t1_id, (SELECT t2_int FROM t2 WHERE t2.t2_int = t1_int group by t2_int) as t2_int from t1
----
logical_plan
01)Projection: t1.t1_id, __scalar_sq_1.t2_int AS t2_int
02)--Left Join: t1.t1_int = __scalar_sq_1.t2_int
03)----TableScan: t1 projection=[t1_id, t1_int]
04)----SubqueryAlias: __scalar_sq_1
05)------Aggregate: groupBy=[[t2.t2_int]], aggr=[[]]
06)--------TableScan: t2 projection=[t2_int]

# A GROUP BY on a non-correlated column is still rejected, see
# `aggregated_correlated_scalar_subquery_with_extra_group_by_columns` below.

#non_aggregated_correlated_scalar_subquery_with_limit
statement error DataFusion error: Invalid \(non-executable\) plan after Analyzer\ncaused by\nError during planning: Correlated scalar subquery must be aggregated to return at most one row
Expand Down