Skip to content

fix: handle nested arrays in UNNEST by checking for grouping in left query (#17949) - #20094

Open
zhang-arvin wants to merge 1 commit into
apache:masterfrom
zhang-arvin:fix/issue-17949-unnest-nested-arrays
Open

fix: handle nested arrays in UNNEST by checking for grouping in left query (#17949)#20094
zhang-arvin wants to merge 1 commit into
apache:masterfrom
zhang-arvin:fix/issue-17949-unnest-nested-arrays

Conversation

@zhang-arvin

Copy link
Copy Markdown

Description

Fixes #17949: UNNEST with nested arrays fails when the left side of the correlate contains GROUP BY aggregation.

Root Cause

DruidCorrelateUnnestRel.toDruidQuery() uses computeLeftRequiresSubquery() to determine whether to wrap the left query in a QueryDataSource. However, computeLeftRequiresSubquery() only checks the PartialDruidQuery stage (SCAN), but when UNNEST is pulled up above a GROUP BY, the left side may contain grouping/aggregation pushed into the DruidQuery itself. In this case, getDataSource() returns the raw table scan instead of a subquery that wraps the aggregation.

Fix

Added an additional check in toDruidQuery(): when the computed updatedLeftQuery is a GroupByQuery (i.e., contains grouping), wrap it in a QueryDataSource even if computeLeftRequiresSubquery() returns false. This ensures the grouping/aggregation logic is preserved in the UNNEST data source.

Changes

  • sql/src/main/java/org/apache/druid/sql/calcite/rel/DruidCorrelateUnnestRel.java: Added instanceof GroupByQuery check in the toDruidQuery() method to detect when the left query contains grouping, and use QueryDataSource accordingly.

@FrankChen021 FrankChen021 left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Severity Findings
P0 0
P1 1
P2 0
P3 0
Total 1

Reviewed 1 of 1 changed files.


This is an automated review by Codex GPT-5.6-Luna(max)

final RowSignature leftSignature = DruidRels.dataSourceSignature(newLeftDruidRel);
if (whereFilter == null) {
if (computeLeftRequiresSubquery(newLeftDruidRel)) {
if (computeLeftRequiresSubquery(newLeftDruidRel) || updatedLeftQuery.getQuery() instanceof GroupByQuery) {

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[P1] GroupBy detection is unreachable

This check runs only when the partial-query stage is SCAN, WHERE_FILTER, or SELECT_PROJECT. DruidQuery emits GroupByQuery only when an aggregate exists, which places the stage at AGGREGATE or later. Therefore the new condition cannot change the data-source choice for the reported grouped-left-side case, leaving the fix ineffective.

@zhang-arvin
zhang-arvin force-pushed the fix/issue-17949-unnest-nested-arrays branch from ce94d2c to 45a1b0c Compare August 21, 2026 15:50
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Incorrect output for UNNEST on nested arrays

2 participants