[Alena0704]

What does this PR do?

Fix incorrect results when ORCA decorrelates aggregate subqueries or removes outer rows before a scalar-subquery filter can evaluate.

The following examples use the same data:

CREATE TABLE cb (a int, b int) DISTRIBUTED BY (a);
INSERT INTO cb VALUES (0, 1);
SET optimizer = on;

1. COUNT inside IN

SELECT * FROM cb x
WHERE x.a IN (
    SELECT count(*) FROM cb y WHERE y.b = x.a
);

Before the fix: no rows. After the fix: (0, 1).

The inner input is empty, but count(*) returns zero. The condition is therefore 0 IN (0), which is true.

Plans before and after

Before:

Gather Motion 3:1  (slice1; segments: 3)
  ->  Hash Join
        Hash Cond: ((y.b = x.a) AND ((count(*)) = (x.a)::bigint))
        ->  GroupAggregate
              Group Key: y.b
              ->  Sort
                    Sort Key: y.b
                    ->  Redistribute Motion 3:3  (slice2; segments: 3)
                          Hash Key: y.b
                          ->  Seq Scan on cb y
        ->  Hash
              ->  Seq Scan on cb x
Optimizer: GPORCA

After:

Gather Motion 3:1  (slice1; segments: 3)
  ->  Result
        Filter: (x.a = COALESCE((count(*)), '0'::bigint))
        ->  Hash Left Join
              Hash Cond: (x.a = y.b)
              ->  Seq Scan on cb x
              ->  Hash
                    ->  GroupAggregate
                          Group Key: y.b
                          ->  Sort
                                Sort Key: y.b
                                ->  Redistribute Motion 3:3  (slice2; segments: 3)
                                      Hash Key: y.b
                                      ->  Seq Scan on cb y
Optimizer: GPORCA

The left join preserves the outer row, and COALESCE restores zero before the comparison.

2. Empty-input values of other aggregates

SELECT x.a,
       (SELECT regr_count(y.a, y.b)
        FROM cb y WHERE y.b = x.a) AS n
FROM cb x;

Before the fix: (0, NULL). After the fix: (0, 0).

Like count, regr_count returns zero on empty input. The fix also preserves the actual empty-input results of user-defined aggregates.

Plans before and after

Before:

Gather Motion 3:1  (slice1; segments: 3)
  ->  Hash Left Join
        Hash Cond: (x.a = y.b)
        ->  Seq Scan on cb x
        ->  Hash
              ->  GroupAggregate
                    Group Key: y.b
                    ->  Sort
                          Sort Key: y.b
                          ->  Redistribute Motion 3:3  (slice2; segments: 3)
                                Hash Key: y.b
                                ->  Seq Scan on cb y
Optimizer: GPORCA

After:

 Result
   ->  Gather Motion 3:1  (slice1; segments: 3)
         ->  Hash Left Join
               Hash Cond: (x.a = y.b)
               ->  Seq Scan on cb x
               ->  Hash
                     ->  HashAggregate
                           Group Key: y.b
                           ->  Redistribute Motion 3:3
                                 Hash Key: y.b
                                 ->  Seq Scan on cb y
               SubPlan 1
                 ->  Result
                       One-Time Filter: ((true) IS NULL)
                       ->  Aggregate
                             ->  Result
                                   One-Time Filter: false
Optimizer: GPORCA

For this dataset, ORCA chooses a correlated aggregate SubPlan that preserves the empty-input result. Supported queries remain eligible for join plans.

3. Filters that accept NULL

SELECT * FROM cb x
WHERE (
    SELECT y.a FROM cb y WHERE y.b = x.a
) IS NULL;

Before the fix: no rows. After the fix: (0, 1).

The scalar subquery returns no rows, so its value is NULL and IS NULL is true. The outer row must survive until the filter is evaluated.

Plans before and after

Before:

Result
  Filter: (SubPlan 1)
  ->  Gather Motion 3:1  (slice1; segments: 3)
        ->  Seq Scan on cb x
  SubPlan 1
    ->  Result
          Filter: (y.b = x.a)
          ->  Materialize
                ->  Gather Motion 3:1  (slice2; segments: 3)
                      ->  Seq Scan on cb y
                            Filter: (a IS NULL)
Optimizer: GPORCA

After:

Gather Motion 3:1  (slice1; segments: 3)
  ->  Result
        Filter: ((SubPlan 1) IS NULL)
        ->  Seq Scan on cb x
        SubPlan 1
          ->  Result
                Filter: (y.b = x.a)
                ->  Materialize
                      ->  Broadcast Motion 3:3  (slice2; segments: 3)
                            ->  Seq Scan on cb y
Optimizer: GPORCA

The fixed plan evaluates IS NULL on the scalar subquery result, preserving the outer row when the subquery is empty.

The implementation preserves aggregate values before evaluating expressions and HAVING, and reuses scalar-subquery handling for supported aggregate IN/ANY queries.

When HAVING, grouping, or LIMIT can remove the aggregate row, quantified semantics are retained: an empty scalar subquery yields NULL, while IN over an empty result is false.

Fixes the GPORCA cases reported in #2046.

There was found an additional issue during development. Testing nested aggregate subqueries exposed a separate statistics issue: fractional row estimates could produce join selectivity above one or trigger an assertion. A separate commit fixes this by keeping the affected join scale factors and their bound at least one.

Type of Change

• Bug fix (non-breaking change)
• New feature (non-breaking change)
• Breaking change (fix or feature with breaking changes)
• Documentation update

Test Plan

• Added/updated SQL regression tests and expected output.
• Built the updated server and ORCA code.
• Verified the [Bug] Wrong results: GPORCA returns NULL instead of the empty-input value for correlated aggregate subqueries other than count #2046 reproducer.
• Passed make installcheck.
• Passed make -C src/test installcheck-cbdb-parallel.

Additional Context

The COUNT/IN regression scenarios were adapted from Greengage commit 2ef1914. This PR uses a different implementation and preserves decorrelation for supported cases.

Checklist

• Followed contribution guide
• Added/updated documentation
• Reviewed code for security implications
• This PR contains AI-assisted code generation
• Requested review from cloudberry committers

[yjhjstz]

Thanks for working on this. I built the branch locally (3518a1c7170, cassert, -O0, --enable-pax --with-zstd) on a 3-segment demo cluster and A/B'd it against the same tree with the src/ diff reverted (clean rebuild of gporca/gpopt/cdb each way).

The correctness goal holds up well: the empty-input values for correlated count/regr_count/user-defined aggregates, and the IS NULL / COALESCE(...) = 0 / IS DISTINCT FROM filters built on top of them, are genuinely fixed, and 32 ORCA-heavy regression tests show no expected-output diffs caused by the change.

Two issues I think should be addressed before merge.

---

1. agg_empty_input_is_safe() disables decorrelation for most user-defined aggregates

src/backend/cdb/cdbsubselect.c:899:

safe = !OidIsValid(aggform->aggfinalfn) ||
    (initValueIsNull && func_strict(aggform->aggfinalfn));

Any user-defined aggregate whose final function is not STRICT is classified unsafe. CSubqueryHandler::Psd() (CSubqueryHandler.cpp:1233-1238) then forces m_fCorrelatedExecution, so the correlated subquery loses decorrelation entirely.

Repro on this branch with 20,000 outer x 100,000 inner rows. The two aggregates are identical except for the strictness of the final function:

create function uf(int) returns int as 'select coalesce($1, 0)' language sql;   -- non-strict
create function sf(int) returns int as 'select $1' language sql strict;         -- strict
create aggregate uagg(int) (sfunc = int4pl, stype = int4, finalfunc = uf);
create aggregate sagg(int) (sfunc = int4pl, stype = int4, finalfunc = sf);

select a, (select sagg(bi.b) from bi where bi.a = bo.d) from bo;
select a, (select uagg(bi.b) from bi where bi.a = bo.d) from bo;

sagg (strict final function) decorrelates — 15 ms:

 Gather Motion 3:1  (slice1; segments: 3) (actual rows=20000 loops=1)
   ->  Hash Left Join (actual rows=6800 loops=1)
         Hash Cond: (bo.d = bi.a)
         ->  Redistribute Motion 3:3  (slice2; segments: 3) (actual rows=6800 loops=1)
               ->  Seq Scan on bo (actual rows=6781 loops=1)
         ->  Hash (actual rows=340 loops=1)
               ->  GroupAggregate (actual rows=340 loops=1)
                     ->  Sort (actual rows=34000 loops=1)
                           ->  Seq Scan on bi (actual rows=34000 loops=1)
 Optimizer: GPORCA
 Execution Time: 15.412 ms

uagg (non-strict final function) does not — 56,919 ms:

 Result (actual rows=20000 loops=1)
   ->  Gather Motion 3:1  (slice1; segments: 3) (actual rows=20000 loops=1)
         ->  Seq Scan on bo (actual rows=6781 loops=1)
   SubPlan 1
     ->  Aggregate (actual rows=1 loops=20000)
           ->  Result (actual rows=100 loops=20000)
                 Filter: (bi.a = bo.d)
                 ->  Materialize (actual rows=100000 loops=20000)
                       ->  Gather Motion 3:1  (slice2; segments: 3) (actual rows=100000 loops=1)
                             ->  Seq Scan on bi (actual rows=34000 loops=1)
 Optimizer: GPORCA
 Execution Time: 56918.902 ms

A non-STRICT final function is precisely what you write when an aggregate must return a specific value on empty input — PostGIS ST_Union, hll, tdigest and friends all qualify — so this is not an exotic shape. Could the condition be narrowed (treat the aggregate as unsafe only when the empty-input value genuinely cannot be determined), or at minimum be gated by a GUC so it can be turned off?

---

2. PexprSubstituteColRefs() shares relational subtrees, producing non-disjoint Apply children

CSubqueryHandler.cpp:672-676 — the non-scalar branch shares the child verbatim:

else
{
    pexprChild->AddRef();
    pdrgpexprChildren->Append(pexprChild);
}

and the function is called twice on the same expression to build the matched and the empty CASE branch — lines 845/847 for HAVING, 857/859 for projections. So when a projection or HAVING above the correlated GROUP BY () aggregate contains a subquery, that subquery's entire relational subtree ends up in the result twice with the same colrefs, and the resulting Apply has non-disjoint children.

select a from o where a in (select count(*) + (select max(x) from k) from i where i.a = o.d);

On this branch:

CLogical.cpp:265: Failed assertion: pcrs->IsDisjoint(pcrsChild) && "Input columns are not disjoint"
  gpopt::CLogical::PcrsDeriveOutputCombineLogical + 211
  gpopt::CLogicalLeftOuterApply::DeriveOutputColumns + 116
  ...
  gpopt::CXformSubqueryUnnest::PexprSubqueryUnnest + 1032
  gpopt::CXformSubqueryUnnest::Transform + 69
LOG:  GPORCA failed to produce plan (unexpected)

and the query falls back to the Postgres planner. On the pre-PR baseline (same worktree, src/ diff reverted, clean rebuild) ORCA plans this query without hitting the assertion.

Three more variants reach the same new assertion — these already fell back on the baseline, but on different, pre-existing assertions:

select ... from i where i.a = o.d having count(*) > (select max(x) from k)
select ... from i where i.a = o.d having count(*) in (select x from k)
select count(*) + (case when count(*) in (select x from k) then 1 else 0 end) from i where i.a = o.d

The part that worries me most is that in a release build the assertion is compiled out and ORCA keeps optimizing the malformed expression. Either remap the colrefs when copying the subtree, or bail out of PexprScalarAggApply() when the pulled-up projections/HAVING contain a subquery.

---

I also have a few lower-severity notes — pre-PR minidumps replaying with the wrong IsEmptyInputNull/IsEmptyInputSafe because CMDAggregateGPDB::Serialize() only emits the attributes when they are false while the parse handler defaults them to true; IsPolymorphicType() bailing out on array_agg/string_agg/json_agg, which adds an extra join and slice to a very common query shape; and get_agg_empty_input_const() not checking aggkind, so a user-defined ordered-set aggregate with a strict final function loses evaluation of its direct arguments. Happy to post those separately if they are useful.

[Alena0704]

Thanks for working on this. I built the branch locally (3518a1c7170, cassert, -O0, --enable-pax --with-zstd) on a 3-segment demo cluster and A/B'd it against the same tree with the src/ diff reverted (clean rebuild of gporca/gpopt/cdb each way).

The correctness goal holds up well: the empty-input values for correlated count/regr_count/user-defined aggregates, and the IS NULL / COALESCE(...) = 0 / IS DISTINCT FROM filters built on top of them, are genuinely fixed, and 32 ORCA-heavy regression tests show no expected-output diffs caused by the change.

Two issues I think should be addressed before merge.

1. agg_empty_input_is_safe() disables decorrelation for most user-defined aggregates

src/backend/cdb/cdbsubselect.c:899:

safe = !OidIsValid(aggform->aggfinalfn) ||
    (initValueIsNull && func_strict(aggform->aggfinalfn));

Any user-defined aggregate whose final function is not STRICT is classified unsafe. CSubqueryHandler::Psd() (CSubqueryHandler.cpp:1233-1238) then forces m_fCorrelatedExecution, so the correlated subquery loses decorrelation entirely.

Repro on this branch with 20,000 outer x 100,000 inner rows. The two aggregates are identical except for the strictness of the final function:

create function uf(int) returns int as 'select coalesce($1, 0)' language sql;   -- non-strict
create function sf(int) returns int as 'select $1' language sql strict;         -- strict
create aggregate uagg(int) (sfunc = int4pl, stype = int4, finalfunc = uf);
create aggregate sagg(int) (sfunc = int4pl, stype = int4, finalfunc = sf);

select a, (select sagg(bi.b) from bi where bi.a = bo.d) from bo;
select a, (select uagg(bi.b) from bi where bi.a = bo.d) from bo;

sagg (strict final function) decorrelates — 15 ms:

 Gather Motion 3:1  (slice1; segments: 3) (actual rows=20000 loops=1)
   ->  Hash Left Join (actual rows=6800 loops=1)
         Hash Cond: (bo.d = bi.a)
         ->  Redistribute Motion 3:3  (slice2; segments: 3) (actual rows=6800 loops=1)
               ->  Seq Scan on bo (actual rows=6781 loops=1)
         ->  Hash (actual rows=340 loops=1)
               ->  GroupAggregate (actual rows=340 loops=1)
                     ->  Sort (actual rows=34000 loops=1)
                           ->  Seq Scan on bi (actual rows=34000 loops=1)
 Optimizer: GPORCA
 Execution Time: 15.412 ms

uagg (non-strict final function) does not — 56,919 ms:

 Result (actual rows=20000 loops=1)
   ->  Gather Motion 3:1  (slice1; segments: 3) (actual rows=20000 loops=1)
         ->  Seq Scan on bo (actual rows=6781 loops=1)
   SubPlan 1
     ->  Aggregate (actual rows=1 loops=20000)
           ->  Result (actual rows=100 loops=20000)
                 Filter: (bi.a = bo.d)
                 ->  Materialize (actual rows=100000 loops=20000)
                       ->  Gather Motion 3:1  (slice2; segments: 3) (actual rows=100000 loops=1)
                             ->  Seq Scan on bi (actual rows=34000 loops=1)
 Optimizer: GPORCA
 Execution Time: 56918.902 ms

A non-STRICT final function is precisely what you write when an aggregate must return a specific value on empty input — PostGIS ST_Union, hll, tdigest and friends all qualify — so this is not an exotic shape. Could the condition be narrowed (treat the aggregate as unsafe only when the empty-input value genuinely cannot be determined), or at minimum be gated by a GUC so it can be turned off?

2. PexprSubstituteColRefs() shares relational subtrees, producing non-disjoint Apply children

CSubqueryHandler.cpp:672-676 — the non-scalar branch shares the child verbatim:

else
{
    pexprChild->AddRef();
    pdrgpexprChildren->Append(pexprChild);
}

and the function is called twice on the same expression to build the matched and the empty CASE branch — lines 845/847 for HAVING, 857/859 for projections. So when a projection or HAVING above the correlated GROUP BY () aggregate contains a subquery, that subquery's entire relational subtree ends up in the result twice with the same colrefs, and the resulting Apply has non-disjoint children.

select a from o where a in (select count(*) + (select max(x) from k) from i where i.a = o.d);

On this branch:

CLogical.cpp:265: Failed assertion: pcrs->IsDisjoint(pcrsChild) && "Input columns are not disjoint"
  gpopt::CLogical::PcrsDeriveOutputCombineLogical + 211
  gpopt::CLogicalLeftOuterApply::DeriveOutputColumns + 116
  ...
  gpopt::CXformSubqueryUnnest::PexprSubqueryUnnest + 1032
  gpopt::CXformSubqueryUnnest::Transform + 69
LOG:  GPORCA failed to produce plan (unexpected)

and the query falls back to the Postgres planner. On the pre-PR baseline (same worktree, src/ diff reverted, clean rebuild) ORCA plans this query without hitting the assertion.

Three more variants reach the same new assertion — these already fell back on the baseline, but on different, pre-existing assertions:

select ... from i where i.a = o.d having count(*) > (select max(x) from k)
select ... from i where i.a = o.d having count(*) in (select x from k)
select count(*) + (case when count(*) in (select x from k) then 1 else 0 end) from i where i.a = o.d

The part that worries me most is that in a release build the assertion is compiled out and ORCA keeps optimizing the malformed expression. Either remap the colrefs when copying the subtree, or bail out of PexprScalarAggApply() when the pulled-up projections/HAVING contain a subquery.

I also have a few lower-severity notes — pre-PR minidumps replaying with the wrong IsEmptyInputNull/IsEmptyInputSafe because CMDAggregateGPDB::Serialize() only emits the attributes when they are false while the parse handler defaults them to true; IsPolymorphicType() bailing out on array_agg/string_agg/json_agg, which adds an extra join and slice to a very common query shape; and get_agg_empty_input_const() not checking aggkind, so a user-defined ordered-set aggregate with a strict final function loses evaluation of its direct arguments. Happy to post those separately if they are useful.

Thank you! Agree with this. I need some time to figure out how to address these issues

[Alena0704]

Thanks for working on this. I built the branch locally (3518a1c7170, cassert, -O0, --enable-pax --with-zstd) on a 3-segment demo cluster and A/B'd it against the same tree with the src/ diff reverted (clean rebuild of gporca/gpopt/cdb each way).

The correctness goal holds up well: the empty-input values for correlated count/regr_count/user-defined aggregates, and the IS NULL / COALESCE(...) = 0 / IS DISTINCT FROM filters built on top of them, are genuinely fixed, and 32 ORCA-heavy regression tests show no expected-output diffs caused by the change.

Two issues I think should be addressed before merge.

1. agg_empty_input_is_safe() disables decorrelation for most user-defined aggregates

src/backend/cdb/cdbsubselect.c:899:

safe = !OidIsValid(aggform->aggfinalfn) ||
    (initValueIsNull && func_strict(aggform->aggfinalfn));

Any user-defined aggregate whose final function is not STRICT is classified unsafe. CSubqueryHandler::Psd() (CSubqueryHandler.cpp:1233-1238) then forces m_fCorrelatedExecution, so the correlated subquery loses decorrelation entirely.

Repro on this branch with 20,000 outer x 100,000 inner rows. The two aggregates are identical except for the strictness of the final function:

create function uf(int) returns int as 'select coalesce($1, 0)' language sql;   -- non-strict
create function sf(int) returns int as 'select $1' language sql strict;         -- strict
create aggregate uagg(int) (sfunc = int4pl, stype = int4, finalfunc = uf);
create aggregate sagg(int) (sfunc = int4pl, stype = int4, finalfunc = sf);

select a, (select sagg(bi.b) from bi where bi.a = bo.d) from bo;
select a, (select uagg(bi.b) from bi where bi.a = bo.d) from bo;

sagg (strict final function) decorrelates — 15 ms:

 Gather Motion 3:1  (slice1; segments: 3) (actual rows=20000 loops=1)
   ->  Hash Left Join (actual rows=6800 loops=1)
         Hash Cond: (bo.d = bi.a)
         ->  Redistribute Motion 3:3  (slice2; segments: 3) (actual rows=6800 loops=1)
               ->  Seq Scan on bo (actual rows=6781 loops=1)
         ->  Hash (actual rows=340 loops=1)
               ->  GroupAggregate (actual rows=340 loops=1)
                     ->  Sort (actual rows=34000 loops=1)
                           ->  Seq Scan on bi (actual rows=34000 loops=1)
 Optimizer: GPORCA
 Execution Time: 15.412 ms

uagg (non-strict final function) does not — 56,919 ms:

 Result (actual rows=20000 loops=1)
   ->  Gather Motion 3:1  (slice1; segments: 3) (actual rows=20000 loops=1)
         ->  Seq Scan on bo (actual rows=6781 loops=1)
   SubPlan 1
     ->  Aggregate (actual rows=1 loops=20000)
           ->  Result (actual rows=100 loops=20000)
                 Filter: (bi.a = bo.d)
                 ->  Materialize (actual rows=100000 loops=20000)
                       ->  Gather Motion 3:1  (slice2; segments: 3) (actual rows=100000 loops=1)
                             ->  Seq Scan on bi (actual rows=34000 loops=1)
 Optimizer: GPORCA
 Execution Time: 56918.902 ms

A non-STRICT final function is precisely what you write when an aggregate must return a specific value on empty input — PostGIS ST_Union, hll, tdigest and friends all qualify — so this is not an exotic shape. Could the condition be narrowed (treat the aggregate as unsafe only when the empty-input value genuinely cannot be determined), or at minimum be gated by a GUC so it can be turned off?

2. PexprSubstituteColRefs() shares relational subtrees, producing non-disjoint Apply children

CSubqueryHandler.cpp:672-676 — the non-scalar branch shares the child verbatim:

else
{
    pexprChild->AddRef();
    pdrgpexprChildren->Append(pexprChild);
}

and the function is called twice on the same expression to build the matched and the empty CASE branch — lines 845/847 for HAVING, 857/859 for projections. So when a projection or HAVING above the correlated GROUP BY () aggregate contains a subquery, that subquery's entire relational subtree ends up in the result twice with the same colrefs, and the resulting Apply has non-disjoint children.

select a from o where a in (select count(*) + (select max(x) from k) from i where i.a = o.d);

On this branch:

CLogical.cpp:265: Failed assertion: pcrs->IsDisjoint(pcrsChild) && "Input columns are not disjoint"
  gpopt::CLogical::PcrsDeriveOutputCombineLogical + 211
  gpopt::CLogicalLeftOuterApply::DeriveOutputColumns + 116
  ...
  gpopt::CXformSubqueryUnnest::PexprSubqueryUnnest + 1032
  gpopt::CXformSubqueryUnnest::Transform + 69
LOG:  GPORCA failed to produce plan (unexpected)

and the query falls back to the Postgres planner. On the pre-PR baseline (same worktree, src/ diff reverted, clean rebuild) ORCA plans this query without hitting the assertion.

Three more variants reach the same new assertion — these already fell back on the baseline, but on different, pre-existing assertions:

select ... from i where i.a = o.d having count(*) > (select max(x) from k)
select ... from i where i.a = o.d having count(*) in (select x from k)
select count(*) + (case when count(*) in (select x from k) then 1 else 0 end) from i where i.a = o.d

The part that worries me most is that in a release build the assertion is compiled out and ORCA keeps optimizing the malformed expression. Either remap the colrefs when copying the subtree, or bail out of PexprScalarAggApply() when the pulled-up projections/HAVING contain a subquery.

I also have a few lower-severity notes — pre-PR minidumps replaying with the wrong IsEmptyInputNull/IsEmptyInputSafe because CMDAggregateGPDB::Serialize() only emits the attributes when they are false while the parse handler defaults them to true; IsPolymorphicType() bailing out on array_agg/string_agg/json_agg, which adds an extra join and slice to a very common query shape; and get_agg_empty_input_const() not checking aggkind, so a user-defined ordered-set aggregate with a strict final function loses evaluation of its direct arguments. Happy to post those separately if they are useful.

Hi,

I've reworked the implementation since your review of 3518a1c7170. The goal remains the same: preserve aggregate results on empty input while allowing supported queries to decorrelate.

The approach builds on ORCA's existing scalar-subquery handling, which already restores COUNT's zero result on empty input. The original COUNT/IN bug exposed a gap in applying the same semantics to IN/ANY. I extended the scalar-subquery handling and reused it for supported single-row aggregate IN/ANY queries.

I removed the additional metadata describing aggregate behavior on empty input. COUNT and built-in MIN/MAX use known empty-input values; other aggregates are evaluated on empty input only when no matching group exists. These results are preserved before evaluating expressions, HAVING, and surrounding predicates.

Nested subqueries in projections or HAVING retain correlated execution, avoiding the shared-column-reference assertion reported in the review. IN/ANY forms that can return no rows retain quantified semantics.

The PR now contains three commits:

1. 275e1351647 — fix join selectivity for fractional row estimates.
2. 9490dccff55 — preserve scalar aggregate results, including HAVING and filters that accept NULL.
3. c47d048beb4 — reuse scalar aggregate handling for IN/ANY while preserving empty-result and NULL semantics.

While testing nested aggregate subqueries, I also found a separate cardinality-estimation bug: fractional row estimates could produce join scale factors below one, making selectivity exceed one or triggering an assertion. The first commit fixes the affected scale factors and their bound.

Examples, results, and query plans before and after the fix are included in the PR description.

[Alena0704]

Since my previous update, I narrowed the fix to avoid changing queries that do not need empty-input handling.
Simple built-in SUM, AVG, MIN, and MAX now keep their previous plans. The existing handling of ANY/ALL and COUNT with correlated HAVING is also preserved.
I added protection for volatile functions in separate commit (second commit), including aggregate FINALFUNC. These subqueries must not share one result between different outer rows. The general volatile-function check is in a separate commit before the aggregate fixes.

Orca unlike Postgres optimizer can evaluate a volatile expression for rows that do not match the correlated subquery, causing an unexpected error. Bug reproduction:
SET optimizer = on;

CREATE TABLE o(k int) DISTRIBUTED BY (k);
CREATE TABLE i(k int, b int) DISTRIBUTED BY (k);

INSERT INTO o VALUES (1);
INSERT INTO i VALUES (1, 1), (2, 0);
ANALYZE o;
ANALYZE i;

SELECT k FROM o
WHERE 0 IN (
    SELECT floor(random() / i.b)::int
    FROM i
    WHERE i.k = o.k
);

Only (1, 1) should match, so the query should return 1.
Before the fix, ORCA turns the subquery into a Hash Semi Join. It evaluates random() / b before checking the join condition, including for the unrelated row (2, 0). This causes: ERROR: division by zero

postgres=# SELECT k FROM o
postgres-# WHERE 0 IN (
postgres(#     SELECT floor(random() / i.b)::int
postgres(#     FROM i
postgres(#     WHERE i.k = o.k
postgres(# );
ERROR:  division by zero  (seg0 slice1 127.0.0.1:6002 pid=66392)
postgres=# explain SELECT k FROM o
postgres-# WHERE 0 IN (
postgres(#     SELECT floor(random() / i.b)::int
postgres(#     FROM i
postgres(#     WHERE i.k = o.k
postgres(# );
                                         QUERY PLAN                                         
--------------------------------------------------------------------------------------------
 Gather Motion 3:1  (slice1; segments: 3)  (cost=0.00..862.00 rows=1 width=4)
   ->  Hash Semi Join  (cost=0.00..862.00 rows=1 width=4)
         Hash Cond: (o.k = i.k)
         ->  Seq Scan on o  (cost=0.00..431.00 rows=1 width=4)
               Filter: (NOT (k IS NULL))
         ->  Hash  (cost=431.00..431.00 rows=1 width=4)
               ->  Result  (cost=0.00..431.00 rows=1 width=4)
                     Filter: (0 = ((floor((random() / (i.b)::double precision)))::integer))
                     ->  Seq Scan on i  (cost=0.00..431.00 rows=1 width=8)
 Optimizer: GPORCA
(10 rows)

postgres=# set optimizer=off
postgres-# ;
SET
postgres=# explain SELECT k FROM o
postgres-# WHERE 0 IN (
postgres(#     SELECT floor(random() / i.b)::int
postgres(#     FROM i
postgres(#     WHERE i.k = o.k
postgres(# );
                                               QUERY PLAN                                                
---------------------------------------------------------------------------------------------------------
 Gather Motion 3:1  (slice1; segments: 3)  (cost=0.00..1.03 rows=1 width=4)
   ->  Seq Scan on o  (cost=0.00..1.01 rows=1 width=4)
         Filter: (SubPlan 1)
         SubPlan 1
           ->  Result  (cost=0.00..1.09 rows=2 width=4)
                 Filter: (i.k = o.k)
                 ->  Materialize  (cost=0.00..1.05 rows=2 width=8)
                       ->  Broadcast Motion 3:3  (slice2; segments: 3)  (cost=0.00..1.04 rows=2 width=8)
                             ->  Seq Scan on i  (cost=0.00..1.01 rows=1 width=8)
 Optimizer: Postgres query optimizer
(10 rows)

postgres=# SELECT k FROM o
postgres-# WHERE 0 IN (
postgres(#     SELECT floor(random() / i.b)::int
postgres(#     FROM i
postgres(#     WHERE i.k = o.k
postgres(# );
 k 
---
 1
(1 row)

After the fix, ORCA keeps a correlated SubPlan, checks i.k = o.k first, and returns 1.