fix(gfql): one Cypher type contract for sum()/avg() across pandas, polars and cuDF

[lmeyerov]

The defect

Two aggregate divergences between the pandas and polars engines, wrong in opposite directions:

query	pandas (before)	polars (before)
avg(<string column>)	GFQLTypeError	silent null
sum(<string column>)	'abac' (string CONCATENATION)	raw polars.exceptions.InvalidOperationError

Both are pre-existing on engine='polars'. They matter now because #1743 changes engine=auto to
resolve to polars by default, which would promote the silent null to every user who never
specifies an engine — a typed error degrading into a quiet wrong answer, by default.

Because the two engines are wrong in opposite directions, "make polars match pandas" was not
available. The contract had to be established first.

The intended contract, and its source

graphistry/compute/gfql/agg_types.py carries the sources in its module docstring. In short:

• avg(input) / sum(input) accept INTEGER | FLOAT | DURATION (and null) ONLY.
  Neo4j declares exactly that signature for both
  (neo4j/docs-cypher modules/ROOT/pages/functions/aggregating.adoc:
  "Returns the average of a set of INTEGER, FLOAT or DURATION values", input : INTEGER | FLOAT | DURATION),
  and enforces it at runtime. Verified live against two independent implementations:

  # Neo4j 5.26.26 (docker, cypher-shell)
  UNWIND [{s:'a'},{s:'b'}] AS r RETURN avg(r.s);
    -> AVG(...) can only handle numerical values, duration, or null.
  UNWIND [{s:'a'},{s:'b'}] AS r RETURN sum(r.s);
    -> SUM(...) can only handle numerical values, duration, or null.
  UNWIND [date('2020-01-01')] AS x RETURN sum(x);
    -> Type mismatch: expected Float, Integer or Duration but was Date
  UNWIND [null,null] AS x RETURN sum(x), avg(x);   -> 0, NULL
  UNWIND [] AS x RETURN sum(x), avg(x);            -> 0, NULL
  
  # Kuzu 0.11.3 (openCypher, embedded)
  MATCH (p:P) RETURN avg(p.s)  -> Binder exception: Function AVG did not receive correct
                                  arguments: Actual: (STRING)
  MATCH (p:P) RETURN sum(p.s)  -> Binder exception: Function SUM ... Actual: (STRING)
  MATCH (p:P) RETURN min(p.s)  -> 'a'      # min/max/count/collect take ANY
  

• min / max / count / collect accept ANY (same doc: input : ANY; openCypher TCK
  Aggregation2 scenarios [7]–[12] cover min()/max() over strings, lists and mixed values).

• Nulls are excluded from every aggregate; sum over an empty-or-all-null set is 0 and avg
  over one is null (Neo4j "Considerations", confirmed live above).

GFQL's own conventions agree: pandas already raised for avg(<string>), and GFQLTypeError is the
established class for this failure.

So sum(<string>) == 'abac' on pandas was itself the wrong contract, and this PR changes it.
That is a deliberate, called-out behaviour change on the pandas engine, not a side effect: string
concatenation has no meaning in Cypher, it is a silent wrong answer to a question the user asked
about numbers, and it was already inconsistent with avg(), which raised.

The divergence table (the real deliverable)

Full sweep: 8 aggregates × 9 dtypes × {grouped, whole-table} × {pandas, polars, cuDF} —
288 cells. Repro harness reproduced as graphistry/tests/compute/gfql/test_aggregate_type_contract.py.

24 divergences before, 0 after. Grouped by cause:

aggregate	dtype	pandas (before)	polars (before)	cuDF (before)	after (all engines)
avg	string / nullable-string	GFQLTypeError	null	GFQLTypeError	GFQLTypeError E302
avg	categorical	GFQLTypeError	null	GFQLTypeError	GFQLTypeError E302
avg	datetime	mean timestamp	GFQLTypeError	GFQLTypeError	GFQLTypeError E302
sum	string / nullable-string	'abac'	raw polars.InvalidOperationError	GFQLTypeError	GFQLTypeError E302
sum	datetime / categorical	GFQLTypeError	null	GFQLTypeError	GFQLTypeError E302
sum	all-null	0	raw polars.InvalidOperationError	GFQLTypeError	0
avg	all-null	null	null	GFQLTypeError	null
min / max	categorical	GFQLTypeError	ok	ok	ok (lexicographic)
collect, collect(DISTINCT)	categorical	ok	ok	GFQLTypeError	ok
count(DISTINCT)	categorical	2, 3	2, 3	'c', 'd' — a category LABEL instead of a count (and ValueError whole-table)	2, 3

Bold = silent wrong answer. Note the last row: the sweep turned up a second quiet-wrong-answer
class on cuDF that nobody was looking for.

What changed

One shared contract module, graphistry/compute/gfql/agg_types.py, holding the Cypher sources, the
sum/avg aggregate set, the two engine dtype classifiers, the all-null substitution value, and
one raiser — so the error class, ErrorCode and message cannot drift between engines. The
aggregate kernels are written three times, and all three now consult it:

1. graphistry/compute/gfql/row/pipeline.py — pandas/cuDF row pipeline
2. graphistry/compute/gfql/lazy/engine/polars/row_pipeline.py — native polars row pipeline
3. graphistry/compute/gfql_fast_paths.py — OLAP single-hop grouped fast path (both of its
   engine branches; it reimplements the aggregates, so without a guard there whether an ill-typed
   aggregate was caught depended on whether the query happened to match the fast-path shape)

Plus:

• No raw polars.exceptions.* escapes the GFQL surface. On the pandas side execute_call
  already wraps any kernel exception as GFQLTypeError(E303); the native polars path runs before
  execute_call and skipped that wrapper entirely. Native row ops now funnel polars errors through
  the same wrapper with the same code and message shape, preserving the polars text as the cause.

• Categoricals are decategorized once, up front, on both engines. Cypher treats a categorical as
  a string column; the host kernels did not agree with each other about it. This also dodges a
  polars 1.35.2 rust panic (categorical.rs:350 not implemented) on a grouped min/max over a
  Categorical — it surfaces as a pyo3_runtime.PanicException, which is not a polars exception, so
  no python-side wrapper can catch it. Found by the GPU lane on the RAPIDS 26.02 image.

• Error messages name the operation and the user's output alias (the cypher lowering
  materializes the aggregate's argument into an internal __cypher_agg__ column, so naming the raw
  column would hand the user a name that appears nowhere in their query):

  [incompatible-column-type] Aggregation avg() requires numeric or duration values, but the
  argument of 'mean_s' has type String | field: __cypher_agg__ | value: 'String' | suggestion:
  Cypher restricts avg() to INTEGER/FLOAT/DURATION; use count()/collect()/min()/max() over the
  argument of 'mean_s', or cast it to a number
  

Cost — not assumed, ordered for it

The dtype verdict is O(1) (a dtype check) and runs first; the O(n) all-null scan is consulted
only for a column the dtype has already rejected, i.e. only on queries that were previously wrong
or erroring. A served numeric aggregate pays one dtype string check and nothing else.

Measured, 200k nodes / 400k edges, 12 reps, two interleaved base/head rounds:

shape	base	head
pandas row-pipeline group+agg	85.3 / 86.3 ms	87.7 / 85.8 ms
pandas OLAP fast path	110.9 / 108.2 ms	111.5 / 112.4 ms
polars row-pipeline group+agg	3.00 / 2.81 ms	3.01 / 3.06 ms
polars OLAP fast path	19.4 / 19.4 ms	18.5 / 18.8 ms

Round-to-round spread within each arm is ~±3%, and every delta sits inside it — this design does
not resolve anything smaller, and no smaller claim is made.

Tests

graphistry/tests/compute/gfql/test_aggregate_type_contract.py, parametrized over
pandas / polars / cudf / polars-gpu with named skips (not available_nonpandas_engines(), which
silently shrinks the parametrization when a stack is missing):

• negative lane — what must raise, which typed error, and that the message names the operation
  and the alias; plus that no polars.exceptions.* class reaches the caller
• positive lane — int / float / bool / duration still aggregate; min/max/count/collect
  still accept strings and categoricals; all-null → 0/null; a 0-row frame is not treated as
  all-null (that path must keep its source dtype)
• differential matrix — the whole aggregate × dtype cross-product against the pandas oracle, on
  value and error class
• fast-path lane — with a spy asserting the fast path is actually entered, so it cannot
  silently degrade into testing the ordinary row pipeline

Where they run: the file is under graphistry/tests/compute/, so the main lane collects it, and
it is added to bin/test-polars.sh — the native polars aggregate guard and the polars-error wrap
run in no other lane. (Note #1805, open, widens that script further.)

GPU receipt — dgx-spark GB10, --gpus all, graphistry/test-rapids-official:26.02-gfql-polars
(cudf 26.02.01 / polars 1.35.2 / cudf_polars available), via the dgx safety harness:

cudf 26.02.01 polars 1.35.2 cudf_polars True
403 passed in 9.48s

That run is what surfaced the polars 1.35.2 categorical panic (10 failures before the decategorize
fix, 0 after) — the local polars 1.42 cannot see it.

Mutation checks — 10/10 detected. Each fix reverted individually, suite re-run:

mutation	outcome
polars sum/avg dtype guard removed	DETECTED
pandas sum/avg dtype guard removed	DETECTED
OLAP fast-path polars guard removed	DETECTED
OLAP fast-path pandas guard removed	DETECTED
polars all-null substitution reverted to kernel	DETECTED
polars Null-dtype substitution reverted	DETECTED
pandas all-null substitution reverted	DETECTED
pandas categorical decategorize removed	DETECTED
polars categorical decategorize removed	DETECTED
raw-polars-exception wrap removed	DETECTED

Two of these were not detected on the first pass — the fast-path guards and the polars-error wrap
were being covered by tests that never reached them (the fast path declines the query shape I first
wrote, and the dtype guard now prevents the one polars error the wrap was known to catch). Both were
rewritten until they fail on revert.

Verification

• graphistry/tests/compute: 7095 passed; failure set byte-identical to origin/master (233b64c)
  — 123 pre-existing failures from missing optional deps (umap/dask/cupy), 0 new
• bin/test-polars.sh: 0 failures not also present on the base commit
• bin/lint.sh clean, bin/typecheck.sh clean (mypy: no issues in 326 source files)
• No Any / cast / getattr / setattr / bare list / Dict[str, Any] introduced

Filed, not fixed

Siblings the sweep found that are out of this PR's scope, each filed with a precise repro:

• GFQL accepts sum()/avg() over BOOLEAN, which Cypher rejects — extension or conformance gap? #1820 — sum()/avg() over BOOLEAN is accepted by GFQL on every engine but is a type error in Neo4j. Left in place deliberately (both engines already agree; summing an indicator column is idiomatic in the dataframe surface GFQL also serves) and recorded in agg_types.py as a choice with a reason. Needs an owner call: strict conformance, or document the extension.
• OLAP single-hop grouped fast path: sole min()/max() won't lower, shape is fast-path-only, and pandas branch returns mixed dtypes #1821 — OLAP single-hop grouped fast path: a SOLE min()/max() aggregate fails to lower at all (add a second aggregate and it works); the labelled-pattern shape is served ONLY by the fast path, so any future decline turns a working query into a KeyError rather than a slower correct answer; and its pandas branch returns int64 as float64 and sum(<bool>) as mixed False/2 in one column.
• polars 1.35.2 (RAPIDS 26.02 image) rust-panics on grouped min/max over Categorical — a pyo3 PanicException no GFQL wrapper can catch #1822 — polars 1.35.2 (the RAPIDS 26.02 image) rust-panics on a grouped min/max over a Categorical. Worked around here by decategorizing, but a pyo3_runtime.PanicException is not a polars exception, so the new wrapper cannot catch it — the general hazard needs its own survey, as does the 1.35.2-vs-1.42 version drift that made this invisible locally.

Does this unblock #1743 on this axis?

Yes. The avg(<string>) silent null and the sum(<string>) raw-exception leak are gone on
engine='polars', independently of the routing change: this branch is off origin/master
(233b64c) and #1743 does not need to carry it. On the aggregate-type axis specifically, routing
engine=auto to polars no longer changes any answer — the 288-cell sweep is identical across
pandas, polars and cuDF. This says nothing about #1743's other open axes (the postload/postchain
policy-hook gap, the frame-type and dtype contract changes).

🤖 Generated with Claude Code

https://claude.ai/code/session_015YsqAZQLbqjSDrYSFz2GoB