fix(arrays): a spread inside an associative literal is no longer discarded by the parser - #1111
Guikingone wants to merge 1 commit into
Conversation
…arded by the parser
$idx = [3, 4];
$a = [...$idx, "c" => 8];
count($a); // was 1 -- the spread's entries were gone, silently
THE SPREAD NEVER REACHED THE AST.
The issue points at `lower_hash_spread_into_hash_from_value`'s `Op::ArrayToHash`
promotion. That helper is never called for this shape: `--emit-ir` shows `hash_new
capacity=1` and no `hash_spread` at all.
`ExprKind::ArrayLiteralAssoc` is a list of KEY/VALUE pairs and a spread has no key, so the
parser dropped it, in two places. The ellipsis arm pushes the spread only `if !is_assoc`,
and `promote_indexed_array_items_to_assoc` -- which runs when a `=>` turns an indexed
literal associative -- `continue`s on a `Spread` element. The second is how
`[...$idx, "c" => 8]` loses it: the spread is parsed while the literal is still indexed,
and the promotion throws it away when the `=>` arrives.
That also makes the issue's perimeter too narrow. It reports an INDEXED source; every
source kind was dropped, including an associative one that skips the promotion entirely:
[...["a" => 1, "b" => 2], "c" => 8] // was 1, PHP 3
[...[5 => 7], "c" => 8] // was 1, PHP 2
[...["x", "y"], "c" => "z"] // was 1, PHP 3
A spread is now carried as a pair whose BOTH halves are the `Spread` marker.
`ArrayLiteralAssoc` keeps its shape, because giving it an item enum is a 145-reference
refactor across some 110 files, and `Spread` is an ordinary `ExprKind` that every generic
key/value walker already meets inside an indexed literal -- so none of them needs a new
arm, and the duplication only makes such a walker visit the source twice, which is
conservative. Nothing evaluates the key: the checker's inference,
`assoc_array_literal_type_for_ir` and `lower_assoc_array_literal` each check the value for
`Spread` first, and the variant's own doc says so.
A KEY COMPUTED AT PARSE TIME CANNOT FOLLOW A SPREAD.
How many keys a spread contributes is a runtime fact, so the automatic key assigned to an
unkeyed element after one is always a guess. `[1, ...[3, 4], 5, "c" => 8]` gave the `5` key
1 and overwrote the spread's first entry -- four entries where PHP has five. Such an element
is now spread as a one-element array instead, which asks the runtime for the same next-free
integer key PHP uses. That is the only place the number is known.
THE SECOND HALF: A CONSUMING CONVERSION, HANDED A BORROWED SOURCE.
`Op::ArrayToHash` consumes its source -- the conversion routes an indexed array through
`__rt_array_hash_union` and then `__rt_decref_array`s the input, which the emitted assembly
shows plainly. That is right where the promotion replaces a local's own value, and wrong
for a spread. Once the spread actually ran, the source was freed under the caller:
$idx = [3, 4];
$a = [...$idx, "c" => 8];
count($idx); // 0
So the promotion is given a reference of its own, but only when the source is read out of
storage that KEEPS ITS OWN REFERENCE. That takes two facts, and all three reviewers found
the first cut short of one or both.
The op is not enough. `take_owned_temp` loads a hidden `OwnedTemp` slot and clears it
WITHOUT releasing, so `[...($c ? [1, 2] : [3, 4]), "c" => 8]` hands over the only reference
there is: acquiring another leaked one array per evaluation, 40 live blocks over 40
iterations. The slot's KIND is what separates that from a user local's load.
The list is not enough either. A static property is read with no retain and the symbol goes
on owning what it holds -- the same shape as `LoadStaticLocal` under a different name --
so leaving `LoadStaticProperty` out freed the class's array: `count(C::$stat)` after
`[...C::$stat, "c" => 8]` answered 0, and spreading it twice was a double free.
`value_is_owning_temporary` cannot make this call either way: it answers TRUE for a plain
`load_local` of an array by design, as a PROVISIONAL owner whose release the builder prunes
later if the slot stays concrete. That machinery exists to make a release safe, not an
acquire.
Two reviewer findings did NOT reproduce, and are recorded here because the reasoning behind
them was sound: a typed instance-property source (`[...$o->items, "c" => 8]`) and a
nested-literal source (`[...[...$idx, "k" => 1], "c" => 8]`) both match host PHP and are
heap-clean. Both reviewers flagged their own confidence as medium or low and named the
emitter they had not read; the property read does materialize an owned value.
The destination's value type now also says what the spread INSERTS rather than what the
source declares: the promotion hardcodes `AssocArray<Int, Mixed>`, so an indexed source
contributes `Mixed`-tagged entries whatever its element type was.
19 shapes measured against host PHP 8.5.10, including PHP's renumbering rather than an
approximation of it: `[...[5 => 7], "c" => 8]` answers at key `0`, and `array_keys()` gives
`[0, 'c']` on both sides. Unchanged: an indexed spread into an indexed literal, and an
associative spread into one -- both already worked and take different helpers.
The issue's SECOND symptom is not fixed and is not this: `new $k(...$intkeyed, ...$idx)`
does not compile on this base at all, with `EIR validation failed: OperandTypeMismatch
{ expected: "Heap(Array)", actual: Heap(Hash) }` and the identical InstId before and after
this change -- verified by reverting the whole patch, rebuilding, and re-running. The issue
reports a segfault there, so that shape changed between `c59e49b5de` and `89a923f4b4` for
unrelated reasons.
Closes illegalstudio#1049
Claude-Session: https://claude.ai/code/session_01KSAAWPyNBq6dP2b5puN3wr
|
| if is_assoc { | ||
| // An associative literal is a list of KEY/VALUE pairs and a spread has no key: it | ||
| // carries its own. Dropping it here is what made `[...$src, "c" => 8]` answer with | ||
| // only the string-keyed entry -- silently, and for every source kind, not just the | ||
| // indexed one the issue reports (#1049). | ||
| // | ||
| // It is carried as a pair whose BOTH halves are the spread marker. `Spread` is an | ||
| // ordinary `ExprKind` that every generic walker already meets inside an indexed | ||
| // literal, so none of them needs a new arm; the duplication only makes such a | ||
| // walker visit the source twice, which is conservative. Nothing evaluates the key: | ||
| // each consumer that types or lowers a pair checks the value for `Spread` first. | ||
| assoc_elems.push((spread.clone(), spread)); |
There was a problem hiding this comment.
Trailing elements overwrite spreads
When a spread appears after the literal has already become associative, a following unkeyed element still receives the parser's static next_auto_key. For ["c" => 8, ...[3, 4], 5], the spread inserts runtime keys 0 and 1, but the trailing 5 is then stored at key 0, overwriting 3. The result has three entries instead of PHP's four. Unkeyed elements parsed after an associative-position spread need runtime key assignment, as the promotion path already does for earlier positional elements.
Prompt To Fix With AI
This is a comment left during a code review.
Path: src/parser/expr/prefix.rs
Line: 529-540
Comment:
**Trailing elements overwrite spreads**
When a spread appears after the literal has already become associative, a following unkeyed element still receives the parser's static `next_auto_key`. For `["c" => 8, ...[3, 4], 5]`, the spread inserts runtime keys `0` and `1`, but the trailing `5` is then stored at key `0`, overwriting `3`. The result has three entries instead of PHP's four. Unkeyed elements parsed after an associative-position spread need runtime key assignment, as the promotion path already does for earlier positional elements.
---
For each issue above, determine whether it is valid and should be fixed. If so, fix it directly.
The spread never reached the AST
The issue points at
lower_hash_spread_into_hash_from_value'sOp::ArrayToHashpromotion. Thathelper is never called for this shape:
--emit-irshowshash_new capacity=1and nohash_spreadat all.ExprKind::ArrayLiteralAssocis a list of key/value pairs and a spread has no key, so the parserdropped it — in two places. The ellipsis arm pushes only
if !is_assoc, andpromote_indexed_array_items_to_assoc— which runs when a=>turns an indexed literalassociative —
continues on aSpreadelement. The second is how[...$idx, "c" => 8]loses it.So the issue's perimeter is too narrow. It reports an indexed source; every source kind was
dropped, including an associative one that skips the promotion entirely:
A spread is now carried as a pair whose both halves are the
Spreadmarker.ArrayLiteralAssockeeps its shape, because giving it an item enum is a 145-reference refactoracross ~110 files — and
Spreadis an ordinaryExprKindthat every generic key/value walkeralready meets inside an indexed literal, so none of them needs a new arm and the duplication only
makes such a walker visit the source twice, which is conservative.
A key computed at parse time cannot follow a spread
How many keys a spread contributes is a runtime fact, so an automatic key assigned to an unkeyed
element after one is always a guess:
Such an element is now spread as a one-element array, which asks the runtime for the same
next-free-integer key PHP uses. That is the only place the number is known.
A consuming conversion, handed a borrowed source
Op::ArrayToHashconsumes its source — the conversion routes an indexed array through__rt_array_hash_unionand then__rt_decref_arrays the input, which the emitted assembly showsplainly. Right where the promotion replaces a local's own value; wrong for a spread. Once the
spread actually ran, the source was freed under the caller:
count($idx)answered 0.So the promotion gets a reference of its own — but only when the source is read out of storage that
keeps its own reference. That takes two facts, and all three reviewers found the first cut
short of one or both:
[...($c ? [1, 2] : [3, 4]), "c" => 8]take_owned_temploads a hiddenOwnedTempand clears it without releasing, so it hands over the only reference; acquiring another leaked one array per evaluation — 40 live blocks over 40 iterations[...C::$stat, "c" => 8]LoadStaticLocalunder another name — so omittingLoadStaticPropertyfreed the class's array, and spreading it twice was a double freevalue_is_owning_temporarycannot make this call either way: it answers TRUE for a plainload_localof an array by design, as a provisional owner whose release the builder prunes laterif the slot stays concrete. That machinery exists to make a release safe, not an acquire.
Both shapes were reproduced against host PHP before the fix and after it, and both are now
regression tests.
Two reviewer findings did not reproduce, recorded because the reasoning was sound: a typed
instance-property source (
[...$o->items, "c" => 8]) and a nested-literal source(
[...[...$idx, "k" => 1], "c" => 8]) both match host PHP and are heap-clean. Both reviewersflagged their own confidence as medium or low and named the emitter they had not read; the property
read does materialize an owned value.
Verification
19 shapes measured against host PHP 8.5.10, aarch64. PHP's renumbering is matched rather than
approximated:
[...[5 => 7], "c" => 8]answers at key0, andarray_keys()gives[0, 'c']onboth sides.
Unchanged: an indexed spread into an indexed literal, and an associative spread into one — both
already worked and take different helpers.
All thirteen suites green on this commit, 6848 tests:
assoc_literal_spreadsarrays::callables::oop::runtime_gc::strings::regressions::closures::generators::spl::evalerror_testsparser_testsNine behaviour tests in
tests/codegen/arrays/assoc_literal_spreads.rs, six heap-debug tests intests/codegen/runtime_gc/assoc_literal_spreads.rs— the source kinds separately, because they siton opposite sides of the retain gate.
Known not fixed, reported on the issue
The issue's second symptom —
new $k(...$intkeyed, ...$idx)— does not compile on this base atall:
EIR validation failed: OperandTypeMismatch { expected: "Heap(Array)", actual: Heap(Hash) },with the identical InstId before and after this change. I verified that by reverting the whole
patch, rebuilding and re-running. The issue reports a segfault there, so that shape changed between
c59e49b5deand89a923f4b4for unrelated reasons.Closes #1049.
🤖 Generated with Claude Code
https://claude.ai/code/session_01KSAAWPyNBq6dP2b5puN3wr