Skip to content

feat(ts)!: rs.ArraySlice for slices of arrays, replacing the as_chunks_mut proxies - #608

Merged
moq-bot[bot] merged 22 commits into
mainfrom
quest/m3/ts-slice-object
Nov 7, 2026
Merged

moq-bot[bot] merged 22 commits into
mainfrom
quest/m3/ts-slice-object

Conversation

@kixelated

@kixelated kixelated commented Nov 6, 2026 •

Copy link
Copy Markdown
Collaborator

Part 3 of quest/m3/ts-slice-object.md, after moq-dev/rust2#576 and moq-dev/rust2#584.

Problem

A slice of arrays had each array as a JS array of its own, so the arrays as_chunks_mut and as_rchunks_mut lend from a flat slice had no JS array to be. They were a Proxy (Runs, ByteRuns) whose elements were arrayView proxies or subarray views of each array, made and cached on first read, with detaching copies before a whole write. In a micro-bench of the codec bench's packets workload, that cost 10-20x reading each array where the flat buffer holds it.

Approach

  • A new runtime type, rs.ArraySlice<S>, is every &[[T; N]], &mut [[T; N]], &[[T; N]; M], and Box<[[T; N]]>. Element i is the slice of array i: an rs.Slice<T>, an rs.ByteSlice for bytes, or an rs.ArraySlice for arrays of arrays. It reads and writes array i where it is, whether each array is a JS array of its own (items and the of that lends one, as a Vec<[T; N]> or [[T; N]; M] holds them) or n elements of one flat slice (as as_chunks splits it). No Proxy.
  • std's methods that lend arrays or parts of the slice (iter/iter_mut, split_at_mut, chunks_mut, windows, split*, first_chunk*, swap, ...) map to the runtime's arraySlice* functions (ts/src/lower/array_slice.rs's native). Any other std function given a shared slice of arrays reads rs.arraySliceAsSlice: the arrays themselves, so a callback's ptr::eq sees the array it was lent, or copies of those as_chunks split from one flat slice, which have no JS array of their own. A &mut one reads rs.arraySliceBy's copies, written back once it returns, as Rust moves their elements. A function that would return a borrow of those copies is refused with a span.
  • as_chunks/as_rchunks (shared and mutable) and slice::from_mut/array::from_mut of a &mut [T; N] lend an rs.ArraySlice of the same storage. Cell::from_mut of an array of arrays is rs.cellOfArraySlice. mem::replace/mem::swap of an array, clone_from of arrays of arrays and of a Box<[[T; N]]>, raw pointers to arrays of arrays, and Debug of their iterators all go through it.
  • An array copied out and written back (rs.arraySliceBy, rs.arraySliceReplace, rs.arraySliceSwap, rs.cellOfArraySlice) copies its elements by the element type (copy::elem_copy): the type's copy shim where JS keeps a Copy value as an object, rs.objectCopy where a write lands inside the object, or none. So the copy shares no object with the arrays it was read from, and a &Cell that does share one is untouched.
  • Every other form a slice of arrays takes is rs.ArraySlice too: constants and Default, Option::as_slice/as_mut_slice of an array payload (rs.optionAsArraySlice), Cow<[[T; N]]> (rs.Cow<T, rs.ArraySlice<S>>, borrowed by rs.cowArraySlice), assume_init_ref/assume_init_mut of MaybeUninit arrays and Box::assume_init (rs.arraySliceOfSlice), generic structs at an array instance, and concat/join parts and separators. join lends an array separator as it is, since it clones it anyway, so each element's Clone reads the separator's own.
  • What std's collections and pointers hand out is an rs.ArraySlice of the same storage too: VecDeque's as_slices/as_mut_slices/make_contiguous (rs.dequeArraySlices, rs.dequeArrayMakeContiguous), BinaryHeap::as_slice, an array pattern's rest of an owned array of arrays, and a borrowed Cow's payload when matched. A raw pointer to a slice of arrays is an rs.RawArraySlice of it, so Box::into_raw/Box::from_raw give back the same arrays; making one from a pointer to arrays stays refused, as on main. The split iterators lend each array only when they read it.
  • A mutable reborrow &mut *p of a MaybeUninit of an array through a raw pointer is refused with a span: where p points into part of an array, or into arrays as_chunks_mut split from one slice, no JS array holds it alone, and a write through a copy was lost (on main too, for part of a plain array). A shared reborrow still reads it.
  • The Runs/ByteRuns proxies and View.detach are gone. arrayView remains only where a &mut [T; N] into part of an array is taken whole as a JS array; that remainder is now the whole quest.

Impact

  • Breaking for hand-written callers: a slice of arrays is an rs.ArraySlice rather than an rs.Slice<T[]> / rs.Slice<Uint8Array>. Pass one as rs.arraySliceFrom(arrays, rs.sliceFrom) (rs.byteSliceFrom for bytes, rs.arraySliceOf(rs.sliceFrom) for arrays of arrays) and read one as rs.arraySliceItems(slice). ts/README.md documents it.
  • Runtime exports: added ArraySlice, ArrayRef, LentArray, arraySlice* (arraySliceFrom, arraySliceOf, arraySliceAt, arraySliceItems, arraySliceAsSlice, arraySliceBy, arraySliceSet, arraySliceReplace, arraySliceSwap, arraySliceIter, arraySliceSplit*, arraySliceChunks*, arraySliceWindows, ...), cellOfArraySlice, RawArraySlice, rawArraySliceOf, dequeArraySlices, dequeArrayMakeContiguous, arraySliceBoxCloneFrom, arraySliceBoxCloneFromFn, arraySliceCopy, ElemCopy, optionAsArraySlice, arraySliceOfSlice, and cowArraySlice. rs.Cow takes a second type parameter, its borrowed form, rs.Slice<T> by default. arraySliceSwap, arraySliceBy, and arraySliceReplace take the element copy as an argument. Removed byteSliceAsChunks, byteSliceAsChunksMut, byteSliceAsRchunks, byteSliceAsRchunksMut, sliceAsChunksMut, and sliceAsRchunksMut. sliceAsChunks/sliceAsRchunks now return rs.ArraySlice.
  • No wire impact.

Decisions

Decided with the maintainer on 2026-11-06:

  1. How a &mut [[T; N]] holds its arrays: ✅ (a) a runtime slice-of-arrays type for every slice of arrays, nested or flat, which replaces the Runs/ByteRuns proxies (rejected: (b) eager subarray views for bytes only, keeping the Proxy for other elements; (c) keep the proxies and drop the item). The quest narrows to it and is sized [L].
  2. slice::from_mut/array::from_mut of a &mut u8: ✅ kept, deferred to its own quest at the end of m3, quest/m3/ts-byte-slice-place.md (rejected: drop it until a consumer appears).
  3. Comparing *const [T] of values JS copies: ✅ moved into quest/m3/ts-runtime-throws.md with the form_urlencoded 1.2.2 debug_assert!(ptr::eq(..)) finding (rejected: wait on rs.String's UTF-8 route).

Decided with the maintainer on 2026-11-07, at the round-3 design checkpoint (two medium findings, both conversions that copied arrays where Rust borrows them):

  1. ✅ Continue, with local fixes (rejected: merge as is; rethink to rs.ArraySlice only for what as_chunks splits, keeping rs.Slice<T[]> for other slices of arrays; drop). Where each array is a JS array of its own, rs.arraySliceItems slices the list of them instead of copying each, so arrays keep their identity for std callbacks such as binary_search_by and for ptr::eq. join reads its separator without copying it. The arrays as_chunks splits from one flat buffer have none of their own, so a callback still gets a copy; that ptr::eq gap is recorded in quest/m3/ts-runtime-throws.md.
  2. ✅ Rename rs.Runs to rs.ArraySlice, matching rs.ByteSlice and rs.RawSlice, with its helpers rs.arraySlice* (rejected: rs.Slices, rs.Chunks, keep rs.Runs).

The quest file keeps only the arrayView remainder, sized [M].

Choices made without asking:

  • array_slice::give converts a slice of arrays for every std function that isn't a trait's method, not just [T]'s own methods, so Vec::extend_from_slice(&[[T; N]]) and the like get the slice they read.
  • A copy of an array copies its elements by the element type in the translator, not by inspecting the storage at run time: a storage-directed copy in rs.sliceToArray (an earlier head) also copied arrays a &Cell shares, which Rust doesn't.
  • rs.sliceItems keeps one signature for rs.Slice, and the translator emits rs.arraySliceItems for an array slice. An overload made rs.sliceItems lose its type when passed as a function, as Option::copied passes it.
  • A mutable reborrow of a MaybeUninit array through a raw pointer is refused for every non-byte array, not only arrays of arrays, since part of a plain array loses the write the same way and no fixture used the path.
  • With the rename, the type an rs.ArraySlice lends each array as is rs.ArrayRef (was Run), the array it lends rs.LentArray (was RunArray; rs.ArrayOf was taken), the element copy rs.ElemCopy (was RunCopy), and the old runsArray/runsItems are arraySliceItems/arraySliceAsSlice, after sliceItems and as_slice. The translator's module is lower/array_slice.rs.

Bench

Rerun after the round-3 checkpoint's fixes and the rename: just ts bench at 06e0753, with --route main= origin/main f56859c's translation (.tmp/bench.sh under nix develop .#bench). Nanoseconds per packet, median ± half the IQR:

runtime packets/ts/number main → PR packets/ts/number-copy main → PR A/A at the median workload
Node 24.21 568 ±6.2% → 71.3 ±13% 948 ±5.5% → 348 ±35% 3.6%
Bun 1.4.2 621 ±24% → 251 ±32% 742 ±13% → 341 ±40% 10%
Chromium 156 555 ±12% → 136 ±8.2% 1410 ±21% → 708 ±22% 4.2%
Firefox 159 726 ±26% → 242 ±69% 1420 ±52% → 801 ±39% 9.5%

The speedup holds: 2-8x on packets/ts/number and 1.8-2.7x on number-copy. Only the two packets loops differ in the generated code (rs.arraySliceIter(rs.sliceAsChunks(..)) for rs.iterLend(rs.iterSlice(rs.sliceItems(rs.byteSliceAsChunksMut(..))))). Every other family's geometric mean of PR ÷ main is 0.73-1.21, within its runtime's A/A spread over a family's geometric mean (12-35%); their translated code is unchanged and only the runtime helpers differ.

Validation

  • just ci at 8b3cee4: passes (fixtures 89/89, including the lang fixtures' differential tests against native Rust).
  • slices::array_forms covers each form a slice of arrays takes; slices::array_identity pins a callback's ptr::eq on a lent array (of bytes, words, and arrays, of all of a slice and of part of it) and join's separator Clone reading its own address, each failing against native Rust with its fix reverted (74 and 79 differences); slices::array_sources covers the deque, heap, Rc, Vec::leak, raw-pointer round trip, MaybeUninit reborrow, and Cow::Borrowed cases; arrays::rests covers rest patterns written through. reject's pointers::uninit_rows_mut and pointers::uninit_part_mut pin the new refusal.

Alternatives

  • Bytes only, with eager subarray views (decision 1b): 3-4x the flat form, and it keeps the Proxy for other element types.
  • One rs.Slice per array made eagerly: allocates every array's slice up front; rs.ArraySlice makes one only when it is read.

Follow-ups

Models

Size: L
Implementer: Claude Code, claude-opus-5-5, unknown effort
Implementer: Claude Code, claude-opus-5-5, high effort

Review

mechanical fix by codex, gpt-6.1-sol, medium effort: Merged base and preserved both sides in four content conflicts; regenerated ts/fixtures/lang/manual/expected/uninit.ts; passed UPDATE_EXPECT=1 just ts test --test fixtures lang::manual -- --exact, just check base, and just test base. (head 64ed907)

mechanical fix by codex, gpt-6.1-sol, medium effort: Merged base and re·solved three generated TypeId snapshot conflicts using base normalization; ran just check base and just test base (passed). (head 29cd81c)

Recorded by hand: quest review ran from a local branch not named after the quest and printed record skipped.

  • Reviewer: Codex, gpt-6.1-sol, xhigh effort
  • Session: 01a1b4bb-259f-7702-9a0a-f128a9b641dc, round 3, started after the round-3 design checkpoint of the earlier session
  • Head: 8b3cee4
  • Base: origin/main at efa6300
  • Score: 10/10, The reported write-loss path now produces a compile error with a source span, both rejection probes pass, and no further defects were found.
  • Findings: none.
  • Earlier rounds of this session:
    • Round 2, at a4ec4fa: fixed: medium, ts/src/lower/mir/rvalue.rs:67: Preserve flat storage through mutable wrapped reborrows; refused with a span for every non-byte array.
    • Round 1, at 57e8ff3: fixed: medium, ts/src/lower/mod.rs:1503: Adapt owned-array rest borrows to ArraySlice; fixed: medium, ts/src/lower/mir/rvalue.rs:66: Convert wrapped multidimensional-array reborrows correctly; fixed: medium, ts/src/lower/mod.rs:1698: Update borrowed Cow payload extraction for ArraySlice; fixed: medium, ts/runtime/rs/iter.ts:480: Lend split-iterator rows lazily.
  • A first session (01a1b47a-a145-7b00-9697-4a5b4b214e02, Codex gpt-6.1-sol at high effort, since it read the size as M) found three at 06e0753: the deque's slices and the raw-pointer round trip, both fixed, and ptr::eq of as_chunks rows in a callback, refuted by decision 4.

🤖 Generated with Claude Code

@pullfrog

pullfrog Bot commented Nov 6, 2026

Copy link
Copy Markdown

Pullfrog  | Review this ➔

@kixelated kixelated changed the title rs.Slice rs.Slice: runs of as_chunks_mut, from_mut of a byte, *const [T] comparison Nov 6, 2026
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
kixelated and others added 4 commits November 6, 2026 16:38
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
… in, and pass as slices to std's other functions

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@kixelated kixelated changed the title rs.Slice: runs of as_chunks_mut, from_mut of a byte, *const [T] comparison feat(ts)!: rs.Runs for slices of arrays, replacing the as_chunks_mut proxies Nov 7, 2026
kixelated and others added 3 commits November 6, 2026 19:00
…eir elements' type

Constants, defaults, Option::as_slice, a Cow's borrow, and the erased
SliceOf and Ptr are runs too; to_vec reads runs as they are; a wrapped
array is an element, not runs; and runs copied out and written back copy
an element only as its type says: a Copy object's copy, or objectCopy of
one a write lands in, never a reference.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…nitialized MaybeUninits, and joined slices of arrays

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
kixelated and others added 5 commits November 6, 2026 20:30
…they are

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…tive Rust

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Matches rs.ByteSlice and rs.RawSlice, as the maintainer decided at the
round-3 design checkpoint.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@kixelated kixelated changed the title feat(ts)!: rs.Runs for slices of arrays, replacing the as_chunks_mut proxies feat(ts)!: rs.ArraySlice for slices of arrays, replacing the as_chunks_mut proxies Nov 7, 2026
@kixelated

Copy link
Copy Markdown
Collaborator Author

Design change: after the round-3 checkpoint, the maintainer chose to continue with local fixes and to rename the type.

  • Where each array of an array slice is a JS array of its own, rs.arraySliceItems now slices the list of arrays instead of copying each, so arrays keep their identity for std callbacks (binary_search_by) and ptr::eq. join lends its array separator without copying it, since it clones it anyway. The arrays as_chunks splits from one flat buffer have none of their own and still reach a callback as copies; that ptr::eq gap is recorded in quest/m3/ts-runtime-throws.md.
  • rs.Runs is now rs.ArraySlice, matching rs.ByteSlice and rs.RawSlice, and its helpers are rs.arraySlice* (runsArray → arraySliceItems, runsItems → arraySliceAsSlice).
  • A new differential case, slices::array_identity, pins both identity cases against native Rust.

Rejected: merge as is; rs.ArraySlice only for as_chunks output; drop; the names rs.Slices, rs.Chunks, and keeping rs.Runs. The next review starts a new session with quest review --new.

(Written by Claude Opus 5.5 with high effort)

kixelated and others added 4 commits November 6, 2026 22:13
…s of arrays

VecDeque's as_slices, as_mut_slices, and make_contiguous and BinaryHeap's
as_slice of arrays lend an rs.ArraySlice of their own arrays, and a raw
pointer to a slice of arrays is an rs.RawArraySlice of it, so Box::into_raw
and from_raw give back the same storage.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
# Conflicts:
#	ts/runtime/rs/collections.ts
…d a borrowed Cow, and split arrays lent lazily

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…aw pointer

A write through it landed in a copy where the pointer was into part of an
array, or into arrays as_chunks_mut split from one slice, which no JS array
holds alone. A shared reborrow still reads the array.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@kixelated
kixelated marked this pull request as ready for review November 7, 2026 13:13
@coderabbitai

coderabbitai Bot commented Nov 7, 2026 •

Copy link
Copy Markdown

Important

  • Trigger review

This repository does not receive automatic reviews because it has fewer than 10 stars.

Run configuration

  • Source mode: Auto
  • Repository: Unavailable
  • Branch: Unavailable
  • File: No configuration file from this repository applied
  • Commit: Unavailable
  • Layers: Organization UI
  • Review profile: CHILL
  • Plan: Advanced
  • Run ID: 1cba2c23-d340-45a7-84a1-339546f4b2bf
  • Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

Share

@kixelated
kixelated added this pull request to the merge queue Nov 7, 2026
@github-merge-queue
github-merge-queue Bot removed this pull request from the merge queue due to a conflict with the base branch Nov 7, 2026
@moq-bot

moq-bot Bot commented Nov 7, 2026

Copy link
Copy Markdown

quest land watch: the merge queue removed this pull request at 8b3cee4 because it conflicts with main (merge_conflict).

The land fixer works on it in the background, and the watcher re-enqueues it at the back of the queue once it is fixed or judged flaky, or hands it back here with what it needs.

(written by quest land watch)

@moq-bot
moq-bot Bot enabled auto-merge November 7, 2026 14:31
@moq-bot

moq-bot Bot commented Nov 7, 2026

Copy link
Copy Markdown

quest land watch: armed auto-merge on this pull request at 29cd81c, so it joins the back of the merge queue once its required checks pass, with the land fixer's fix (codex, gpt-6.1-sol, medium effort): Merged base and re·solved three generated TypeId snapshot conflicts using base normalization; ran just check base and just test base (passed)..

(written by quest land watch)

@moq-bot
moq-bot Bot added this pull request to the merge queue Nov 7, 2026
@github-merge-queue
github-merge-queue Bot removed this pull request from the merge queue due to a conflict with the base branch Nov 7, 2026
@moq-bot

moq-bot Bot commented Nov 7, 2026

Copy link
Copy Markdown

quest land watch: the merge queue removed this pull request at 29cd81c because it conflicts with main (merge_conflict).

The land fixer works on it in the background, and the watcher re-enqueues it at the back of the queue once it is fixed or judged flaky, or hands it back here with what it needs.

(written by quest land watch)

@moq-bot
moq-bot Bot enabled auto-merge November 7, 2026 15:33
@moq-bot

moq-bot Bot commented Nov 7, 2026

Copy link
Copy Markdown

quest land watch: armed auto-merge on this pull request at 64ed907, so it joins the back of the merge queue once its required checks pass, with the land fixer's fix (codex, gpt-6.1-sol, medium effort): Merged base and preserved both sides in four content conflicts; regenerated ts/fixtures/lang/manual/expected/uninit.ts; passed UPDATE_EXPECT=1 just ts test --test fixtures lang::manual -- --exact, just check base, and just test base..

(written by quest land watch)

@moq-bot
moq-bot Bot added this pull request to the merge queue Nov 7, 2026
Merged via the queue into main with commit 6e0ce8a Nov 7, 2026
7 checks passed
@moq-bot
moq-bot Bot deleted the quest/m3/ts-slice-object branch November 7, 2026 15:46
kixelated added a commit that referenced this pull request Nov 7, 2026
UnsafeCell::raw_get now translates only when every origin of its pointer
is a known place of its cell: &raw or a reference of a local or field,
through copies, casts, and derefs of such pointers, or MaybeUninit::as_ptr.
A parameter, a Vec's or slice's pointer, null, and pointer arithmetic are
refused with a span. raw_get returns to the eager form; the lazy place,
the mixed-origins handling, and rawUnsize's pointer-to-no-memory case go.

MaybeUninit::assume_init_mut is refused where its &mut is a place but one
to what it holds is the object, as of an UnsafeCell<u32>.

From #608, arrays of arrays keep main's representation: Cell::from_mut of
one is cellOfArraySlice and Cell's writes of one rebind it. Its
UnsafeCell::from_mut and Cell::swap are refused, as the cell may read a
copy.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant