Skip to content

perf(builtin): inline fast paths for slice access and pointer values - #531

Merged
paralin merged 1 commit into
s4wave:masterfrom
cogentcore:perf/slice-access-fast-paths
Oct 24, 2026
Merged

paralin merged 1 commit into
s4wave:masterfrom
cogentcore:perf/slice-access-fast-paths

Conversation

@rcoreilly

Copy link
Copy Markdown
Contributor

arrayIndex, sliceRange, sliceLen, pointerValue and copyElement are called for nearly every slice element access and pointer dereference, and each tried a long chain of representations. As large functions they were not inlined, so every element access in a hot loop was a call, and the property loads inside each helper saw every kind of slice in the program.

This splits each into a small fast path for the common representations (a window object, a typed slice or a plain array; a variable reference or a struct; a value that is not an object) and a slow function holding the existing code for everything else, unchanged. V8 now inlines them into hot loops (checked with --trace-turbo-inlining), where the window that sliceRange makes for a plain array also no longer needs to be allocated.

Measured on a tensor element-access benchmark (Cogent Lab tensor.Value with three indexes, as neural network simulations use for every neuron variable) under Node 26:

  • tensor.Value: 33 ns to 23 ns per access
  • tensor.Set: 29 ns to 22.5 ns
  • indexing a []int struct field in a loop: 39 ns to 17 ns
  • indexing a []float32: 15 ns to 10 ns

In the browser, one trial of an axon simulation (CPU profile in Chrome) went from 3.98 s to 3.26 s of compute.

The full Go test suite (including compliance) passes, as do the TypeScript type check and the vitest suite.

arrayIndex, sliceRange, sliceLen, pointerValue and copyElement are called
for nearly every slice element access and pointer dereference, and each
tried a long chain of representations. As large functions, engines did
not inline them, so every element access in a hot loop was a call whose
property loads saw every kind of slice in the program.

Split each into a small fast path for the common representations (a
window object, a typed slice or a plain array; a variable reference or a
struct; a value that is not an object) and a slow function with the
existing code for the others. V8 now inlines them into hot loops, where
the window that sliceRange makes for an array also need not be
allocated.

In a benchmark of tensor element access (Cogent Lab tensor.Value with
three indexes) under Node 26, an access takes 23 ns instead of 33 ns,
plain indexing of a []int field in a loop 17 ns instead of 39 ns, and of
a []float32 10 ns instead of 15 ns.

Signed-off-by: Randall C. O'Reilly <oreilly@ucdavis.edu>
@paralin
paralin merged commit 44ff1a5 into s4wave:master Oct 24, 2026
7 checks passed
@rcoreilly

Copy link
Copy Markdown
Contributor Author

Pushed a second commit that gives sliceElems and sliceSlot, which element stores (including compound assignments) go through, the same fast-path split. A read-modify-write of a tensor element went from 34 ns to 29 ns in the same benchmark. Full Go suite, typecheck and vitest still pass.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants