FE-1314, FE-1237: Run experiments in parallel across workers, add place token capacity by kube · Pull Request #9162 · hashintel/hash

kube · 2026-08-04T22:59:41Z

🌟 What is the purpose of this PR?

Two related changes to the simulator, plus the investigation that motivated them.

Experiments now run in parallel. An experiment used to run all of its runs in one Web Worker, so a 1000-run experiment used one core no matter how many the machine had. Runs are independent, so they are now split across one worker per logical core (minus one, to keep the editor responsive). Measured ~4.1× at 8 shards on a 10-core machine.

The property that matters is that this cannot change what an experiment reports, and that is checked exactly rather than assumed — every frame of the merged timeline is fingerprinted and compared across shard counts, for both a distribution and a scalar-mean metric.

Places gain an optional token capacity (FE-1237). A transition whose firing would take one of its output places above capacity is not enabled — the supply-side mirror of an input arc that cannot be satisfied — so a full place blocks the transitions feeding it and the limit is never exceeded. This is the standard capacity-constraint semantics the issue describes.

Also included: libs/@hashintel/petrinaut-core/docs/simulation-performance.md, which records where simulation time actually goes (profiled, not guessed) and evaluates WASM, native, and WebGPU against those numbers. It is the reason this PR is threading and capacity rather than a WASM rewrite — see Next steps.

🔗 Related links

FE-1314: Run experiment runs in parallel across workers (internal)
FE-1237: Place capacity — limit the number of tokens allowed on a place (internal)
FE-948: Show & kill combinatorial explosion (internal) — the quadratic blow-up documented in §2.4 of the new doc. Sharding divides it by core count; it does not fix it.
gpuweb#2805: Double precision floats — WGSL has no f64, which is why the GPU analysis lands where it does.

🔍 What does this change?

Parallel experiments

createMonteCarloExperiment fans out over N transports instead of one, merging their metric frames, aggregating progress, and handling complete/cancel/error across shards.
runtime/shard-plan.ts sizes the pool and splits runs into contiguous slices, spreading the remainder across leading shards so no single shard becomes the straggler everyone waits on.
MonteCarloSimulatorConfig.runIndexOffset — seeds derive from the run's global index, so run i gets the same seed whichever worker owns it. This is what makes shard layout invisible in the output.
metrics/merge.ts — streaming monoid merge of per-frame state. Two things worth a reviewer's attention:
- Scalar frames now carry runAggregate, the pre-reduction accumulator state, because frameValue is already reduced and cannot be merged — a mean of means is not a mean. Time aggregation is recomputed on the main thread from merged frame values. Distribution metrics aggregate per run over time and are therefore already correct shard-locally.
- Frames release on a watermark: a frame number finalises only once every still-running shard has reported it, with finished shards dropped from the watermark rather than blocking it. That is also why merged output matches an unsharded run — a finished shard has no active runs left to contribute, exactly like the completed runs a single simulator skips.
experimentShardCount on ExperimentsProvider (and shardCount on createMonteCarloExperiment) lets a host cap or pin parallelism. A caller-supplied transport stays one shard, being a single channel.

Place capacity

Place.capacity?: number | null, with a zod schema entry and a place-inspector control.
engine/capacity.ts precomputes per-transition constraints at build time. Three decisions in there rather than consequences:
- Net change, not gross output. Constraints are output arc weights minus standard input arc weights on the same place, so a 1-in/1-out self loop is never blocked by its own full place. Read and inhibitor arcs consume nothing and so do not make room.
- Same-frame pending output counts. Output is applied once at end-of-frame, so the frame's counts lag during evaluation. Without folding in what earlier transitions already committed, two producers into one capped place would each individually fit and jointly overflow.
- Deadlock includes capacity. Both engines' structural-enablement checks consider it, so a net whose remaining transitions are all blocked by full places reports deadlock rather than stepping to maxTime doing nothing.
Constraint lists are empty for nets without capacities, so nets that do not use the feature pay nothing in the hot path.
An initial marking already above a capacity is rejected at build time: capacity blocks transitions, so it cannot repair a starting state that already violates the bound.

Investigation record

docs/simulation-performance.md — profiled cost breakdown, the ~36× gap between the current engine and equivalent flat JavaScript, and the case for whole-loop codegen before WASM.
benchmarks/ — the harnesses behind every number in that doc. sharded-experiment.mjs drives the real runtime over real worker threads and exits non-zero if any shard count changes results.

Pre-Merge Checklist 🚀

🚢 Has this modified a publishable library?

This PR:

modifies an npm-publishable library and I have added a changeset file(s)

📜 Does this require a change to the docs?

The changes in this PR:

require changes to docs which are made as part of this PR

experiments.md (it claimed one worker per experiment), plus a new token-capacity section in drawing-a-net.md and capacity's effect on enablement and deadlock in simulation.md.

🕸️ Does this require a change to the Turbo Graph?

The changes in this PR:

do not affect the execution graph

⚠️ Known issues

Concurrent experiments do not share a worker pool. Three 8-way experiments spawn 24 workers and compete for cores. Memory is roughly unchanged, since sharding splits runs rather than duplicating them, so this degrades rather than breaks — and the user docs say to run experiments one at a time for maximum speed. A shared pool would still be better.
Scaling is sub-linear past ~4 shards (1.89× / 2.98× / 4.11× at 2 / 4 / 8). Partly efficiency cores, partly that each shard repays the per-run buildSimulation cost documented in §2.3 — ~80 µs/run, so ~1000 new Function calls for a 1000-run experiment. Worth fixing, but it is a separate change.
Metric state crosses threads as [number, number][], not typed arrays, so it is cloned rather than transferred.
Capacity does not participate in the HIR artifact fingerprint. It changes enablement but not any compiled program, so existing artifacts stay valid. If capacity ever feeds into codegen, that has to change.
The fixed-size frame layout capacity unlocks is not built — the runtime still uses growable frames.

🐾 Next steps

Ordered by measured value, from §4 of the new doc:

FE-948 / the quadratic enumeration blow-up. enumerateWeightedMarkingIndices eagerly materialises every k-combination per arc before yielding one, while the caller returns on the first accepted combination. A coloured place with a weight-2 input arc costs 13.4 ms per run-frame at 400 tokens — roughly 6.7 hours for a 1000-run × 1800-frame experiment. Sharding divides this; it does not fix it. This is the highest-value change left and is independent of everything else.
Share one compiled SimulationDefinition across a shard's runs instead of calling buildSimulation per run.
Skip the unconditional frame copy when no place has dynamics (a measured 20%).
Mutable-in-place metric accumulators — the histogram add currently clones a Map per sample, and its per-run-frame cost grows with run count (425 ns at 125 runs → 1038 ns at 4000).
Then whole-loop codegen, and only then WASM. The doc argues WASM buys ~1.5–3× over good numeric JS, and there is ~36× of architectural headroom in front of it.

One open question for GPU, recorded in §10: whether f32 results diverging from the CPU path's f64 is acceptable for the same seed and engine version. That is the remaining gate on any WebGPU work.

🛡 What tests cover this?

New, all passing (1084 total across the two packages):

metrics/merge.test.ts (12) — the merge monoid. Includes the case that would break under naive merging: shards of 3 runs averaging 10 and 1 run at 100 must pool to 32.5, not 55. Also covers last resolving by shard order when shards reply out of order, watermark withholding, early-finishing shards, and time-aggregation recomputation.
runtime/shard-plan.test.ts (14) — slice coverage and contiguity, remainder spreading, never more shards than runs, core-count detection and fallbacks.
runtime/experiment.test.ts (+9) — shard fan-out with correct runIndexOffset, ready/complete gating on all shards, cancel propagation, teardown on error, and progress reporting the slowest shard.
engine/capacity.test.ts (13) — constraint derivation: net change, self loops, read/inhibitor arcs not making room, capacity 0, invalid values treated as unbounded.
monte-carlo/capacity.test.ts (7) — capacity through the real stepping loop: the limit holding, multi-token firings not overshooting, two producers in one frame, self loops not blocking, deadlock, and nets without capacities unchanged.

Existing suites also caught two genuine errors during development, both fixed: allFinished was being derived from "shard stopped reporting", which made a cancelled experiment claim its runs had finished (cancelling is not finishing); and the provider test surfaced merge behaviour that had to be pinned to one shard to keep testing worker plumbing rather than sharding.

❓ How to test this?

Automated, from libs/@hashintel/petrinaut-core:

yarn build && node benchmarks/sharded-experiment.mjs

Every row must print identical to 1 shard: YES — that is the guarantee, and the script exits non-zero otherwise. Expect a speedup curve topping out near core count.

Manually:

Open Simulate → Experiments, create an experiment with ~1000 runs, and run it. It should finish several times faster than on main, and the editor should stay responsive.
Note the resulting distributions, then re-run with the same seed and configuration — the numbers must match. They should also match main's output for the same seed.
Select a place, tick Token capacity, set a small value, and run a simulation. The place should never exceed it, and transitions feeding it should stop firing once it is full.
Point two transitions at one capped place and confirm they cannot jointly overflow it.
Give a place a capacity below its initial marking and confirm the run reports an error rather than starting.

📹 Demo

Not captured — the visible change is wall-clock time and one new inspector field. The benchmark output above is the substantive evidence.

Experiments split their runs across one worker per logical core (minus one, capped at the run count) instead of running everything in a single worker. Measured ~4.1x at 8 shards on a 10-core machine. Sharding cannot change results. Per-run seeds derive from the run's global index rather than its position within a shard, so run i gets the same seed whichever worker owns it, and each worker's per-frame metric state is recombined through the accumulator monoids. Scalar frames now carry the pre-reduction `runAggregate`, because `frameValue` is already reduced and a mean of means is not a mean; time aggregation is recomputed from merged frame values. Frames release on a watermark: a frame finalises once every still-running shard has reported it, with finished shards dropped rather than blocking it — which is also why merged output matches an unsharded run, since a finished shard has no active runs left to contribute. Hosts can cap or pin parallelism with `experimentShardCount` on `ExperimentsProvider`, or `shardCount` on `createMonteCarloExperiment`. A caller-supplied `transport` stays a single shard, being one channel. Places also gain an optional token capacity. A transition whose firing would take one of its output places above capacity is not enabled — the supply-side mirror of an input arc that cannot be satisfied — so a full place blocks the transitions feeding it and the limit is never exceeded. The check uses the net change per firing, so a transition that both consumes from and produces into a place is not blocked by its own output, and it folds in output committed earlier in the same frame so several producers cannot jointly overflow one place. Both engines' structural-enablement checks include capacity, so a net blocked only by full places reports deadlock instead of stepping to maxTime. An initial marking already above a capacity is rejected at build time. Constraints are precomputed per transition and empty for nets without capacities, so nets that do not use the feature pay nothing. Also adds docs/simulation-performance.md recording where simulation time actually goes, plus the benchmark harnesses behind those numbers — one of which fails if any shard count changes an experiment's results.

vercel · 2026-08-04T22:59:47Z

The latest updates on your projects. Learn more about Vercel for GitHub.

Project	Deployment	Actions	Updated (UTC)
hash	Ready	Preview	Aug 5, 2026 4:22pm
petrinaut	Ready	Preview	Aug 5, 2026 4:22pm

1 Skipped Deployment

Project	Deployment	Actions	Updated (UTC)
hashdotdesign-tokens	Ignored	Preview	Aug 5, 2026 4:22pm

The §10 rewrite reported earlier never applied: its search text predated an oxfmt pass that rewrote *existing* to _existing_, so the replacement silently missed. The section still asked three questions that had been answered. Restating them as decisions, and restarting the 'Still open' list at 1 so MD029 passes.

Experiments can run on the GPU where the net allows it, selected in Settings. On the SIR example at 4096 runs x 600 frames with a distribution metric: 3.4 ms against the CPU engine's 6060 ms, agreeing to within 0.5% at every sampled frame. The unit of work is an experiment, not a frame. One invocation owns one run and advances it 300 frames per dispatch with the stepping loop inside the shader, and per-frame cross-run distributions are reduced on the device into histograms in workgroup-shared memory (measured 2x faster than global atomics). This shape is the point: a GPU path implementing the CPU's synchronous per-frame `advanceAll()` would need a readback per frame and would be slower than the CPU, so this implements the experiment handle and not the simulator interface. User code reaches the GPU through a second HIR backend, emit-wgsl.ts, alongside the existing JavaScript one. Dynamics default to RK4: a token's derivative depends only on that token, so all four stages fit in one invocation with no extra dispatch — measured at 2.5x Euler's cost for about four orders of magnitude less truncation error. Contrary to what docs/simulation-performance.md predicted, f32 is not the limiting factor for ODEs; integrator truncation error dwarfs rounding error, so f32 Euler tracks f64 Euler to three significant figures and f32 RK4 beats it badly. The backend is a subset engine and refuses rather than approximates. Typed places need a declared capacity, string and uuid attributes cannot be held in 32-bit WGSL, weighted arcs on typed places are rejected because choosing among token combinations does not vectorise, and only place-token-count metrics are served. Everything else falls back to the CPU with a reason. Histogram saturation is reported rather than silently clipping a distribution's tail. Building this caught two things worth recording. Comparing against the CPU engine exposed a first version that redrew the acceptance variate every frame and diverged 21% by frame 599: the CPU commits its generator state only when a transition fires, which holds u fixed and makes firing an exponential waiting time rather than a per-frame Bernoulli trial. And the CPU generator itself is defective — its multiply exceeds 2^53 for 99.6% of its seed space, giving a 10,466-state cycle that every seed enters after a median 3,864 draws. That is latent at today's experiment sizes and becomes real past ~9,000 draws per run; it is documented in §8a as separate work, not fixed here. Results agree statistically but not seed for seed, since WGSL cannot reproduce the CPU stream and the shader uses PCG instead.

+    const value = this.emit(expr, env);
+    const literal = /^-?\d+(?:\.0)?$/u.exec(
+      value.kind === "f32" ? value.code : "",
+    );


The frontend build failed with `Module not found: Can't resolve 'module'`. Cause: the settings dialog imported `isWebGpuAvailable` from `@hashintel/petrinaut-core/webgpu`, and that entry re-lowers user code to HIR, so it bundles the TypeScript compiler and its Node builtins. A one-line capability check dragged the whole compiler into the browser bundle. Detection now lives in `webgpu/support.ts` with no imports at all, is re-exported from the main entry, and the dialog imports it from there. `runner.ts` delegates to it rather than keeping a second copy. Verified on the import graph: the UI bundle no longer references `lower-net-hir`, and the main core entry contains no backend symbols and no Node builtins. Also fixes ten lint errors that only `yarn lint:eslint` reports — I had run bare `oxlint`, which omits `--type-aware` and the unused-disable-directive check. Redundant `!== undefined` guards against a `string | null` return, unnecessary optional chains on the non-nullable `GPUAdapter.info`, a dead `no-await-in-loop` directive, and two `expect.stringContaining` assertions whose `any` return the type-aware pass rejects.

The backend existed but nothing called it — the setting only persisted a preference. Experiments now attempt the GPU when it is selected and fall back to the CPU when the net does not qualify, with the reason shown to the user and recorded on the experiment. `createGpuMonteCarloExperiment` presents a GPU run through the same `MonteCarloExperiment` handle the CPU path uses, so the provider only chooses which to build and needs no branch after that. Supportability is resolved before the handle exists: a handle that could fail on `start()` would leave the caller unable to fall back once the experiment is already registered and showing as running. `ExperimentRecord` gains `computeBackend` and `computeBackendFallbackReason`. Recording which one ran matters, because the two are not numerically interchangeable — WebGPU cannot reproduce the CPU generator's stream, so a seed gives different but statistically equivalent trajectories. The choice travels on `CreateExperimentInput` rather than being read in the provider: `UserSettingsProvider` is mounted inside `ExperimentsProvider` (petrinaut-provider.tsx) and so is invisible to it. The create-experiment drawer reads the setting and passes it. Cancellation is observed at chunk boundaries, since a submitted dispatch cannot be interrupted. A cancelled run keeps the frames it already computed and reports `allFinished: false`, matching how the CPU path treats abandoned runs. Store and event plumbing moved to `runtime/experiment-stores.ts`, shared by both backends so a missed `Object.is` guard cannot land in one and not the other. Verified in a browser against the built output: handle drives Running -> Complete with per-chunk progress, 600 frames per metric, mean infected 51.9 at frame 599 against the CPU engine's 52.0; cancelling mid-run stops at the chunk boundary keeping 300 of 6000 frames; an expression metric is declined with a reason rather than re-measured. One test I first wrote passed vacuously — it asserted the record's default "cpu" rather than the patched value, because the handle only resolves once a shard reports ready. Both provider tests now drive `ready` and assert the patch.

The frontend build failed with `Module not found: Can't resolve 'module'`. I fixed this same regression one commit earlier for `isWebGpuAvailable`, wrote up the hazard in that commit message, and then reintroduced it by importing `createGpuMonteCarloExperiment` from `@hashintel/petrinaut-core/webgpu` in the experiments provider — which reaches `lowerNetHir`, the TypeScript frontend, and its Node builtins. Guarding the import site was the wrong fix twice over. The real problem is that the GPU backend re-lowered the net's user code to get HIR, and lowering cannot happen in a browser bundle. But nothing had to lower it again: `compileHirArtifacts` already builds the HIR and then discards the tree after emitting JavaScript, and HIR is a JSON-serializable structure by design. So artifacts now carry it — `HirLambdaArtifact.hir` and `HirDynamicsArtifact.hir` — and `hir-from-artifacts.ts` reads it back. The artifacts are produced in the language worker, which has the compiler, and cross to the browser with the rest of the artifact. `lowerNetHir` is no longer reachable from the `./webgpu` entry, which now declares no external imports at all, and no petrinaut dist chunk imports a Node builtin. Note that dynamics artifacts are keyed by differential equation while the shader generator needs them per place, so the mapping is done here rather than assumed. Behaviour is unchanged: verified in a browser through the artifacts path, mean infected 51.9 at frame 599 against the CPU engine's 52.0, identical to the previous measurement.

The GPU backend needs HIR in the browser, but lowering it there pulls in the TypeScript compiler and breaks the frontend bundle. Artifacts now carry the HIR from the language worker instead — behind `includeHir`, since it roughly triples artifact size and artifacts are cloned to every shard worker. The experiments provider asks for it only when the chosen backend is `webgpu`. `check-browser-safe-entries.mjs` walks the built entry graphs for Node-only imports as part of `yarn build`, so that class of bug fails in a second rather than nine minutes into a frontend build. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

The drawer's Summary section gains a CPU/GPU badge beside its heading and an Elapsed field that becomes Duration once the run stops. Both facts were already recorded on every experiment but never shown, so a result could not be attributed to a backend afterwards — and the two do not produce identical trajectories for a given seed. Timing measures stepping only. Setup differs between the backends, so charging it to the timer would make them look different for reasons unrelated to how fast they simulate; `finishedAt` is stamped centrally in `patchExperiment` so completion, failure and cancellation all record a stop time. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

Reports what the compiler made of a net: which user code lowered to HIR, and what stops the net running on the GPU. With a node selected it narrows to that node's code. `analyzeCompilation` is the new source of truth, because the pipeline has three gates that fail in different places and only the first produces a usable message — the other two collapsed into one fallback sentence with no way to find the transition responsible. Two behaviours the eligibility gate does not refuse are now visible: the GPU never runs transition kernels, so typed tokens it produces arrive with zeroed attributes, and conditions cannot read token attributes at all whatever the arc weight. The second was missing from experiments.md. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

Settings gains a WebGPU master switch in place of the backend picker, and the Create Experiment drawer gains a Run on GPU switch. The backend was never a global preference — it is a property of the run, and the old shape made a whole session CPU or GPU. Both can now be in flight at once, which is what makes comparing them on one model practical. The switch greys out when the model cannot run on the GPU, with the reason on hover, chosen by `summarizeGpuUnavailability` in order of how actionable each kind is. Whether the GPU is used is derived from `requested && available` in one place, so a net edited into ineligibility neither shows as on nor submits a GPU experiment. Also fixes a leak found while checking that concurrency: the provider did not dispose handles on `error`, and disposal is what calls `device.destroy()`, so every failed GPU experiment held a live GPUDevice. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

kube self-assigned this Aug 4, 2026

github-actions Bot added area/infra Relates to version control, CI, CD or IaC (area) area/libs Relates to first-party libraries/crates/packages (area) type/eng > frontend Owned by the @frontend team labels Aug 4, 2026

vercel Bot temporarily deployed to Preview – petrinaut August 4, 2026 23:02 Inactive

vercel Bot deployed to Preview – hash August 4, 2026 23:09 View deployment

vercel Bot deployed to Preview – petrinaut August 4, 2026 23:25 View deployment

vercel Bot deployed to Preview – hash August 4, 2026 23:30 View deployment

github-actions Bot added the area/deps Relates to third-party dependencies (area) label Aug 5, 2026

github-advanced-security AI found potential problems Aug 5, 2026

View reviewed changes

Comment thread libs/@hashintel/petrinaut-core/src/webgpu/emit-wgsl.ts

Comment on lines +550 to +553

const value = this.emit(expr, env);

const literal = /^-?\d+(?:\.0)?$/u.exec(

value.kind === "f32" ? value.code : "",

);

vercel Bot deployed to Preview – petrinaut August 5, 2026 00:36 View deployment

vercel Bot had a problem deploying to Preview – hash August 5, 2026 00:43 Failure

vercel Bot deployed to Preview – petrinaut August 5, 2026 01:12 View deployment

vercel Bot deployed to Preview – hash August 5, 2026 01:17 View deployment

vercel Bot deployed to Preview – petrinaut August 5, 2026 10:00 View deployment

vercel Bot had a problem deploying to Preview – hash August 5, 2026 10:02 Failure

vercel Bot deployed to Preview – petrinaut August 5, 2026 10:37 View deployment

vercel Bot deployed to Preview – hash August 5, 2026 10:43 View deployment

vercel Bot deployed to Preview – petrinaut August 5, 2026 11:15 View deployment

vercel Bot deployed to Preview – hash August 5, 2026 11:20 View deployment

github-actions Bot added the area/apps > hash.design Affects the `hash.design` design site (app) label Aug 5, 2026

vercel Bot deployed to Preview – petrinaut August 5, 2026 12:41 View deployment

vercel Bot deployed to Preview – hash August 5, 2026 12:47 View deployment

vercel Bot deployed to Preview – petrinaut August 5, 2026 14:06 View deployment

vercel Bot deployed to Preview – hash August 5, 2026 14:12 View deployment

vercel Bot deployed to Preview – petrinaut August 5, 2026 16:16 View deployment

vercel Bot deployed to Preview – hash August 5, 2026 16:22 View deployment

Navigation Menu

Uh oh!

FE-1314, FE-1237: Run experiments in parallel across workers, add place token capacity - #9162

FE-1314, FE-1237: Run experiments in parallel across workers, add place token capacity#9162
kube wants to merge 10 commits into
mainfrom
cf/fe-1314-run-experiment-runs-in-parallel-across-workers

kube commented Aug 4, 2026

Uh oh!

vercel Bot commented Aug 4, 2026 •

edited

Loading

Uh oh!

Reviewers

Assignees

Labels

Milestone

Development

Uh oh!

2 participants

Uh oh!

Conversation

kube commented Aug 4, 2026

🌟 What is the purpose of this PR?

🔗 Related links

🔍 What does this change?

Parallel experiments

Place capacity

Investigation record

Pre-Merge Checklist 🚀

🚢 Has this modified a publishable library?

📜 Does this require a change to the docs?

🕸️ Does this require a change to the Turbo Graph?

⚠️ Known issues

🐾 Next steps

🛡 What tests cover this?

❓ How to test this?

📹 Demo

Uh oh!

vercel Bot commented Aug 4, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Milestone

Development

Uh oh!

2 participants

vercel Bot commented Aug 4, 2026 •

edited

Loading