Repository navigation
fix(frontend): stop unit-test memory growth that times out the macOS leg - #9198
aglinxinyuan wants to merge 3 commits into
Conversation
The unit-test builder runs Vitest with isolate: false, so each worker runs its spec files against one jsdom document and one copy of Vitest. Several worker-wide holders kept every finished test's TestBed graph reachable, and on the 7 GB macOS runner the resulting growth (about 2.4 GB per worker) swapped the VM into stalls that timed out whichever test was running. - Release what root services start: WorkflowActionService destroys its shared model, WorkflowWebsocketService stops its heartbeat and closes the socket, JupyterPanelService removes its window listener, and SharedModel.destroy() destroys its y-websocket provider rather than only disconnecting it when connected. - Empty jsdom's selector cache between tests (new setup file). - Backport Vitest 5's weak mock registry into @vitest/spy 4.1.11 with a Yarn patch, guarded by worker-memory.spec.ts. Closes apache#9197 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Backport auto-label reportThis
|
Automated Reviewer SuggestionsBased on the
|
…st-timeouts # Conflicts: # frontend/src/app/workspace/service/workflow-websocket/workflow-websocket.service.ts
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## main #9198 +/- ##
=========================================
Coverage 91.98% 91.98%
+ Complexity 5084 5083 -1
=========================================
Files 1281 1281
Lines 55806 55810 +4
Branches 7076 7075 -1
=========================================
+ Hits 51333 51337 +4
Misses 2732 2732
Partials 1741 1741
*This pull request uses carry forward flags. Click here to find out more. ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
|
| config | throughput | MB/s | latency | max Δ latest / 7d | |
|---|---|---|---|---|---|
| 🔴 | bs=10 sw=10 sl=64 | 449 | 0.274 | 21,044/34,134/34,134 us | 🔴 +20.2% / 🔴 +147.8% |
| 🔴 | bs=100 sw=10 sl=64 | 935 | 0.571 | 104,841/132,479/132,479 us | 🔴 +21.1% / 🔴 +42.2% |
| ⚪ | bs=1000 sw=10 sl=64 | 1,154 | 0.704 | 863,091/905,924/905,924 us | ⚪ within ±5% / 🔴 -11.6% |
Baseline details
Latest main c57e6b0 from same runner
| config | metric | PR | latest main | 7d avg | Δ latest | Δ 7d |
|---|---|---|---|---|---|---|
| bs=10 sw=10 sl=64 | throughput | 449 tuples/sec | 509 tuples/sec | 941.58 tuples/sec | -11.8% | -52.3% |
| bs=10 sw=10 sl=64 | MB/s | 0.274 MB/s | 0.31 MB/s | 0.575 MB/s | -11.6% | -52.3% |
| bs=10 sw=10 sl=64 | p50 | 21,044 us | 17,511 us | 10,991 us | +20.2% | +91.5% |
| bs=10 sw=10 sl=64 | p95 | 34,134 us | 31,733 us | 13,776 us | +7.6% | +147.8% |
| bs=10 sw=10 sl=64 | p99 | 34,134 us | 31,733 us | 16,872 us | +7.6% | +102.3% |
| bs=100 sw=10 sl=64 | throughput | 935 tuples/sec | 1,015 tuples/sec | 1,264 tuples/sec | -7.9% | -26.0% |
| bs=100 sw=10 sl=64 | MB/s | 0.571 MB/s | 0.619 MB/s | 0.772 MB/s | -7.8% | -26.0% |
| bs=100 sw=10 sl=64 | p50 | 104,841 us | 97,771 us | 86,073 us | +7.2% | +21.8% |
| bs=100 sw=10 sl=64 | p95 | 132,479 us | 109,378 us | 93,135 us | +21.1% | +42.2% |
| bs=100 sw=10 sl=64 | p99 | 132,479 us | 109,378 us | 111,124 us | +21.1% | +19.2% |
| bs=1000 sw=10 sl=64 | throughput | 1,154 tuples/sec | 1,169 tuples/sec | 1,304 tuples/sec | -1.3% | -11.5% |
| bs=1000 sw=10 sl=64 | MB/s | 0.704 MB/s | 0.713 MB/s | 0.796 MB/s | -1.3% | -11.6% |
| bs=1000 sw=10 sl=64 | p50 | 863,091 us | 854,601 us | 841,986 us | +1.0% | +2.5% |
| bs=1000 sw=10 sl=64 | p95 | 905,924 us | 887,282 us | 881,107 us | +2.1% | +2.8% |
| bs=1000 sw=10 sl=64 | p99 | 905,924 us | 887,282 us | 918,208 us | +2.1% | -1.3% |
Raw CSV
config_idx,batch_size,schema_width,string_len,num_batches,total_ms,total_tuples,total_bytes,tuples_per_sec,mb_per_sec,lat_p50_us,lat_p95_us,lat_p99_us
0,10,10,64,20,445.86,200,128000,449,0.274,21044.39,34133.80,34133.80
1,100,10,64,20,2139.44,2000,1280000,935,0.571,104840.86,132478.58,132478.58
2,1000,10,64,20,17328.36,20000,12800000,1154,0.704,863090.91,905923.67,905923.67Drop the GC guard spec, the Renovate grouping rule and the TESTING.md section, fold the selector-cache helper into its setup file, and make the @vitest/spy patch comment self-contained. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
What changes were proposed in this PR?
This fixes the random unit-test timeouts on the macOS frontend leg (#9197) at their cause: memory that grows across spec files inside each Vitest worker.
The Angular unit-test builder runs Vitest with
isolate: false, so a worker runs all its spec files against one jsdom document and one copy of Vitest. Three kinds of worker-wide reference kept every finished test's TestBed graph alive, and any one of them was enough on its own. On the 7 GB macOS runner that came to about 2.4 GB per worker, and the VM swapped until whichever test was running timed out.WorkflowActionService.ngOnDestroy()destroys the shared model.SharedModel.destroy()callswsProvider.destroy()instead of a conditionaldisconnect(). This releases the provider's interval, BroadcastChannel subscription and listeners, and stops a provider that never connected from retrying forever.WorkflowWebsocketService.ngOnDestroy()stops the heartbeat and closes the socket.JupyterPanelService.ngOnDestroy()removes itswindowlistener.#root<N>query TestBed makes for each fixture. The new setup file issrc/jsdom-selector-cache-setup.ts.@vitest/spyheld everyvi.fn()/vi.spyOn()in aSetthat is never pruned. A Yarn patch backports Vitest 5's WeakRef registry, since Angular 21 still requires Vitest 4. The patch carries Vitest's MIT code, so it is listed inLICENSE, with the license text inlicenses-3rd-party-code/vitest.txt.Measured locally in the macOS shape (2 workers, coverage, full suite):
DurationNotes for review:
isolate: true? It also bounds memory, but it roughly doubled the Vitest run locally (230 s against about 100 s).resolutionsentry names the exact@vitest/spyversion, so it stops applying whenever Vitest is bumped.Any related issues, documentation, discussions?
Closes #9197. Related: #7713 raised the test timeouts and #7975 capped the worker heap. Both treated the symptom.
How was this PR tested?
mainstarted requiring Node 24, on this branch's earlier commits:yarn --cwd frontend run test:ci: 244 files passed, 6857 tests passed, 1 skipped.SharedModeldestroy: 2 testsWorkflowActionService,WorkflowWebsocketService,JupyterPanelServiceteardown: 1 test eachmain:yarn install --immutablepasses.prettier-eslintandeslintare clean on the changed files.mainnow needs Node 24 and my machine has Node 20.Was this PR authored or co-authored using generative AI tooling?
Generated-by: Claude Code (Claude Opus 5.5)