Skip to content

Run Firefox tests in parallel - #4233

Merged
nstepien merged 7 commits into
mainfrom
firefox-parallel
Oct 25, 2026
Merged

nstepien merged 7 commits into
mainfrom
firefox-parallel

Conversation

@nstepien

@nstepien nstepien commented Oct 25, 2026 •

Copy link
Copy Markdown
Collaborator

Firefox ran test files one at a time since #3772, which noted that "Tests in FF timeout when more than one worker is used". With Vitest 5, a project with fileParallelism: false also only starts once every other project is done, so Firefox was the long pole of the suite.

Why parallel Firefox tests failed

Vitest runs all of a project's pages in a single Firefox, each in its own context. Playwright's Firefox protocol synthesizes every page's mouse events in Firefox's parent process with the same pointer ID, DEFAULT_MOUSE_POINTER_ID (PageHandler.js in Juggler). Firefox's parent process tracks pointer capture per pointer ID across all tabs, and sends every mouse event with a captured pointer ID to the capturing tab (PointerEventHandler.cpp, EventStateManager::DispatchCrossProcessEvent).

The grid captures the pointer while dragging a column's resize handle (HeaderCell.tsx) or the drag-fill handle (DataGrid.tsx). While one page drags, the other pages' clicks go to it: they're lost, and their pointerups end its drag early. With one page holding a captured drag and another clicking, the clicking page received 106 of 168 clicks, and the dragging page received exactly the 62 missing pointerups. Run one after the other, every event arrived. Window focus isn't involved.

Changes

  • Firefox runs its tests in parallel, alongside Chromium and WebKit.
  • Each test file that drags a resize or drag-fill handle, dragFill.test.tsx and column/resizable.test.tsx, runs in a Firefox instance of its own, firefox (dragFill) and firefox (resizable), with a browser of its own. With a single file each, they don't need fileParallelism: false, which would make them wait for every other test. Vitest starts instances in the order of their names, which sort after browser (webkit), so with few workers, as in CI, WebKit starts first.
  • The dragFill command now waits for the grid to render the dragged-over cells before releasing the mouse, instead of waiting 20 ms in WebKit only. The grid fills the cells that it last rendered as dragged over, and with more tests running at once, 20 ms wasn't always enough.
  • AGENTS.md explains the split, and how to run all the Firefox instances.
  • On Windows, Vitest uses half the cores as workers, maxWorkers: '50%', to avoid the timeouts explained below.

Results

Full vp test runs take about 30 s on my machine, against 45–66 s on main. In CI, vp test took 74.6 s and 99.2 s in this PR's last two runs, against 106–152 s in recent runs where Firefox ran its tests one at a time, though CI timings vary a lot from one run to the next.

Timeouts on Windows

On Windows, the main Firefox instance's clicks often timed out at "performing click action" while WebKit ran:

  • While WebKit runs on Windows, threads of other processes wait on win32k's session-wide window manager lock in window-message system calls. The main thread of Firefox's parent process waits on it 10–16% of the time, against 0.2% without WebKit, and Firefox's mouse events then take 73–127 ms instead of 7–10 ms. Linux has no such lock, so CI isn't affected.
  • Juggler dispatches the mouse events of all of a browser's pages one after the other, in one queue on that thread (activateAndRun in TargetRegistry.js). With 12 pages in the main Firefox instance, the queue was busy 93% of the time, and one click's three events took 0.8–1 s each, which used up the 2 s action timeout.

How many pages run at once depended on stdin. Vitest's browser instances are projects, and projects don't inherit the watch: false of vitest run (vitest-dev/vitest#11721). With a terminal as stdin, they get watch mode's half the cores as maxWorkers, and runs passed. With another stdin, as in Claude Code or cached vp run tasks, they get all cores but one, and 6 of 8 full runs failed on my 32-core machine.

On Windows, maxWorkers is now '50%', as in a terminal: 6 of 6 full runs with a captured stdin passed, in 27.5–29.0 s instead of ~26 s. It's a safety margin rather than a guarantee, since heavy work alongside the tests can still slow Firefox down enough. Runs from a terminal and CI don't change. Running Firefox once WebKit is done, with sequence.groupOrder: 1, also avoided the timeouts, but full runs took 35–38 s.

Upstream

I reported the shared pointer ID in microsoft/playwright#43631, with a Playwright-only repro. Giving each page its own pointer ID in Juggler would let the pointer-capture files run in the main Firefox instance too. Projects not inheriting watch: false is vitest-dev/vitest#11721.

Vitest 5.0.5 launches a browser for each instance before the run, even when the files given on the command line exclude all of its tests, so running a single test file in Firefox launches three Firefox browsers (vitest-dev/vitest#11546). vitest-dev/vitest#11706 fixes it, but no release includes it yet.

CI, which runs vp run ci, passes.

🤖 Generated with Claude Code

Firefox sends the mouse events of all its pages with the same pointer ID, so a
page that captures the pointer receives the other pages' mouse events. The two
test files that drag a resize or drag-fill handle now run one at a time in a
browser of their own, and the other Firefox tests run in parallel, once the
other browsers are done.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
nstepien and others added 5 commits October 25, 2026 01:43
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The test files that capture the pointer each get a Firefox instance of their
own, which runs alongside Chromium and WebKit, instead of one serial instance
that ran after all the other tests. The other Firefox tests still run in
parallel once Chromium and WebKit are done, as their mouse events time out when
they compete with them for the CPU.

The `dragFill` command now waits for the grid to render the dragged-over cells
before releasing the mouse, instead of waiting 20ms in WebKit, which wasn't
always enough.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The main Firefox instance no longer waits for Chromium and WebKit to finish
in a project of its own, as the pointer-capture tests already run in browsers
of their own.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
On Windows, WebKit makes threads in other processes wait on win32k's
session-wide lock, which slows down the mouse events of Playwright's
Firefox. Firefox dispatches the mouse events of all its pages one after
the other, so with all cores but one as workers, its clicks time out.

In a terminal, the browser instances already get half the cores, as in
watch mode, since they don't inherit the `watch: false` of `vitest run`.
Runs with a captured stdin, like those of agents or cached `vp run`
tasks, got all cores but one and failed most of the time.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@nstepien
nstepien merged commit ab64935 into main Oct 25, 2026
6 checks passed
@nstepien
nstepien deleted the firefox-parallel branch October 25, 2026 16:08
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant