Repository navigation
Run Firefox tests in parallel - #4233
Merged
Merged
Conversation
Firefox sends the mouse events of all its pages with the same pointer ID, so a page that captures the pointer receives the other pages' mouse events. The two test files that drag a resize or drag-fill handle now run one at a time in a browser of their own, and the other Firefox tests run in parallel, once the other browsers are done. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The test files that capture the pointer each get a Firefox instance of their own, which runs alongside Chromium and WebKit, instead of one serial instance that ran after all the other tests. The other Firefox tests still run in parallel once Chromium and WebKit are done, as their mouse events time out when they compete with them for the CPU. The `dragFill` command now waits for the grid to render the dragged-over cells before releasing the mouse, instead of waiting 20ms in WebKit, which wasn't always enough. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The main Firefox instance no longer waits for Chromium and WebKit to finish in a project of its own, as the pointer-capture tests already run in browsers of their own. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
On Windows, WebKit makes threads in other processes wait on win32k's session-wide lock, which slows down the mouse events of Playwright's Firefox. Firefox dispatches the mouse events of all its pages one after the other, so with all cores but one as workers, its clicks time out. In a terminal, the browser instances already get half the cores, as in watch mode, since they don't inherit the `watch: false` of `vitest run`. Runs with a captured stdin, like those of agents or cached `vp run` tasks, got all cores but one and failed most of the time. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
6 tasks done
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Firefox ran test files one at a time since #3772, which noted that "Tests in FF timeout when more than one worker is used". With Vitest 5, a project with
fileParallelism: falsealso only starts once every other project is done, so Firefox was the long pole of the suite.Why parallel Firefox tests failed
Vitest runs all of a project's pages in a single Firefox, each in its own context. Playwright's Firefox protocol synthesizes every page's mouse events in Firefox's parent process with the same pointer ID,
DEFAULT_MOUSE_POINTER_ID(PageHandler.jsin Juggler). Firefox's parent process tracks pointer capture per pointer ID across all tabs, and sends every mouse event with a captured pointer ID to the capturing tab (PointerEventHandler.cpp,EventStateManager::DispatchCrossProcessEvent).The grid captures the pointer while dragging a column's resize handle (
HeaderCell.tsx) or the drag-fill handle (DataGrid.tsx). While one page drags, the other pages' clicks go to it: they're lost, and their pointerups end its drag early. With one page holding a captured drag and another clicking, the clicking page received 106 of 168 clicks, and the dragging page received exactly the 62 missing pointerups. Run one after the other, every event arrived. Window focus isn't involved.Changes
dragFill.test.tsxandcolumn/resizable.test.tsx, runs in a Firefox instance of its own,firefox (dragFill)andfirefox (resizable), with a browser of its own. With a single file each, they don't needfileParallelism: false, which would make them wait for every other test. Vitest starts instances in the order of their names, which sort afterbrowser (webkit), so with few workers, as in CI, WebKit starts first.dragFillcommand now waits for the grid to render the dragged-over cells before releasing the mouse, instead of waiting 20 ms in WebKit only. The grid fills the cells that it last rendered as dragged over, and with more tests running at once, 20 ms wasn't always enough.AGENTS.mdexplains the split, and how to run all the Firefox instances.maxWorkers: '50%', to avoid the timeouts explained below.Results
Full
vp testruns take about 30 s on my machine, against 45–66 s onmain. In CI,vp testtook 74.6 s and 99.2 s in this PR's last two runs, against 106–152 s in recent runs where Firefox ran its tests one at a time, though CI timings vary a lot from one run to the next.Timeouts on Windows
On Windows, the main Firefox instance's clicks often timed out at "performing click action" while WebKit ran:
activateAndRuninTargetRegistry.js). With 12 pages in the main Firefox instance, the queue was busy 93% of the time, and one click's three events took 0.8–1 s each, which used up the 2 s action timeout.How many pages run at once depended on stdin. Vitest's browser instances are projects, and projects don't inherit the
watch: falseofvitest run(vitest-dev/vitest#11721). With a terminal as stdin, they get watch mode's half the cores asmaxWorkers, and runs passed. With another stdin, as in Claude Code or cachedvp runtasks, they get all cores but one, and 6 of 8 full runs failed on my 32-core machine.On Windows,
maxWorkersis now'50%', as in a terminal: 6 of 6 full runs with a captured stdin passed, in 27.5–29.0 s instead of ~26 s. It's a safety margin rather than a guarantee, since heavy work alongside the tests can still slow Firefox down enough. Runs from a terminal and CI don't change. Running Firefox once WebKit is done, withsequence.groupOrder: 1, also avoided the timeouts, but full runs took 35–38 s.Upstream
I reported the shared pointer ID in microsoft/playwright#43631, with a Playwright-only repro. Giving each page its own pointer ID in Juggler would let the pointer-capture files run in the main Firefox instance too. Projects not inheriting
watch: falseis vitest-dev/vitest#11721.Vitest 5.0.5 launches a browser for each instance before the run, even when the files given on the command line exclude all of its tests, so running a single test file in Firefox launches three Firefox browsers (vitest-dev/vitest#11546). vitest-dev/vitest#11706 fixes it, but no release includes it yet.
CI, which runs
vp run ci, passes.🤖 Generated with Claude Code