Skip to content

fix(junit): create a new Playwright when the previous launcher run closed it - #1988

Open
monkey (Develop-KIM) wants to merge 2 commits into
microsoft:mainfrom
Develop-KIM:fix-1751
Open

monkey (Develop-KIM) wants to merge 2 commits into
microsoft:mainfrom
Develop-KIM:fix-1751

Conversation

@Develop-KIM

Copy link
Copy Markdown
Contributor

Summary

  • Surefire's rerunFailingTestsCount (this repo's own CI uses it through PW_MAX_RETRIES) runs the launcher again in the same JVM and thread. The first run's PlaywrightRegistry has already closed every Playwright it created, but the ThreadLocal in PlaywrightExtension still hands one of them out, so the rerun launches a browser on a closed connection and fails with Playwright connection closed.
  • Reuses the ThreadLocal instance only when it belongs to the current run's registry and creates a fresh one otherwise. Clearing the ThreadLocal from close() would not reach other threads.
  • This only shows up with JUnit 5.13+: since Migrate ExtensionContext.Store.CloseableResource to AutoCloseable #1860 the registry is AutoCloseable, which older JUnit versions never close, so there the first Playwright just leaks and the rerun reuses it.
  • TestFixturesRerun runs RerunFixture through the launcher twice on one thread and fails on main with the error above. That needs junit-platform-launcher, which comes in through the junit-bom so it follows junit.version.

Fixes #1751

…osed it

Surefire's rerunFailingTestsCount (and similar retry tooling) executes the
launcher again in the same JVM and thread. The first run's PlaywrightRegistry
has already closed every Playwright it created, but the ThreadLocal in
PlaywrightExtension still holds one of them, so the rerun launches a browser
on a closed connection and fails with "Playwright connection closed".

Reuse the ThreadLocal instance only when it belongs to the current run's
registry and create a fresh one otherwise.

Fixes: microsoft#1751
…on project

The Docker job copies playwright/src/test into tools/test-local-installation
and compiles it with that project's own pom, which does not know the
junit-platform-launcher dependency TestFixturesRerun needs.
@Develop-KIM

Copy link
Copy Markdown
Contributor Author

The Docker jobs were failing because tools/test-local-installation compiles a copy of playwright/src/test with its own pom, which didn't have the junit-platform-launcher dependency the new test needs. Added it there in 3d8efa0; the smoke run in that project is green for me locally.

@yury-s Yury Semikhatsky (yury-s) left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The core fix looks right: PlaywrightRegistry.close() clears playwrightList, so owns() correctly returns false for a Playwright left in the ThreadLocal by a previous launcher run, and reuse within a single run is unaffected. I also verified via help:effective-pom that importing junit-bom first in <dependencyManagement> does not override the locally declared entries — junit-jupiter-api keeps provided, engine/params keep test, and junit-platform-launcher resolves to 1.14.1 to match Jupiter 5.14.1.

A few comments inline, plus one that I can't anchor because the file isn't in the diff:

tools/test-local-installation/pom.xml still hand-maintains junit.platform.version (1.11.0) alongside junit.version (5.11.0). Bumping one without the other yields a mismatched launcher/engine pair that only fails at test time in the docker job. Since this PR already imports junit-bom in the root pom, doing the same there would remove the drift.

Playwright playwright = threadLocalPlaywright.get();
if (playwright != null) {
// A previous launcher run on this thread (e.g. a surefire rerun) has already closed its Playwright.
if (playwright != null && registry.owns(playwright)) {

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

PlaywrightRegistry implements only AutoCloseable (it stopped implementing ExtensionContext.Store.CloseableResource in #1860), so on JUnit < 5.13 the store never closes it.

Combined with this new owns() gate, that turns a one-off leak into a per-run leak: on JUnit 5.9–5.12, a Surefire run with rerunFailingTestsCount used to reuse the single leaked Playwright across all reruns; now every rerun creates a new Playwright while the previous registry is still never closed, so up to retries × threads driver/node processes stay alive for the lifetime of the JVM.

Suggest having PlaywrightRegistry implement ExtensionContext.Store.CloseableResource as well, so older runtimes actually close the registry and this gate degrades gracefully (5.13+ closes it once via that path and won't double-close).

PlaywrightRegistry registry = PlaywrightRegistry.getOrCreateFor(extensionContext);
Playwright playwright = threadLocalPlaywright.get();
if (playwright != null) {
// A previous launcher run on this thread (e.g. a surefire rerun) has already closed its Playwright.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: this comment describes the fall-through case but sits directly above the reuse branch, so it reads as if it justifies the if body — easy for a later reader to "fix" by inverting the condition. Moving it below the if block, or rewording to something like "reuse only while the current run's registry still owns it", would avoid that.

@UsePlaywright
public class RerunFixture {
@Test
void usesPage(Page page) {

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This launches two real browsers serially (once per launcher run) while blocking a JUnit ForkJoinPool worker on summary.get(), inside a suite configured with junit.jupiter.execution.parallel.mode.classes.default = concurrent. On the 3-vCPU macos-latest runner — where the matrix already excludes WebKit for headroom — that's a meaningful addition.

The regression being guarded (Playwright connection closed) reproduces without a browser, so injecting APIRequestContext here instead of Page (or just making a driver round-trip on Playwright) would cover the same thing much more cheaply.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Bug]: Playwright-Junit lifecycle incompatible with maven retries 'surefire.rerunFailingTestsCount'

2 participants