Skip to content

test: assert every declared e2e fixture build is visited by a spec - #1260

Merged
mrbobbytables merged 1 commit into
mainfrom
quality/test-fixture-build-visited
Oct 10, 2026
Merged

mrbobbytables merged 1 commit into
mainfrom
quality/test-fixture-build-visited

Conversation

@hivecommons-hive

Copy link
Copy Markdown
Contributor

Test Improvement

Adds one case to tests/e2e-data-fixture-integrity.test.mjs: every name in
coverageBuildNames() must appear as a route in at least one tests/e2e/*.spec.js.

A fixture build exists because tests/e2e/fixtures/data-<name>/ exists, and each one
is a full Docusaurus compile (tests/tools/e2e-coverage-builds.mjs), so the
end-to-end coverage job's wall time is linear in those directories. The four cases
already in this file hold every committed overlay to properties a stale fixture would
break; none of them asks whether a browser ever opens the site compiled from it. A
build no spec visits is served, never requested, and produces no capture artifact, so
tests/tools/e2e-coverage-report.mjs has nothing of that build's to union and the arm
its overlays hold open goes back to being uncovered — at the price of a compile on
every run.

Scope: tests/e2e-data-fixture-integrity.test.mjs only. No production code, no
package.json thresholds, no workflow files, no change to any existing case.

Verification

  • TZ=UTC node --test tests/e2e-data-fixture-integrity.test.mjs — 6 pass, 0 fail.
  • Checked against a planted failure, not only against green: copying one overlay into
    tests/e2e/fixtures/data-unvisited/ makes the new case — and only the new case —
    fail naming that directory, while the four existing overlay cases stay green.
  • npm run test:unit:coverage:check (node v26.10.0, TZ=UTC) exits 0:
    src files | 100.00 | 100.00, harness files | 100.00 | 99.19,
    all files | 99.65 | 96.13.
  • npx prettier --check clean on the edited file.

Related Issue

Closes #1259


Filed by quality agent (hold-gated mode). Human review required.

— hive: agent=quality backend=copilot model=claude-opus-5 copilot=1.0.88

A fixture build exists because tests/e2e/fixtures/data-<name>/ exists, and
each one is a full Docusaurus compile, so the end-to-end coverage job's wall
time is linear in those directories. The existing cases in this file hold
every overlay to properties a stale fixture would break; none of them asks
whether a browser ever opens the site compiled from it.

A build no spec visits is served, never requested, and produces no capture
artifact, so the union gains nothing from it and the arm its overlays hold
open goes back to being uncovered -- at the price of a compile on every run.
That is the same silent-no-op failure the overlay cases exist for, one level
up: the symptom is a slower job and an uncovered arm, never an error.

Matched against the spec sources with comments stripped, because the variant
specs name one another's base URLs in their preambles; a commented mention is
documentation, not a visit. The route boundary keeps a short build name from
being credited by a longer one's URL.

Closes #1259

Signed-off-by: quality <quality@hive.kubestellar.io>
@hivecommons-hive

Copy link
Copy Markdown
Contributor Author

⚠️ The hold label did not get applied when this PR was opened: GitHub returned 504 on POST /repos/cncf/endusers/issues/1260/labels, and a retry through the relay reports the operation needs reconciliation before it can be retried. Agents cannot set hold themselves — it is a hive-controlled label.

This PR is hold-gated and must not be auto-merged. Please apply hold (and quality, testing) manually. Nothing about the diff is affected: it is one new test case in tests/e2e-data-fixture-integrity.test.mjs.


🐝 Hive Agent: quality | Instance: hosted-available-lke648397-260827-5n31 | SHA: 28225fd

— hive: agent=quality backend=copilot model=claude-opus-5 copilot=1.0.88

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[quality] nothing checks that a declared e2e fixture build is visited by a spec

1 participant