Skip to content

test: pin the coverage-only describe gate the e2e specs share - #1238

Merged
mrbobbytables merged 1 commit into
mainfrom
quality/test-e2e-coverage-describe-gate
Oct 9, 2026
Merged

mrbobbytables merged 1 commit into
mainfrom
quality/test-e2e-coverage-describe-gate

Conversation

@hivecommons-hive

Copy link
Copy Markdown
Contributor

Test Improvement

Adds one file, tests/e2e-coverage-describe-gate.test.mjs. No spec is edited and
no production code is touched.

Eleven e2e specs hold cases only the coverage build can satisfy, and all eleven
gate themselves with the same hand-written ternary:

const describeCoverage =
  process.env.E2E_COVERAGE === '1' ? test.describe : test.describe.skip;

Nothing checked it. The gate fails open in the way
tests/e2e-data-dependent-skips.test.mjs was written to close for test.skip():
Playwright reports a skipped case as a non-failure, so a gate that stops matching
leaves its block skipped in both the End-to-end tests and End-to-end coverage
jobs and both stay green. That file states at lines 36-40 that it deliberately does
not scan these blocks — its subject is skips that turn on the committed data, and
these turn on the build. This PR is the other half.

The five checks, and why none is redundant

check the mutation it catches
inventory the gated set is exactly the eleven specs, so deleting a spec or its gated block is an edit here rather than a silent subtraction
shape the canonical expression verbatim, once, with one describeCoverage( block — catches === 1, swapped arms, a second differently-worded gate
no unconditional skip describe.skip( as a call anywhere in tests/e2e/
route a spec carrying a '/e2e-coverage-…' route literal that is not gated — it would 404 the gating job
harness parity the spec gate and isCoverageEnabled() in tests/tools/e2e-coverage.cjs are written out separately and can drift

Measured

Node v26.10.0, TZ=UTC, locally at 593b9fe (this branch's base).

npx prettier --check tests/e2e-coverage-describe-gate.test.mjs clean.
TZ=UTC node --test → 2091 tests, 0 failures (2086 before; this adds 5).

Not vacuous — four mutations, each applied to a real spec, reverted after:

mutation failing check(s)
=== '1' → === 1 in metrics-sparkline.spec.js shape
gate replaced with a bare test.describe inventory, shape
describeCoverage( → test.describe.skip( shape, no-unconditional-skip
a '/e2e-coverage-variant/community' literal added to the ungated footer.spec.js route

git status --porcelain after the run shows only the new file, so nothing leaked.

Interaction with #1203 — one line, whichever lands second

#1203 adds tests/e2e/reference-architectures-no-revision.spec.js, which carries the
same canonical gate verbatim (confirmed against that branch) and visits
/e2e-coverage-no-revision. It therefore needs one entry added to COVERAGE_GATED
in this file; the inventory check is what will say so, by name, rather than letting
it in unnoticed. No other overlap: #1203 edits fixture directories, build plumbing
and variant specs, and this PR edits none of them. Checked against every open
hold-gated PR — no other one touches tests/e2e/*.spec.js gates or adds a unit-suite
contract.

Related Issue

Closes #1237


Human review required.

— hive: agent=quality backend=copilot model=claude-opus-5 copilot=1.0.88

Eleven e2e specs gate their coverage-only blocks with the same hand-written
ternary, and nothing checked it. The gate fails open the way
tests/e2e-data-dependent-skips.test.mjs describes for test.skip(): a gate that
stops matching leaves its block skipped in both CI jobs and both stay green.

tests/e2e-coverage-describe-gate.test.mjs adds five checks to the unit suite --
inventory, canonical shape, no unconditional describe.skip(), coverage-only
routes are gated, and parity with isCoverageEnabled() in
tests/tools/e2e-coverage.cjs.

Closes #1237

Signed-off-by: quality <quality@hive.kubestellar.io>
@hivecommons-hive hivecommons-hive Bot added the hold label Oct 9, 2026
@hivecommons-hive

Copy link
Copy Markdown
Contributor Author

Important

Held for human review by the hive's ACMM level gate.

This PR was opened by the "quality" agent while Hive policy required a human checkpoint for that agent. Non-outreach agents are held at ACMM L3–L5; the outreach agent is always held because it publishes project-facing communication.

Hive will keep the hold label until a human removes it. Operators can make a deliberate one-off release during an ACMM level change with release_level_holds=true, but level changes never release this hold automatically.

@mrbobbytables
mrbobbytables added this pull request to the merge queue Oct 9, 2026
Merged via the queue into main with commit 270c926 Oct 9, 2026
7 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[quality] the coverage-only describe gate in the e2e specs fails open and nothing pins it

1 participant