Skip to content

test(e2e): cover the e2e coverage reporter's partial-capture fallbacks - #1209

Merged
mrbobbytables merged 1 commit into
mainfrom
quality/e2e-coverage-report-regions
Oct 9, 2026
Merged

mrbobbytables merged 1 commit into
mainfrom
quality/e2e-coverage-report-regions

Conversation

@hivecommons-hive

Copy link
Copy Markdown
Contributor

Closes #1208.

What

Adds one new test file, tests/e2e-coverage-partial-capture.test.mjs
(8 tests). No production code changes, no gate threshold changes.

tests/tools/e2e-coverage-report.mjs was the lowest-scoring file in the
harness tree. Every one of its gaps is a sub-line region — the file sits at
100.00% lines — so the line gates could not see any of them.

before after
tests/tools/e2e-coverage-report.mjs regions 93.65% 95.41%
lines 100.00% 100.00%

How

The readers are not exported, so the tests go in through
collectE2ECoverage() by writing the artifact a browser would have written:

  • a capture entry with no functions
  • a capture entry whose function carries no ranges
  • an artifact with no result array — counted as zero captured scripts,
    and the healthy artifact beside it still reported
  • four parameterised v8-coverage-error cases, degrading the message field
    by field down to a payload carrying nothing but its kind, asserting the
    report fails each time
  • uncovered lines returned in ascending numeric order (the existing fixtures
    all produce a single uncovered line, so the comparator never ran)

Two notes for the reviewer:

  • collectE2ECoverage() throws No src/** coverage was attributable when
    nothing executable is measured, so each degenerate artifact is paired with
    a healthy one in the same run. That is the only way to assert the
    degenerate shape is survived rather than merely fatal.
  • Single-line function bodies always report as covered, so the fixture for
    the sort comparator uses multi-line bodies to produce genuinely uncovered
    lines.

Remaining gap

18 regions stay uncovered. They sit behind non-exported readers
(getLineCoverage, getRegionCoverage, isPhantomRegion,
isContradictedRegion) that consume istanbul objects produced by
v8-to-istanbul, so their malformed-shape guards cannot be driven from the
public surface at all. That is a separate deliverable — either export those
readers for direct unit testing or document them as unreachable — and is
filed separately rather than forced into this PR.

Files claimed

tests/e2e-coverage-partial-capture.test.mjs (new) — nothing else.

Disjoint from the open hold-gated PRs: #1206 touches
scripts/svg-active-content.mjs, #1203 touches
tests/tools/e2e-coverage-builds.mjs, and #1207 touches
tests/tools/coverage-report.mjs + package.json. Verified with
gh pr view <n> --json files. In particular this PR deliberately does not
ratchet --check-harness-file-regions from 93 to 95, because package.json
and tests/coverage-gate-thresholds.test.mjs are claimed by #1207; that
ratchet belongs in a follow-up once #1207 lands.

Verification

  • npm run test:unit:coverage:check — exits 0
  • npm run check — exits 0
  • full suite — 2094/2094 pass

node v26.10.0, TZ=UTC, cut from origin/main @ 593b9fe.

Held for human review.

— hive: agent=quality backend=copilot model=claude-opus-5 copilot=1.0.88

tests/tools/e2e-coverage-report.mjs was the lowest-scoring file in the
harness tree at 93.65% regions. What Playwright's coverage.stopJSCoverage()
hands back over CDP is not guaranteed-shaped -- a page torn down
mid-navigation contributes an entry with no `functions`, a script the browser
never entered contributes a function with no `ranges`, a worker that failed
before the capture writes a `v8-coverage-error` artifact instead -- and
collectE2ECoverage() reads all of it through `??` fallbacks that no test
exercised.

tests/e2e-coverage-partial-capture.test.mjs drives each shape by writing the
artifact a browser would have written, because none of the readers is
exported and collectE2ECoverage() is the only way in. The tests pin which
shapes are fatal and which are not: a partial *capture* must not fail the
run, because artifacts are written per page by parallel workers and one
degenerate page would otherwise discard every measurement beside it, while a
capture *error* must fail it, because that means a page's coverage is missing
rather than empty and scoring the run without it reports a percentage over a
denominator that silently lost a page. The error message is covered
field-by-field down to a payload carrying nothing but its kind, which is
exactly the case where the operator has least to go on.

One more region: uncovered lines are now asserted to come back in ascending
numeric order, with multi-line function bodies so an unexecuted function
leaves whole lines uncovered rather than a sub-line region. That list is what
the text artifact prints, and a default lexicographic sort would put 10
before 4 on any file long enough for the list to matter.

tests/tools/e2e-coverage-report.mjs: 93.65% -> 95.41% regions (lines stay at
100.00%). Measured with `npm run test:unit:coverage` (node v26.10.0, TZ=UTC)
at 593b9fe. No production code changes; no gate threshold is touched.

Signed-off-by: quality <quality@hive.kubestellar.io>
@hivecommons-hive hivecommons-hive Bot added the hold label Oct 8, 2026
@hivecommons-hive

Copy link
Copy Markdown
Contributor Author

Important

Held for human review by the hive's ACMM level gate.

This PR was opened by the "quality" agent while Hive policy required a human checkpoint for that agent. Non-outreach agents are held at ACMM L3–L5; the outreach agent is always held because it publishes project-facing communication.

Hive will keep the hold label until a human removes it. Operators can make a deliberate one-off release during an ACMM level change with release_level_holds=true, but level changes never release this hold automatically.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[quality] e2e coverage reporter's partial-capture fallbacks are unexercised

1 participant