Skip to content

test(e2e): add a per-file region floor to the e2e coverage reporter - #1233

Merged
mrbobbytables merged 1 commit into
mainfrom
quality/e2e-per-file-region-floor
Oct 9, 2026
Merged

mrbobbytables merged 1 commit into
mainfrom
quality/e2e-per-file-region-floor

Conversation

@hivecommons-hive

Copy link
Copy Markdown
Contributor

Test Improvement

Adds --check-source-file-regions <pct> to tests/tools/e2e-coverage-report.mjs,
applying the region floor to each src/** file on its own instead of only to
the aggregate.

--check-source-regions is an aggregate over all of src/** together, so the
slack it allows can be spent entirely on one file. At the 95% the e2e-coverage
job asks for, a single component may sit at nothing and the job stays green. The
companion line gate does not catch that either — a lost region need not be a
lost line, because an unexecuted ternary arm or ?? fallback sits on a line the
surrounding statement still covers.

tests/tools/coverage-report.mjs:45-58 already made exactly this argument for the
unit run and closed the hole with --check-source-file-regions, which
package.json passes as 97. This is the same guarantee for the browser run.

The flag is opt-in and defaults to off, so no existing caller changes
behaviour and CI is unaffected by this PR alone.

Files and functions claimed

Nothing else in the repository is touched; CONTRIBUTING.md is deliberately left
alone, because the sentence under its snippet states that the flags shown are the
ones CI enforces, and this PR does not change what CI enforces.

The fixture is the proof, not an illustration

Two bundles, one src file each: one fully covered, one at 50% regions. The run
reports 66.67% aggregate regions, clearing --check-source-regions 66, while
src/components/Uneven/index.js alone sits at 50% and fails
--check-source-file-regions 66. That is the hole, reproduced.

Each file gets its own bundle and its own map deliberately: v8-to-istanbul
resolves a single map naming two sources down to one source, so a merged fixture
would silently measure one file and prove nothing. That was observed, not assumed.

Five tests: the aggregate-vs-per-file demonstration above, the artifact-ordering
contract (a failing gate must still leave the report that explains it), an
exactly-meets-the-floor pass, the default-off case, and rejection of
non-percentage thresholds.

No vacuous-pass guard is included, unlike the unit reporter's equivalent:
collectE2ECoverage already refuses a run that attributed nothing to src/**
("No src/** coverage was attributable"), so the offender list cannot be empty for
the reason that guard exists to catch. Adding one would only create a permanently
unreachable region — the residual #1210 exists to stop accumulating.

Verification

  • node --test tests/e2e-coverage-report-file-region-floor.test.mjs — 5/5 pass.
  • npm run check and npm run test:unit — 2091/2091 pass, no regressions.
  • npm run test:unit:coverage: the new code is fully covered.
    tests/tools/e2e-coverage-report.mjs goes 93.67% → 93.74% regions with the
    uncovered region set unchanged, so it stays above the 93 per-file harness
    floor proposed in test(coverage): give the tests/tools/ harness a floor of its own #1207.
  • npx prettier --check clean on both files.

Related Issue

Refs #1232 (needs-human: the other half of the fix — passing the new flag in the
e2e-coverage render step of .github/workflows/ci.yml, plus the matching
FLOORS entry in tests/e2e-coverage-gate.test.mjs and the CONTRIBUTING.md
snippet — touches .github/workflows/**, which this agent's token tier cannot
push; #1232 carries the exact replacement text). The two halves are independent:
this one is inert until the workflow passes the flag, so they need not land
together.


Filed by quality agent (hold-gated mode). Human review required.

— hive: agent=quality backend=copilot model=claude-opus-5 copilot=1.0.88

--check-source-regions is an aggregate over the whole of src/** together, so
the slack it allows can be spent entirely on one file. At the 95% the
e2e-coverage job asks for, a single component may sit at nothing and the job
stays green. The companion line gate does not catch it either: a lost region
need not be a lost line, because an unexecuted ternary arm or ?? fallback sits
on a line the surrounding statement still covers.

tests/tools/coverage-report.mjs already made this argument for the unit run and
closed the hole with --check-source-file-regions, which package.json passes as
97. This gives the browser run the same flag.

The flag is opt-in and defaults to off, so no existing caller changes
behaviour; wiring it into .github/workflows/ci.yml is the remaining half of
issue #1232.

The fixture is the proof rather than an illustration: two bundles, one src file
each, chosen so the run clears a 66% aggregate while one file sits at 50%.
Each file gets its own bundle and map deliberately -- v8-to-istanbul resolves a
single map naming two sources down to one source, so a merged fixture would
silently measure one file and prove nothing.

No vacuous-pass guard is included, unlike the unit reporter's equivalent:
collectE2ECoverage already refuses a run that attributed nothing to src/**, so
the offender list cannot be empty for the reason that guard exists to catch.
Adding one would only create a permanently unreachable region.

Refs #1232 (needs-human: the .github/workflows/ci.yml half of the fix cannot be
pushed by this agent's token tier, which lacks the Workflows permission)

Signed-off-by: quality <quality@hive.kubestellar.io>
@hivecommons-hive hivecommons-hive Bot added the hold label Oct 9, 2026
@hivecommons-hive

Copy link
Copy Markdown
Contributor Author

Important

Held for human review by the hive's ACMM level gate.

This PR was opened by the "quality" agent while Hive policy required a human checkpoint for that agent. Non-outreach agents are held at ACMM L3–L5; the outreach agent is always held because it publishes project-facing communication.

Hive will keep the hold label until a human removes it. Operators can make a deliberate one-off release during an ACMM level change with release_level_holds=true, but level changes never release this hold automatically.

@hivecommons-hive hivecommons-hive Bot added quality Approved by a Hive merger/owner for auto-merge on green CI testing Approved by a Hive merger/owner for auto-merge on green CI labels Oct 9, 2026
@mrbobbytables

Copy link
Copy Markdown
Member

Heads-up: PRs1211 1240 also edit tests/tools/e2e-coverage-report.mjs. Expect merge conflicts as each lands; please merge in sequence (rebase after each) or consolidate.

@mrbobbytables
mrbobbytables added this pull request to the merge queue Oct 9, 2026
Merged via the queue into main with commit 360f8d0 Oct 9, 2026
7 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

hold quality Approved by a Hive merger/owner for auto-merge on green CI testing Approved by a Hive merger/owner for auto-merge on green CI

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant