Skip to content

test(coverage): give the tests/tools/ harness a floor of its own - #1207

Merged
mrbobbytables merged 1 commit into
mainfrom
quality/harness-coverage-floor
Oct 9, 2026
Merged

mrbobbytables merged 1 commit into
mainfrom
quality/harness-coverage-floor

Conversation

@hivecommons-hive

Copy link
Copy Markdown
Contributor

Test Improvement

tests/tools/ is the third SOURCE_ROOTS entry
(tests/tools/coverage-report.mjs:91), so --require-source-files insists
every module in it be measured — and then no ratio gate scored what the
measurement said. --check-source, --check-source-regions and
--check-source-file-regions are all built from isSourceFile(), which
classifies everything under tests/ as non-source, so the harness entered
none of their ratios. The only floor left over it was --check-regions, an
all-files aggregate over ~8900 regions that the suite's own test files
dominate: tests/tools/e2e-coverage-report.mjs sits at 93.65% regions and
nothing failed.

That tree is where the coverage reporters, the Playwright coverage fixture,
the data-overlay loader and the JSX/DOM harness live, so a gap there is a gap
in the instrument every other gate reads.

What this changes

  • tests/tools/coverage-report.mjs reports a harness files row beside
    src files and all files, built from the same SOURCE_ROOTS entry
    --require-source-files already walks — isHarnessFile() is derived from
    SOURCE_ROOTS rather than restating the directory, so the row and the
    file-set floor cannot come to name different trees.
  • Three new floors, the same three shapes as the source gates, each with the
    same refusal to pass vacuously over a run that recorded no harness file:
    --check-harness, --check-harness-regions, --check-harness-file-regions.
    The per-file failure names the file, its ratio and its uncovered region start
    lines, the way --check-source-file-regions does.
  • src files and harness files now share one accumulator, so a change to how
    one row is built cannot quietly stop applying to the other.
  • package.json's test:unit:coverage:check passes all three — required by
    the existing "every threshold the reporter supports is actually gated"
    test — and tests/coverage-gate-thresholds.test.mjs records the matching
    floors.
  • tests/coverage-report-harness-floor.test.mjs (new, 10 tests) covers the
    predicate, the row, each floor failing on a real run, the per-file message
    shape, and all three vacuous-pass guards.
  • CONTRIBUTING.md and AGENTS.md state the new floors. The four-number phrase
    tests/docs-threshold-parity.test.mjs pins is left exactly as it was;
    the harness floors are stated in a clause of their own.

Floors, and how they were set

Measured locally at 593b9fe on the suite this PR ships
(npm run test:unit:coverage:check, node v26.10.0, TZ=UTC):

src files      | 100.00 |  99.96 | 9157/9157 lines | 2595/2596 regions
harness files  | 100.00 |  97.95 | 3522/3522 lines | 1384/1413 regions
all files      |  99.46 |  95.22 | 52182/52464 lines | 8487/8913 regions

Lowest harness file: tests/tools/e2e-coverage-report.mjs at 93.65% regions.
Floors are set from that and rounded down: --check-harness 100,
--check-harness-regions 97, --check-harness-file-regions 93.

Verified locally: npm run test:unit:coverage:check exits 0, npm run check
exits 0, and the full node --test suite is 2099/2099 green.

Scope and overlap

Files claimed: tests/tools/coverage-report.mjs (the report() summary rows,
the PERCENT_FLAGS table and the gate block in main()),
package.json's scripts.test:unit:coverage:check,
tests/coverage-gate-thresholds.test.mjs, the new
tests/coverage-report-harness-floor.test.mjs, and the coverage-threshold
prose in CONTRIBUTING.md / AGENTS.md.

Related Issue

Closes #1192


Filed by quality agent (hold-gated mode). Human review required.

— hive: agent=quality backend=copilot model=claude-opus-5 copilot=1.0.88

`tests/tools/` is the third SOURCE_ROOTS entry, so --require-source-files
insists every module in it be measured -- and then no ratio gate scored it.
--check-source, --check-source-regions and --check-source-file-regions are
all built from isSourceFile(), which drops everything under tests/, so the
only floor over the harness was --check-regions, an all-files aggregate over
~8900 regions that the suite's own test files dominate. The measuring
apparatus was required to be present and held to no standard, while
tests/tools/e2e-coverage-report.mjs sat at 93.65% regions.

Report `harness files` as its own row beside `src files`, built from the same
SOURCE_ROOTS entry --require-source-files walks (isHarnessFile() is derived
from SOURCE_ROOTS rather than restating the directory, so the row and the
file-set floor cannot come to name different trees). Add --check-harness,
--check-harness-regions and --check-harness-file-regions: the same three
shapes as the source gates, with the same refusal to pass vacuously over a
run that recorded no harness file, and the per-file failure naming the file,
its ratio and its uncovered region start lines.

It stays a separate row rather than widening `src files`, for the reason
isSourceFile() already gives: the harness is not code this repository ships,
and merging the two would let a harness gap be paid for out of src/'s slack
-- the averaging --check-source-file-regions exists to stop.

`src files` and `harness files` now share one accumulator, so a change to how
one row is built cannot quietly stop applying to the other.

Floors, measured on the suite this commit ships (harness files 100.00% lines
/ 97.95% regions, 3522/3522 lines, 1384/1413 regions; lowest harness file
tests/tools/e2e-coverage-report.mjs at 93.65%): --check-harness 100,
--check-harness-regions 97, --check-harness-file-regions 93. Recorded in
tests/coverage-gate-thresholds.test.mjs, whose "every threshold the reporter
supports is actually gated" test requires the gate script to pass all three.

CONTRIBUTING.md and AGENTS.md state the new floors beside the existing ones.
The four-number phrase tests/docs-threshold-parity.test.mjs pins is left
exactly as it was; the harness floors are stated in a clause of their own.

Closes #1192

Signed-off-by: quality <quality@hive.kubestellar.io>
@hivecommons-hive hivecommons-hive Bot added the hold label Oct 8, 2026
@hivecommons-hive

Copy link
Copy Markdown
Contributor Author

Important

Held for human review by the hive's ACMM level gate.

This PR was opened by the "quality" agent while Hive policy required a human checkpoint for that agent. Non-outreach agents are held at ACMM L3–L5; the outreach agent is always held because it publishes project-facing communication.

Hive will keep the hold label until a human removes it. Operators can make a deliberate one-off release during an ACMM level change with release_level_holds=true, but level changes never release this hold automatically.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[quality] tests/tools/ is required to be measured but scored by no source coverage gate

1 participant