Repository navigation
test(ci): pin the supply-chain scanners the compliant tree never exercises - #1247
Merged
Merged
Conversation
…cises Extract the workflow scanners and the three retiring-baseline guards from tests/ci-supply-chain.test.mjs into tests/helpers-ci-supply-chain.mjs, and drive them from tests/ci-supply-chain-helpers.test.mjs over synthetic workflow documents that do offend. Every workflow in .github/workflows/ is compliant and all three KNOWN_* baselines are empty, so the contract never reached a scanner's offence arm or a baseline guard's body: 86.12% lines / 84.94% regions, with the three baseline-guard bodies uncovered in full. A scanner that stopped detecting would have left the contract green over a tree it was no longer checking. Closes #1246 Signed-off-by: quality <quality@hive.kubestellar.io>
Contributor
Author
|
Important Held for human review by the hive's ACMM level gate. This PR was opened by the "quality" agent while Hive policy required a human checkpoint for that agent. Non-outreach agents are held at ACMM L3–L5; the Hive will keep the |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Test Improvement
tests/ci-supply-chain.test.mjsasserts the workflow supply-chain contract —SHA pinning, version comments,
permissions:,persist-credentials,timeout-minutes, runner-image pinning — over the real.github/workflows/**tree. That tree is fully compliant and all three retiring baselines
(
KNOWN_PERSISTING_CHECKOUTS,KNOWN_UNBOUNDED_JOBS,KNOWN_FLOATING_RUNNERS)are
new Set([]), so nothing in the file ever reached a scanner's offence armor a baseline guard's body.
tests/helpers-ci-supply-chain.mjs, each returning its offender list insteadof asserting inline. Behaviour and every assertion message are unchanged.
tests/ci-supply-chain.test.mjs, which still runsthe same 15 contract tests over the same tree.
tests/ci-supply-chain-helpers.test.mjs— 21 tests driving eachscanner over synthetic workflow documents that do offend, and each baseline
guard over a non-empty baseline, covering the "entry names a workflow that no
longer exists", "entry names a job that no longer exists" and "entry is now
compliant" arms. The fixtures are written as YAML and parsed with the same
yamlreader the contract uses, so they cannot drift from the real shape.Why it mattered
With
unpinnedActionsmutated to skip every step — a scanner that detectsnothing — the contract still reports 15 pass / 0 fail, while the new file
reports 20 pass / 1 fail. That is the hole, measured: before this change a
silently-broken scanner was a green check over an unverified contract.
The three "baseline retires itself" tests are the guards that force an
exception to be deleted in the same change that fixes the workflow. They had
never executed a line, so the first contributor to re-add a baseline entry
would have been the first person to run that code.
Measured
npm run test:unit:coverage(TZ=UTC node tests/tools/coverage-report.mjs,node v26.10.0) at
59ba83c:tests/ci-supply-chain.test.mjsbeforetests/ci-supply-chain.test.mjsaftertests/helpers-ci-supply-chain.mjsaftertests/ci-supply-chain-helpers.test.mjsafterAll files
99.47 | 95.51→99.59 | 95.80.src filesstay at100.00 | 100.00. Full unit suite 2167/2167 pass;npm run checkandnpm run test:unit:coverage:checkboth exit 0; Prettierclean on all three files.
No e2e evidence is claimed or needed: these files read
.github/workflows/**off disk and never run in a browser, so no end-to-end suite can cover them.
Scope / overlap
Touches
tests/ci-supply-chain.test.mjs,tests/helpers-ci-supply-chain.mjs(new) and
tests/ci-supply-chain-helpers.test.mjs(new) — and no workflowfile, so nothing here needs workflow-write permission. None of the open
hold-gated PRs touch any of the three: #1225 is the closest neighbour and is
confined to
tests/ci-pipeline-exit-codes.test.mjs; #1231 applies this samehelper-extraction pattern to
tests/docs-contract.test.mjs, a disjoint file.Branch cut from a fresh
origin/main.Related Issue
Closes #1246
Filed by quality agent (hold-gated mode). Human review required.
— hive: agent=quality backend=copilot model=claude-opus-5 copilot=1.0.88