16k-baseline-trigger-guard: refuse an unmeasured baseline comparison on every trigger - #14
Merged
Merged
Conversation
…on every trigger `Gate - baseline may only decrease` is a required status check that had been passing having compared nothing. `github.event.before` is absent from the `schedule` and `workflow_dispatch` event payloads, so ci.yml expanded to `./tools/gate.sh baseline ""`, and `baseline_gate` matched the empty string in the same branch as the all-zeros sentinel: it printed "no previous ref (new branch); nothing to compare" -- about a branch that is `main` -- and returned 0. Every nightly run in this repository's history was green on zero comparisons, on commits whose parents carried the baseline file it declined to read. Measured before any fix: run 34028977996 (schedule, main, 778accb, job 101474937084) and run 34071415451 (workflow_dispatch, 86681a2, job 101589359652), both success, both logs quoted in FINDINGS F1 and F2. The guard for that payload already existed in `Gate - attestation`, and in run 34028977996 it measured while the baseline job did not. This reuses that `elif` rather than deriving a second one. What changed: - .github/workflows/ci.yml -- the attestation job's guard, applied to the baseline job, routing an empty-or-sentinel `before` to `${{ github.sha }}~1`. The `on:` comment claiming the schedule is "NOT for the baseline gate" is corrected; this diff falsified it in both halves. - tools/gate.sh -- `baseline_gate` fails closed on an empty ref. The all-zeros sentinel keeps its pass as its own labelled case. The header comment claimed "Fails closed on zero comparisons" while the empty-ref path did not; it now states its one exception in the same sentence as the rule. - apps/hacktui_core/test/ci_baseline_guard_test.exs -- pins both halves. An ordinary hard-blocking test, NOT a new `Gate -` job, so no ruleset change (CLAUDE.md 4c). It asserts the baseline job's ENTIRE body against a literal, splitting lines the way YAML does, and runs the gate rather than reading it. - tools/mutants/c5.tsv -- the rows that prove the pin bites. Several are reviewers' own survivors; HANDOFF.md section 10 names which. - BACKLOG.md -- section 13 records SCR-69's other half, still open: a force-push sets `before` to an unreachable SHA, which is not the sentinel, so the gate goes red on a tree that is fine. Section 14 records the documentation-drift class with a derivation per class. Section 15 records what the pin covers and what it does not. - HANDOFF.md -- section 10 for this slice. Section 1's stale figures are marked in place and NOT corrected; that sweep is section 14's work. Acceptance criteria met: AC1 (`baseline ""` now rc 1), AC2 (`baseline HEAD~1` unchanged), AC3 (the guards are token-equal, asserted not eyeballed), AC4 and AC8 (mutation harness reports zero survivors), AC6 (format, compile, secret-scan, test 0 failures, credo held, dialyzer held), AC7 (two lanes per round, on checkouts of the index). AC5 is verified after this lands, by dispatching CI on this branch and comparing to the pre-fix dispatch run 34071415451 on the same branch. Ten review rounds, twenty independent lanes, every one on a checkout of the index. The gate has been byte-identical since round 4 and no lane in any round could make it report a false green. What kept failing was the pin and the prose, and it was one mistake at eight levels: an assertion about something the assertion does not read. Reviewers defeated the pin with a parked early return, a second guard on a different payload key, a computed sentinel, an interpreter prefix, an extra space, a glob that contains no `gate.sh` token, and a carriage return that makes a dropped comment execute. Each is now a row that dies. The record was cut on the owner's decision of 2026-09-07: no transcribed count survives in a tracked file -- every number is either the command that derives it or it is absent -- because five consecutive rounds had blocked on a stale one, twice by the same mechanism. Evidence: internal/slices/16k-baseline-trigger-guard/FINDINGS.md (F1-F17), REVIEW.md, and logs/round{1..10}.{scope,r1.verdict,r2.verdict}. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01VNdY6wVS773UJgrHUeBN3Y Reviewed-diff: sha256:ee52b4f3a64c7eee5955fcd22667b99a1e0d12efbf565e41c3ebd289219f060f
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Gate - baseline may only decreaseis a required status check that had been passing having compared nothing.github.event.beforeis absent from thescheduleandworkflow_dispatchevent payloads, soci.ymlexpanded to./tools/gate.sh baseline "", andbaseline_gatematched the empty string in the same branch as the all-zeros sentinel: it printedno previous ref (new branch); nothing to compare— about a branch that ismain— and returned 0. Every nightly run in this repository's history was green on zero comparisons, on commits whose parents carried the baseline file it declined to read.Measured before any fix, both quoted from their job logs in FINDINGS F1/F2:
34028977996schedule778accb810147493708434071415451workflow_dispatch86681a2101589359652The guard for that payload already existed in
Gate - attestation, and in run34028977996it measured while the baseline job did not. This reuses thatelifrather than deriving a second one.What changed
.github/workflows/ci.yml— the attestation job's guard applied to thebaselinejob, routing an empty-or-sentinelbeforeto${{ github.sha }}~1. Theon:comment claiming the schedule is "NOT for the baseline gate" is corrected; this diff falsified it in both halves.tools/gate.sh—baseline_gatefails closed on an empty ref. The all-zeros sentinel keeps its pass as its own labelled case. The header comment claimed "Fails closed on zero comparisons" while the empty-ref path did not.apps/hacktui_core/test/ci_baseline_guard_test.exs(new) — pins both halves. An ordinary hard-blocking test, not a newGate -job, so no ruleset change (CLAUDE.md §4c). It asserts thebaselinejob's entire body against a literal, splits lines the way YAML does, and runs the gate rather than reading it.tools/mutants/c5.tsv— the rows that prove the pin bites. Several are reviewers' own survivors;HANDOFF.md§10 names which.BACKLOG.md— §13 records SCR-69's still-open half; §14 the documentation-drift class; §15 what the pin covers and what it does not.HANDOFF.md— §10 for this slice. §1's stale figures are marked in place and deliberately not corrected.Review
Ten rounds, twenty independent lanes, each on a checkout of the index.
The gate has been byte-identical since round 4 and no lane in any round could make it report a false green. What kept failing was the pin and the prose — one mistake at eight levels, each an assertion about something the assertion does not read. Reviewers defeated the pin with a parked early return, a second guard on a different payload key, a computed sentinel, an interpreter prefix, an extra space, a glob containing no
gate.shtoken, and a carriage return that makes a dropped comment execute. Each is now a mutant row that dies.The record was cut on the owner's decision of 2026-09-07: no transcribed count survives in a tracked file — every number is either the command that derives it or it is absent — because five consecutive rounds had blocked on a stale one, twice by the same mechanism. The final two lanes verified the closing delta was additive-free by exact preimage reconstruction.
Still open, deliberately
SCR-69's other half: a force-push sets
beforeto a well-formed but unreachable SHA, which is not the sentinel, so the gate goes red on a tree that is fine. Opposite failure direction, different remedy, recorded inBACKLOG.md§13.🤖 Generated with Claude Code
https://claude.ai/code/session_01VNdY6wVS773UJgrHUeBN3Y