Skip to content

16k-baseline-trigger-guard: refuse an unmeasured baseline comparison on every trigger - #14

Merged
HackTuah merged 1 commit into
mainfrom
slice/16k-baseline-trigger-guard
Sep 7, 2026
Merged

16k-baseline-trigger-guard: refuse an unmeasured baseline comparison on every trigger#14
HackTuah merged 1 commit into
mainfrom
slice/16k-baseline-trigger-guard

Conversation

@HackTuah

@HackTuah HackTuah commented Sep 7, 2026

Copy link
Copy Markdown
Member

Gate - baseline may only decrease is a required status check that had been passing having compared nothing.

github.event.before is absent from the schedule and workflow_dispatch event payloads, so ci.yml expanded to ./tools/gate.sh baseline "", and baseline_gate matched the empty string in the same branch as the all-zeros sentinel: it printed no previous ref (new branch); nothing to compare — about a branch that is main — and returned 0. Every nightly run in this repository's history was green on zero comparisons, on commits whose parents carried the baseline file it declined to read.

Measured before any fix, both quoted from their job logs in FINDINGS F1/F2:

run event head job result
34028977996 schedule 778accb8 101474937084 success, 0 comparisons
34071415451 workflow_dispatch 86681a2 101589359652 success, 0 comparisons

The guard for that payload already existed in Gate - attestation, and in run 34028977996 it measured while the baseline job did not. This reuses that elif rather than deriving a second one.

What changed

  • .github/workflows/ci.yml — the attestation job's guard applied to the baseline job, routing an empty-or-sentinel before to ${{ github.sha }}~1. The on: comment claiming the schedule is "NOT for the baseline gate" is corrected; this diff falsified it in both halves.
  • tools/gate.shbaseline_gate fails closed on an empty ref. The all-zeros sentinel keeps its pass as its own labelled case. The header comment claimed "Fails closed on zero comparisons" while the empty-ref path did not.
  • apps/hacktui_core/test/ci_baseline_guard_test.exs (new) — pins both halves. An ordinary hard-blocking test, not a new Gate - job, so no ruleset change (CLAUDE.md §4c). It asserts the baseline job's entire body against a literal, splits lines the way YAML does, and runs the gate rather than reading it.
  • tools/mutants/c5.tsv — the rows that prove the pin bites. Several are reviewers' own survivors; HANDOFF.md §10 names which.
  • BACKLOG.md — §13 records SCR-69's still-open half; §14 the documentation-drift class; §15 what the pin covers and what it does not.
  • HANDOFF.md — §10 for this slice. §1's stale figures are marked in place and deliberately not corrected.

Review

Ten rounds, twenty independent lanes, each on a checkout of the index.

The gate has been byte-identical since round 4 and no lane in any round could make it report a false green. What kept failing was the pin and the prose — one mistake at eight levels, each an assertion about something the assertion does not read. Reviewers defeated the pin with a parked early return, a second guard on a different payload key, a computed sentinel, an interpreter prefix, an extra space, a glob containing no gate.sh token, and a carriage return that makes a dropped comment execute. Each is now a mutant row that dies.

The record was cut on the owner's decision of 2026-09-07: no transcribed count survives in a tracked file — every number is either the command that derives it or it is absent — because five consecutive rounds had blocked on a stale one, twice by the same mechanism. The final two lanes verified the closing delta was additive-free by exact preimage reconstruction.

Still open, deliberately

SCR-69's other half: a force-push sets before to a well-formed but unreachable SHA, which is not the sentinel, so the gate goes red on a tree that is fine. Opposite failure direction, different remedy, recorded in BACKLOG.md §13.

🤖 Generated with Claude Code

https://claude.ai/code/session_01VNdY6wVS773UJgrHUeBN3Y

…on every trigger

`Gate - baseline may only decrease` is a required status check that had been passing
having compared nothing. `github.event.before` is absent from the `schedule` and
`workflow_dispatch` event payloads, so ci.yml expanded to `./tools/gate.sh baseline ""`,
and `baseline_gate` matched the empty string in the same branch as the all-zeros
sentinel: it printed "no previous ref (new branch); nothing to compare" -- about a branch
that is `main` -- and returned 0. Every nightly run in this repository's history was green
on zero comparisons, on commits whose parents carried the baseline file it declined to
read. Measured before any fix: run 34028977996 (schedule, main, 778accb, job
101474937084) and run 34071415451 (workflow_dispatch, 86681a2, job 101589359652), both
success, both logs quoted in FINDINGS F1 and F2.

The guard for that payload already existed in `Gate - attestation`, and in run
34028977996 it measured while the baseline job did not. This reuses that `elif` rather
than deriving a second one.

What changed:

- .github/workflows/ci.yml -- the attestation job's guard, applied to the baseline job,
  routing an empty-or-sentinel `before` to `${{ github.sha }}~1`. The `on:` comment
  claiming the schedule is "NOT for the baseline gate" is corrected; this diff falsified
  it in both halves.
- tools/gate.sh -- `baseline_gate` fails closed on an empty ref. The all-zeros sentinel
  keeps its pass as its own labelled case. The header comment claimed "Fails closed on
  zero comparisons" while the empty-ref path did not; it now states its one exception in
  the same sentence as the rule.
- apps/hacktui_core/test/ci_baseline_guard_test.exs -- pins both halves. An ordinary
  hard-blocking test, NOT a new `Gate -` job, so no ruleset change (CLAUDE.md 4c). It
  asserts the baseline job's ENTIRE body against a literal, splitting lines the way YAML
  does, and runs the gate rather than reading it.
- tools/mutants/c5.tsv -- the rows that prove the pin bites. Several are reviewers' own
  survivors; HANDOFF.md section 10 names which.
- BACKLOG.md -- section 13 records SCR-69's other half, still open: a force-push sets
  `before` to an unreachable SHA, which is not the sentinel, so the gate goes red on a
  tree that is fine. Section 14 records the documentation-drift class with a derivation
  per class. Section 15 records what the pin covers and what it does not.
- HANDOFF.md -- section 10 for this slice. Section 1's stale figures are marked in place
  and NOT corrected; that sweep is section 14's work.

Acceptance criteria met: AC1 (`baseline ""` now rc 1), AC2 (`baseline HEAD~1` unchanged),
AC3 (the guards are token-equal, asserted not eyeballed), AC4 and AC8 (mutation harness
reports zero survivors), AC6 (format, compile, secret-scan, test 0 failures, credo held,
dialyzer held), AC7 (two lanes per round, on checkouts of the index). AC5 is verified
after this lands, by dispatching CI on this branch and comparing to the pre-fix dispatch
run 34071415451 on the same branch.

Ten review rounds, twenty independent lanes, every one on a checkout of the index. The
gate has been byte-identical since round 4 and no lane in any round could make it report
a false green. What kept failing was the pin and the prose, and it was one mistake at
eight levels: an assertion about something the assertion does not read. Reviewers defeated
the pin with a parked early return, a second guard on a different payload key, a computed
sentinel, an interpreter prefix, an extra space, a glob that contains no `gate.sh` token,
and a carriage return that makes a dropped comment execute. Each is now a row that dies.

The record was cut on the owner's decision of 2026-09-07: no transcribed count survives in
a tracked file -- every number is either the command that derives it or it is absent --
because five consecutive rounds had blocked on a stale one, twice by the same mechanism.

Evidence: internal/slices/16k-baseline-trigger-guard/FINDINGS.md (F1-F17), REVIEW.md, and
logs/round{1..10}.{scope,r1.verdict,r2.verdict}.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VNdY6wVS773UJgrHUeBN3Y

Reviewed-diff: sha256:ee52b4f3a64c7eee5955fcd22667b99a1e0d12efbf565e41c3ebd289219f060f
@HackTuah
HackTuah merged commit 602383a into main Sep 7, 2026
30 of 33 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant