diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml index 80a05a7..26aead3 100644 --- a/.github/workflows/ci.yml +++ b/.github/workflows/ci.yml @@ -17,8 +17,30 @@ on: pull_request: schedule: # For the advisory job: new CVEs land against unchanged dependencies, so the audits - # need a clock. NOT for the baseline gate -- that compares git-tracked values and a - # scheduled run would re-compare a commit its own push run already checked. + # need a clock. + # + # This used to add "NOT for the baseline gate -- that compares git-tracked values and a + # scheduled run would re-compare a commit its own push run already checked" (quoted in + # full: an earlier draft of this comment elided the middle clause, which is the half + # that is still true). Slice 16k falsified the rest in both halves, and + # review caught the stale sentence in the same diff that falsified it. First: the + # baseline job carries no `if:` excluding `schedule`, so it has always RUN on the + # nightly -- it simply measured nothing, which is the defect 16k fixes. Second, now + # that it measures, the comparison is not always a re-comparison: it compares + # `github.sha~1`, and after a rebase-merge of a multi-commit branch that is the + # branch's second-to-last commit, not the previous `main` tip, so the nightly's + # comparison is NARROWER than the push run's rather than a repeat of it. + # + # A consequence, derived and not yet observed here (this history is fully linear -- + # `git rev-list --min-parents=2 --count --all` is 0): if a branch lowers a baseline in one + # commit and restores it in the next, the push run compares old-tip -> tip and sees no + # change, while the nightly compares those two commits and sees a raise. That would go + # red on `main`. It blocks no merge -- required checks are evaluated on a PR's own head. + # It is also self-clearing, which an earlier draft of this comment denied when it called + # it "noise nobody could clear": the next push to `main` moves `github.sha`, so the next + # nightly compares a different pair, and a `_corrections` entry clears it deliberately. + # Recorded here so the next person to see it has the mechanism rather than a mystery. + # # Note: scheduled runs only fire from the default branch. - cron: "17 6 * * *" workflow_dispatch: @@ -192,6 +214,22 @@ jobs: "+refs/heads/${{ github.base_ref }}:refs/remotes/origin/${{ github.base_ref }}" \ || { echo "::error::cannot fetch base ref; refusing to pass unmeasured"; exit 1; } ./tools/gate.sh baseline "origin/${{ github.base_ref }}" + elif [ "${{ github.event.before }}" = "0000000000000000000000000000000000000000" ] || [ -z "${{ github.event.before }}" ]; then + # Same guard the attestation job carries above, reused rather than re-derived. + # `github.event.before` is ABSENT from the schedule and workflow_dispatch + # payloads, so this expanded to `baseline ""` and gate.sh matched it in the same + # branch as the all-zeros sentinel: "no previous ref (new branch); nothing to + # compare", return 0. Measured on run 34028977996 (schedule, main, 778accb8) -- + # the gate reported success having compared zero keys, on a commit whose parent + # 268a83e8 carries the baseline file it declined to read. Nightly runs had been + # green on nothing since the schedule trigger was added. + # + # sha~1, not "nothing": on a multi-commit push this sees only the last commit, + # which is the HEAD~1 hole the else branch describes -- but a partial comparison + # is strictly more than none, and this branch is reached only when the payload + # gave us no better ref. gate.sh now REFUSES an empty ref outright, so this + # cannot silently regress to a pass. + ./tools/gate.sh baseline "${{ github.sha }}~1" else # github.event.before, NOT HEAD~1. One run is created per push, at the tip, so # HEAD~1 compares only the last commit -- a raise in an earlier commit of the diff --git a/BACKLOG.md b/BACKLOG.md index b6fb7d4..4732282 100644 --- a/BACKLOG.md +++ b/BACKLOG.md @@ -292,3 +292,153 @@ The MCP surface changed in slice 16j. Changes are pinned by tests in `ping` carrying a `_meta` revision is a known `beam_mcp` defect, tracked as SCR-257, and is pinned by no test here because a test would pin the defect. + +## 13. The baseline gate still goes red on a force-pushed ref — SCR-69, second half + +Slice 16k closed one half of SCR-69: `Gate - baseline may only decrease` used to go **green +having compared nothing** whenever the event payload carried no `before`, which is every +`schedule` and every `workflow_dispatch` run. Measured on runs `34028977996` and +`34071415451`, both green on zero comparisons. `.github/workflows/ci.yml` now carries the +same guard the attestation job has, and `baseline_gate` refuses an empty ref outright. + +**The other half is open.** A force-push sets `github.event.before` to a well-formed SHA that +is reachable from no ref. That is not the all-zeros sentinel, so it reaches +`tools/gate.sh`'s `git cat-file -e "$ref:$BASELINE"`, which fails, and the gate goes **red on +a tree that is fine**. + +Measured: run `33709449011` (`push`, head `27de8a1b`) failed with +`FAIL -- .claude/gate-baseline.json absent at d025d350…`. The tree was fine, and the run that +proves it is **`33709453098`** — the `pull_request` run on the *same head*, created four +seconds later (`02:55:25Z` and `02:55:29Z` per `gh api .../actions/runs/ --jq .created_at`), +which took the `base_ref` branch and compared both keys for real: + +``` +$ gh api repos/ScriptKittyOS/Ultraviolet/actions/jobs/100505623057/logs \ + | sed 's/\x1b\[[0-9;]*m//g' +[...] ./tools/gate.sh baseline "origin/main" +[...] baseline credo_issues: 76 -> 76 +[...] baseline dialyzer_warnings: 43 -> 43 +``` + +`[...]` marks an elided ISO-8601 timestamp on each line, and the `sed` strips the ANSI colour +codes the API returns. The first line quoted is part of the runner's echo of the `run:` script, +not a trace; the untaken branch echoed beside it, and its invocation, are not reproduced. + +No branch tally appears here. Consecutive review rounds each corrected the tally in this +sentence and each correction was wrong again — once by counting the current file against a log +from the older one. Read the log for what that run's script was, and +`sed -n '/^ baseline:/,/^ advisories:/p' .github/workflows/ci.yml | grep -cE '^\s*(if|elif|else)\b'` +for what it is now — scoped to the job, because unscoped over the whole file it counts every +job's branches. Nothing else was +added to the block -- an earlier draft carried an editorial comment inside it, which is the +same defect as an unmarked elision in the other direction. Marked rather than silently trimmed: a block under a +bare `$` prompt claims to be what the command prints. + +**Not** run `33877201424`, which an earlier draft of this entry cited as the green control. +Review measured that its `before` was the all-zeros sentinel, so it compared **zero keys** — +it is a second sighting of the bug 16k closed, not evidence about the tree. It would also no +longer reproduce: `ci.yml`'s new `elif` now routes all-zeros to `${{ github.sha }}~1`, so that +same push performs a real comparison. Recorded because citing it would have sent whoever picks +this up chasing a green that the current code cannot produce. + +`fetch-depth: 0` does not help; a full-depth fetch fetches refs, and an orphaned commit is +reachable from none. + +This is the opposite failure direction from the one 16k fixed, and it wants a different +remedy: a reachability probe before the ref is used, and a decision about what to compare +against when the recorded start point no longer exists. Deliberately left out of 16k rather +than folded in. + +## 14. Documentation drift — a class, to be derived and swept as one unit + +Tracked documentation carries statements that were true when written and are not true now. +This is recorded as a **class with a derivation**, not as a list of instances: a list gets +fixed and the class stays open. Each entry names the **derivation** that finds the whole +population, so the sweep can prove it is complete rather than assert it. Only the first is a +runnable one-liner; the rest are procedures, and saying so is the difference between a +derivation and a command that does not exist. Review measured this distinction: the headline +here first claimed every entry "names the command", and most do not. + +| class | derivation | known instances | +|---|---|---| +| a `mix` task cited in a tracked `.md` that does not exist | every `mix ` in `git grep -ho 'mix [a-z][a-z0-9_]*\(\.[a-z][a-z0-9_]*\)*' -- '*.md'`, checked against `mix help` | `mix phx.server` at `docs/operator_boot_runbook.md:9` — there is no Phoenix application in this umbrella | +| an umbrella app named in a doc that is not in `apps/` | every `hacktui[_a-z]*` cited as an app, checked against `ls apps/` | `ARCHITECTURE.md`'s "Umbrella apps" list has one more entry than `apps/` has directories — `sed -n '/^## Umbrella apps/,/^## Current/p' ARCHITECTURE.md \| grep -c '^- '` against `ls apps/ \| wc -l`, both run and checked, not written from memory. The extra one is `hacktui` itself, which is the umbrella **root** (`mix.exs`, `apps_path: "apps"`), not an app — so the sweep must decide whether to drop it or relabel it, and this cell deliberately does not pre-judge that. One reviewer read it as a real instance and one as a false positive; the disagreement is the finding | +| a doc quoting a command whose real output contradicts the doc's stated expectation | run each fenced read-only command and compare | `README.md:890` — `git ls-files \| grep env` is the **unanchored** pattern `CLAUDE.md` §10 records as always matching `envelope.ex`; it returns `.env.example` and `envelope.ex` on this tree, under `README.md:895`'s stated expectation of "no private `.env` files". The anchored form in `tools/gate.sh` returns 0 hits on the same tree. `:891`'s `grep key` returns nothing and is **not** an instance — the first draft of this cell cited the pair | +| a doc asserting a `main` SHA or a gate state that has moved | every `[0-9a-f]{7,40}` near `main`, and every gate-status table | `HANDOFF.md` §1's "`main` is at `5a6e566`" (now `86681a2`), its `test` row and its `248 tests` — cells carrying a slice-15 figure, each marked in place by slice 16k and none corrected. Cited by content, not by line — this table's own subject is citations that go stale, and two earlier drafts of this cell proved it, one citing `HANDOFF.md:17` after the same commit moved it to `:20`, the other quoting a string the same commit had deleted | + +Slice 16k marked `HANDOFF.md` §1 **in place**, so a reader of §1 is warned at the point of +the falsehood rather than hundreds of lines later. The row above says which cells are only +flagged and which was corrected; this sentence repeats neither the list nor the count, because +review blocked here on exactly that repetition — a tally that disagreed with the +row, and then a summary that contradicted it. It is the class this entry is about, occurring +inside the entry about it. + +## 15. What the baseline-gate pin covers, and what it does not + +Slice 16k pins the `baseline` job by asserting its **entire body** — every non-comment, +non-blank line under `jobs:` → `baseline:`, whitespace collapsed — against a literal list in +`apps/hacktui_core/test/ci_baseline_guard_test.exs`, plus the requirement that ` baseline:` +occur exactly once under `jobs:`. + +**It arrived at that shape the hard way.** Rounds 2 through 5 each pinned a *fragment* of the +job — the presence of one line, then the absence of two values, then a list of permitted +invocations, then lines containing `gate.sh` — and a reviewer walked past each one with a +working survivor: valid YAML, the suite green, and this required status check **green having +compared zero baselines**. The last of those was `bash tools/gate?sh baseline "$(printf …)"`, +which runs the gate and contains no `gate.sh` token at all. There is no token a respelling +must contain, because a shell word is resolved at run time and matched here at read time. Any +pin that first *recognises* a subset of the job and then rules on the subset can be spelled +around; a pin that asserts the whole body cannot, because every survivor adds or changes a +line. The mutants in `tools/mutants/c5.tsv` whose names begin `ci_baseline_` or +`gate_baseline_` are that history, and all die. **Which of them are reviewers' own survivors +is recorded in `HANDOFF.md` §10 by naming them.** Review blocked on a count here more than +once: each time a figure correct for one set was restated over a differently-drawn subset and +was off by one. Naming one population in one place removes the mechanism rather than the +instance. + +**Covered — verified by mutant, each one KILLED:** + +| lever | mutant | +|---|---| +| a second guard routing a trigger to the sentinel | `ci_baseline_second_guard` | +| the sentinel computed rather than written | `ci_baseline_computed_sentinel` | +| the command respelled (`bash tools/…`, extra whitespace, a glob) | `ci_baseline_bash_prefixed`, `ci_baseline_extra_space`, `ci_baseline_glob_spelling` | +| `continue-on-error: true` on the job | `ci_baseline_continue_on_error` | +| a job-level `if:` excluding a trigger | `ci_baseline_job_if_skip` | +| a trailing command swallowing the gate's exit code | `ci_baseline_trailing_true` | +| the job's `name:` changed — which changes its **check context**, and a required context that never reports blocks every pull request forever (`CLAUDE.md` §4c) | `ci_baseline_job_renamed` | +| the job key removed, or a second job of the same key added | `ci_baseline_duplicate_job_key` | + +The `name:` row is worth reading twice. An earlier draft of this entry said a rename was +covered by the uniqueness assertion on the job **key**. Review measured that the key is not +the check context — `secret-scan` is the key, `Gate - tracked secret-shaped files` is the +context — so the edit that was covered could not break a required check, and the edit that +could break one passed green. The whole-body assertion covers both. + +**NOT covered — everything outside the job's own body:** + +- the `on:` block. Deleting `schedule:` or `workflow_dispatch:` means the gate never runs on + that trigger at all. That is a different failure from reporting an unmeasured pass, but it + is a way to make this gate stop saying anything. +- workflow-level `env:` (`LOGDIR`), the `concurrency:` group, and the shared + `.github/actions/beam-setup` composite action. +- the branch-protection ruleset itself, which decides whether this context is required at + all. Owner-only (`CLAUDE.md` §4c). +- every other `Gate -` job, none of which has an equivalent pin — and workflow-level + `defaults:`, which this test does not read. What a `run.shell` override does to a job at + run time is not measured here and is not claimed. +- **a second workflow file.** The test reads `.github/workflows/ci.yml` and nothing asserts + that it is the only workflow (`ls .github/workflows/`). Whether a job of the same `name:` in + another file would report under the same required context is not measured here and is not + claimed; what is measured is that this pin would not see it. +- `tools/gate.sh`'s interior beyond the paths the tests exercise. The positive control (a real + ref must produce a comparison) and the refusal tests cover the entry paths; the + comparison loop itself is covered by `gate_baseline_*` mutants, not exhaustively. + +This list is what has been **found**, not a proof of completeness. Successive review rounds +each found one more, and the honest summary is that it is a list of known levers, not a closed +set. + +Extending the same whole-body assertion to the other `Gate -` jobs is the obvious next step +and is deliberately not slice 16k's work: it is one literal per job to maintain, and it should +be decided as one thing rather than smuggled in beside a fix to one of them. diff --git a/HANDOFF.md b/HANDOFF.md index c0fa9bc..e82685d 100644 --- a/HANDOFF.md +++ b/HANDOFF.md @@ -14,8 +14,14 @@ named so you can re-run it rather than trust it. ## 1. Where things stand + + `main` is at **`5a6e566`** (slice 15, the commit that added this file). The suite is -**248 tests, 0 failures** — green for the first time. Re-derived at slice 16 rather than +**248 tests, 0 failures** — green for the first time. Re-derived at slice 16 rather than carried forward: `mix test` summed across the umbrella apps gives 248/0, and it reconciles with CI's advisory integration job (266 tests when `--include integration` adds the 18 integration-tagged ones; 266 − 18 = 248). @@ -25,11 +31,11 @@ integration-tagged ones; 266 − 18 = 248). | compile `--warnings-as-errors` | — | pass, hard-blocking | | format `--check-formatted` | — | pass, hard-blocking | | tracked secret-shaped files | — | pass, hard-blocking | -| test | **0** | pass — still a **ratchet**, not hard-blocking; retirement is slice 16's next commit | +| test | **0** | **STALE — see §10.** This row predates slice 16: `tools/gate.sh` scores `test` with `hard_zero`, and the baseline entry is retired. Not corrected here; BACKLOG.md §14 | | credo `--strict` | **76** | held | | dialyzer | **43** | held | | deps.audit / hex.audit | 12 / 20 | advisory, not blocking (**slice 18**'s work) | -| mutation (`tools/mutate.sh`) | 0 survivors | 14/14 killed, CANARY aborts | +| mutation (`tools/mutate.sh`) | 0 survivors | **STALE — see §10.** This figure is from slice 15; `./tools/mutate.sh tools/mutants/c5.tsv \| tail -2` gives the current one. CANARY aborts | | attestation | — | **GREEN on `main`** — run 33594696902; see §2 | Slices 01 through 13b, and 15, are merged. **14 is not** — it has a PLAN and CRITERIA and @@ -306,3 +312,82 @@ make no SPDX/REUSE claim. Exact commands and their output go in the signoff. - **Reviewers get `CRITERIA.md` before they are spawned**, and must classify every finding as blocking or residual. A round finding only residual items is PASS-with-residuals. Three-round cap unless a blocking class is hit. An unclassified finding is not actionable. + +--- + +## 10. Slice 16k — the baseline gate no longer passes unmeasured (SCR-69, first half) + +`Gate - baseline may only decrease` is a required status check. `github.event.before` is +**absent from the `schedule` and `workflow_dispatch` event payloads**, so +`.github/workflows/ci.yml` expanded to `./tools/gate.sh baseline ""` and `baseline_gate` +matched the empty string in the same branch as the all-zeros sentinel: it printed +`no previous ref (new branch); nothing to compare` and returned **0**. Every nightly run in +this repository's history was green having compared zero baselines. + +Measured before any fix, from the job logs (`gh api repos/ScriptKittyOS/Ultraviolet/actions/jobs//logs`): + +| run | event | head | job | result | +|---|---|---|---|---| +| `34028977996` | `schedule` | `778accb8` | `101474937084` | success, 0 comparisons | +| `34071415451` | `workflow_dispatch` | `86681a2` | `101589359652` | success, 0 comparisons | + +A comparison was available in both cases — each head's parent carries +`.claude/gate-baseline.json`. + +**The fix reuses a guard that already existed.** `Gate - attestation` handles the identical +payload at the `elif` in its own `run:` block, and in run `34028977996` (job `101474937083`) +it measured while the baseline job did not. The same `elif` now sits in the `baseline` job, +routing an empty-or-sentinel `before` to `${{ github.sha }}~1`; `baseline_gate` **refuses an +empty ref outright**. The all-zeros sentinel keeps its pass as its own labelled case and is +now unreachable from CI. + +**Where the guarantee actually lives.** `apps/hacktui_core/test/ci_baseline_guard_test.exs` +is an ordinary hard-blocking test — **no new `Gate -` job**, so no ruleset change (§4c), the +same reasoning slice 16c used for the schema digest. Its tests that call `run_gate/1` **run the gate** +rather than reading it, and those are the ones that cannot be satisfied by a file that merely +looks right. (Drafts of this sentence carried a count of them and each was wrong, because +every disposition that added one left the number behind. The property is stated and no count +of them appears here.) Round 1 shipped only the reading +assertions and both reviewers broke them independently: parking +`[ -z "$ref" ] && { note ...; return 0; }` one line above the inspected branch reinstates the +pre-16k defect exactly while leaving that branch byte-identical, and all text assertions +stayed green. That mutation is now row `gate_baseline_early_bypass` in +`tools/mutants/c5.tsv`, and it dies. + +**Numbers this slice moves** (the §1 table above predates it and is not updated here — +see "still stale" below): + +- mutation harness: **no survivors.** The count of rows, the split between reviewer survivors + and derived rows, and the totals before and after are **not transcribed here** — they go + stale the moment a row is added, and review kept blocking on exactly that. Derive them: + + ``` + ./tools/mutate.sh tools/mutants/c5.tsv | tail -2 + git diff -- tools/mutants/c5.tsv | grep '^+[a-z]' | cut -f1 # the rows this slice adds + ``` + + **Which rows are reviewer survivors** — a lane demonstrated each defeating the pin, and the + row is the receipt: `gate_baseline_early_bypass`, `ci_baseline_second_guard`, + `ci_baseline_computed_sentinel`, `ci_baseline_bash_prefixed`, `ci_baseline_extra_space`, + `ci_baseline_glob_spelling`, `ci_baseline_cr_smuggled_line`, + `gate_baseline_post_sentinel_bypass`. + + **Which rows this slice adds are derived**: `ci_baseline_guard_route`, `ci_baseline_guard_condition`, + `gate_baseline_empty_ref_passes`, `ci_baseline_duplicate_job_key`, + `ci_baseline_trailing_true`, `ci_baseline_continue_on_error`, `ci_baseline_job_if_skip`, + `ci_baseline_job_renamed`. Both lists are of the rows THIS SLICE ADDS; the rows that predate + it are neither. Naming them is durable; counting them was not, and saying "the remainder" + was not either — over the whole file it mislabels every row the slice did not write. + +**Still open, deliberately.** SCR-69's other half: a force-push sets `github.event.before` to +a well-formed but **unreachable** SHA, which is not the sentinel, so `git cat-file -e` fails +and the gate goes **red on a tree that is fine**. Opposite failure direction, different +remedy, out of 16k's scope. Recorded in `BACKLOG.md` §13 with the corrected evidence. + +**Still stale, and NOT fixed by this slice.** §1 above says `main` is at `5a6e566`; it is at +`86681a2`. §1's gate table described `test` as a ratchet awaiting retirement, which slice 16 +completed, and its suite and mutation numbers are likewise from slice 15. Each such cell now +carries an inline STALE marker, so a reader of §1 is warned at the point of the falsehood. +These are instances of a documentation-drift class being derived and swept as its +own unit of work; fixing them here would have been silent scope expansion (§9). +Do not read §1's numbers as current. diff --git a/apps/hacktui_core/test/ci_baseline_guard_test.exs b/apps/hacktui_core/test/ci_baseline_guard_test.exs new file mode 100644 index 0000000..b9f2c95 --- /dev/null +++ b/apps/hacktui_core/test/ci_baseline_guard_test.exs @@ -0,0 +1,434 @@ +defmodule HacktuiCore.CiBaselineGuardTest do + use ExUnit.Case, async: true + + # `Gate - baseline may only decrease` is a required status check. It is handed a git ref and + # compares the baselines at that ref against the working tree's. + # + # `github.event.before` is ABSENT from the `schedule` and `workflow_dispatch` event payloads. + # `${{ github.event.before }}` then expands to the empty string, and until slice 16k + # `baseline_gate` matched the empty string in the same branch as the all-zeros sentinel: + # it printed "no previous ref (new branch); nothing to compare" and returned 0. + # + # Measured, twice, before the fix: + # + # run 34028977996 schedule main 778accb8 job SUCCESS + # run 34071415451 workflow_dispatch slice/16k-baseline-trigger-guard 86681a2 job SUCCESS + # + # Both job logs contain `./tools/gate.sh baseline ""` followed by the "nothing to compare" + # line. Both commits' parents carry `.claude/gate-baseline.json`, so a comparison was + # available in each case and was not made. Nightly runs had been green on nothing. + # + # The correct guard already existed in `Gate - attestation`, above this one. This test holds + # the two together so the baseline job cannot lose it again -- and so nobody re-derives a + # second, differently-shaped guard for the same payload gap. + # + # It is an ordinary hard-blocking test, deliberately NOT a new `Gate -` job: a new required + # context needs a ruleset edit only the owner can make (CLAUDE.md 4c), and a required context + # that never reports blocks every pull request forever. Same reasoning slice 16c used for the + # schema digest. + + @root Path.expand("../../..", __DIR__) + @ci Path.join(@root, ".github/workflows/ci.yml") + @gate Path.join(@root, "tools/gate.sh") + + @zeros "0000000000000000000000000000000000000000" + + # Lines of one top-level job, from ` :` to the next key at the same two-space indent. + # Line-based rather than YAML-parsed because the thing under test is the literal shell text + # the runner executes, not a parsed structure: a YAML load would normalise the very bytes + # the comparison is about. + # + # An earlier version of this comment justified it as "the repository has no YAML + # dependency". That was false and review measured it -- `yaml_elixir 2.12.2` is in + # `mix.lock` and loads in `:test`, and it parses this exact file. The reason above stands + # on its own; the false one is deleted rather than left as the next reader's footgun. + # Scoped to the `jobs:` mapping, and the key must occur EXACTLY ONCE inside it. + # + # Round 4 measured why both. `Enum.find_index` over the whole file takes the first + # two-space-indented `baseline:` anywhere, and `env:` sits above `jobs:` -- so a + # workflow-level env var named `baseline`, holding a multi-line plain scalar with the three + # expected invocations in it, was found first. Every assertion in this file then passed + # against a region that is not the job, while the real job routed the nightly to the + # sentinel. Valid YAML, the suite green, the required check green on zero baselines. + # + # A pin that can be aimed at a decoy is not a pin. Scoping fixes the aim; the uniqueness + # assertion fixes the aiming. The uniqueness assertion also catches a job RENAME, which is + # worth having for its own sake: renaming a job renames its check context, and a required + # context that never reports blocks every pull request forever (CLAUDE.md 4c). + defp job(src, name) do + # Split the way YAML does, not the way Elixir defaults to. YAML 1.2 §5.4 breaks lines on + # LF, CR and CRLF; `String.split(src, "\n")` breaks only on LF. Round 6 measured what that + # costs (pinned by `ci_baseline_cr_smuggled_line`, whose TSV row carries a real carriage + # return -- a bare CR is not a terminator to the harness's reader, only CRLF is): + # `# harmless noteexit 0`, written as one LF-line, is ONE line here (dropped as + # a comment) and TWO lines to a parser, the second an `exit 0`. Two independent parsers -- + # PyYAML and the `yaml_elixir` this repo ships -- agreed, and the whole suite stayed green + # while the step exited 0 without ever invoking the gate. + # + # With this split that smuggled line is its own entry, is not in the allowlist, and fails + # with the text quoted in the message. The fix is here rather than in a "reject CR" guard + # because the defect was that this function's idea of a line disagreed with the runner's. + all = String.split(src, ~r/\r\n|\r|\n/) + + jobs_at = + Enum.find_index(all, &(&1 == "jobs:")) || + flunk("no top-level `jobs:` in .github/workflows/ci.yml") + + lines = Enum.drop(all, jobs_at + 1) + occurrences = Enum.count(lines, &(&1 == " #{name}:")) + + assert occurrences == 1, + """ + ` #{name}:` occurs #{occurrences} times under `jobs:` in .github/workflows/ci.yml; + exactly one is required. Zero means the job was renamed or removed -- which also + renames its check context, and a required context that never reports blocks every + pull request forever (CLAUDE.md 4c). More than one means this test cannot know + which region it is reading, and a test that reads an ambiguous region is not + evidence about either. + """ + + start = Enum.find_index(lines, &(&1 == " #{name}:")) + rest = Enum.drop(lines, start + 1) + + len = + Enum.find_index(rest, &Regex.match?(~r/^ [A-Za-z_][A-Za-z0-9_-]*:/, &1)) || + length(rest) + + Enum.take(rest, len) + end + + # Whitespace-insensitive comparison of the guard's shell text. A reformat is allowed; a + # changed condition is not. Comment lines are dropped so the two jobs' prose may differ. + defp guard_tokens(job_lines) do + job_lines + |> Enum.reject(&Regex.match?(~r/^\s*#/, &1)) + |> Enum.filter( + &(String.contains?(&1, "github.event.before") and + String.starts_with?(String.trim_leading(&1), "elif")) + ) + |> Enum.map(&(&1 |> String.split(~r/\s+/, trim: true) |> Enum.join(" "))) + end + + test "the baseline job guards the empty/sentinel payload exactly as the attestation job does" do + src = File.read!(@ci) + + attestation = guard_tokens(job(src, "attestation")) + baseline = guard_tokens(job(src, "baseline")) + + assert length(attestation) == 1, + "expected exactly one `elif` guard on github.event.before in the attestation job, " <> + "found #{length(attestation)}: #{inspect(attestation)}" + + assert length(baseline) == 1, + "the baseline job has #{length(baseline)} `elif` guards on github.event.before; " <> + "expected exactly one. Without it, `${{ github.event.before }}` expands to the " <> + "empty string on schedule and workflow_dispatch and the gate reports a pass it " <> + "never measured (runs 34028977996 and 34071415451)." + + # The condition, not the command: the two jobs pass different arguments to gate.sh. + strip_body = fn [line] -> line |> String.replace(~r/;\s*then\s*$/, "") end + + assert strip_body.(baseline) == strip_body.(attestation), + """ + the baseline and attestation guards have diverged. + + attestation: #{strip_body.(attestation)} + baseline : #{strip_body.(baseline)} + + One payload gap, one guard shape. If the condition genuinely needs to differ, + that is a slice, not an edit. + """ + end + + test "the baseline job is exactly the job this slice reviewed" do + src = File.read!(@ci) + + # THE WHOLE JOB, not its invocations. Round after round of review got here, and the route + # matters more than the destination, so it is recorded. + # + # Rounds 2 to 5 each pinned a FRAGMENT of this job and each was walked past by a reviewer + # who produced a working survivor -- valid YAML, the suite green, and + # `Gate - baseline may only decrease` GREEN having compared zero baselines: + # + # r2 pinned "the sha~1 line is present" -> a second guard made it unreachable + # r3 pinned "these two values are absent" -> `$(printf '0%.0s' $(seq 40))` + # r4 pinned "invocations are these three" -> `bash tools/...`; one extra space + # r5 pinned "lines containing gate.sh" -> `tools/gate.s?`, which is the same file + # + # Each fix closed one spelling and the next round found another, because **every one of + # them recognised a subset of the job and then made a rule about the subset.** A shell + # word is resolved at run time and matched here at read time, so the two can always be + # separated: `gate.s?`, `gate.$(printf sh)`, `$G`, a symlink. There is no token a + # respelling must contain, and the round-4 comment claiming there was one was false. + # + # So stop recognising. This asserts the job's ENTIRE body -- comment lines and blanks + # dropped, whitespace collapsed -- against a literal, and separately asserts that a + # dropped line cannot execute (see the line-break guard below, which round 6 added after + # a lone CR turned a dropped comment into an `exit 0`). There is then no subset to slip + # through: a second guard, a respelled invocation, `continue-on-error: true`, a job-level + # `if:`, a trailing `true` that swallows the exit code, a renamed `name:` that silently + # breaks a required status check -- each adds or changes a line that survives the drop, + # and any such line not in the list fails. + # + # The cost is real and is the point. Every edit to this job fails this test until the list + # is updated in the same change. This job is a REQUIRED STATUS CHECK; it should not be + # possible to change it quietly, and round after round of evidence says that when it is + # possible, it happens. + # + # `expected` was generated from the file rather than retyped, then checked by making the + # test fail once and reading what it printed. + expected = [ + ~s(name: Gate - baseline may only decrease), + ~s(runs-on: ubuntu-latest), + ~s(timeout-minutes: 10), + ~s(steps:), + ~s(- uses: actions/checkout@v4), + ~s(with:), + ~s(fetch-depth: 0), + ~s(- name: Compare against the ref this push started from), + ~s(run: |), + ~s(set +e), + ~s(if [ -n "${{ github.base_ref }}" ]; then), + ~s(git fetch --depth=1 origin \\), + ~s("+refs/heads/${{ github.base_ref }}:refs/remotes/origin/${{ github.base_ref }}" \\), + ~s(|| { echo "::error::cannot fetch base ref; refusing to pass unmeasured"; exit 1; }), + ~s(./tools/gate.sh baseline "origin/${{ github.base_ref }}"), + ~s(elif [ "${{ github.event.before }}" = "0000000000000000000000000000000000000000" ] || [ -z "${{ github.event.before }}" ]; then), + ~s(./tools/gate.sh baseline "${{ github.sha }}~1"), + ~s(else), + ~s(./tools/gate.sh baseline "${{ github.event.before }}"), + ~s(fi) + ] + + raw = job(src, "baseline") + + # DROPPING A LINE IS ONLY SAFE IF A DROPPED LINE CANNOT EXECUTE, and until round 6 that + # was assumed rather than asserted. The assumption is false as written: this test splits + # on "\n", but YAML 1.2 §5.4 also breaks lines on CR and CRLF. One LF-line beginning with + # `#` and containing a lone CR is ONE dropped comment here and TWO content lines to a + # parser -- so `# noteexit 0` was dropped by the pin and rendered as an `exit 0` in + # the step's script. Measured by review against two independent parsers (PyYAML and the + # `yaml_elixir` this repo already ships), with the suite green and the step exiting 0. + # + # So the drop is now guarded rather than trusted: the body must contain no character a + # YAML parser treats as a line break other than the LF this test split on. With that + # asserted, a line this test calls a comment is a comment to the runner too. + # CR and CRLF are handled by the split in `job/2`. These three are YAML 1.1 line breaks + # the split deliberately does not treat as such; a parser honouring them could still turn + # one line here into two. Refused outright rather than split on, because no legitimate + # line of this workflow contains them. + breaks = %{ + "\u0085" => "NEL (U+0085)", + "\u2028" => "LS (U+2028)", + "\u2029" => "PS (U+2029)" + } + + for {char, label} <- breaks do + offenders = Enum.filter(raw, &String.contains?(&1, char)) + + assert offenders == [], + """ + the `baseline` job contains #{label}, which YAML treats as a line break and this + test does not: + + #{Enum.map_join(offenders, "\n", &(" " <> inspect(&1)))} + + A line this test drops as a comment can therefore be two lines to the runner, the + second of them executable. That is how a `# note` followed by a carriage return + and `exit 0` passed every assertion in this file while making the step exit 0 -- + measured in round 6. CR is handled by splitting on it in `job/2`; #{label} is + refused here because nothing in this workflow legitimately contains it. + """ + end + + body = + raw + |> Enum.reject(&(Regex.match?(~r/^\s*#/, &1) or String.trim(&1) == "")) + |> Enum.map(&(&1 |> String.split(~r/\s+/, trim: true) |> Enum.join(" "))) + + assert body == expected, + """ + the `baseline` job is not the job this slice reviewed. + + expected #{length(expected)} lines, found #{length(body)}. First difference: + + #{first_difference(expected, body)} + + This job is a required status check whose whole subject is refusing to report a + pass it has not measured. Any line here that is not in the list above is + unreviewed, and review found repeatedly that an unreviewed line in this job is how + it goes green on zero baselines. + + If the change is legitimate, update the list in the same commit and say in the + slice record what the job now compares. Comment lines and blank lines are ignored + and whitespace is collapsed, so reformatting and commenting are free. + """ + end + + defp first_difference(expected, body) do + idx = + Enum.find(0..max(length(expected), length(body)), fn i -> + Enum.at(expected, i) != Enum.at(body, i) + end) + + case idx do + nil -> + "(no line differs; the lists differ only in length)" + + i -> + " line #{i + 1}\n expected: #{inspect(Enum.at(expected, i))}\n found: #{inspect(Enum.at(body, i))}" + end + end + + test "baseline_gate refuses an empty ref instead of passing on it" do + src = File.read!(@gate) + + [branch] = + Regex.run( + ~r/^\s*if \[ -z "\$ref" \][^\n]*\n(.*?)^\s*fi$/ms, + src, + capture: :all_but_first + ) || flunk("no `if [ -z \"$ref\" ]` branch in tools/gate.sh") + + # The capture is non-greedy to the first `^ fi$`. If the branch is ever rewritten as a + # one-liner, `[^\n]*\n` swallows its `fi` and the capture runs on into the NEXT branch -- + # measured by review, which watched this assertion fail while printing the all-zeros + # sentinel's body under a heading about the empty-ref branch. The verdict was right and + # the evidence was of the wrong lines. Detect the slide instead of mis-reporting it. + refute String.contains?(branch, "if ["), + """ + the empty-ref branch capture slid past its own `fi` and swallowed a later branch: + + #{branch} + This is a defect in THIS TEST's regex, not necessarily in tools/gate.sh. Re-anchor + the capture before drawing any conclusion about the gate. + """ + + assert String.contains?(branch, "return 1"), + """ + tools/gate.sh's empty-ref branch does not return 1. Its body is: + + #{branch} + An empty string is not a git ref -- it only ever arrives from an event payload + field that was not populated. A gate handed one has measured nothing. + """ + + refute String.contains?(branch, "return 0"), + "tools/gate.sh's empty-ref branch still has a `return 0` path" + + # The all-zeros SENTINEL is a different value with a different meaning -- a push that + # creates a branch -- and legitimately still passes. Pinned so the fix above is not + # "simplified" by folding the two back together. + assert Regex.match?(~r/\[ "\$ref" = "#{@zeros}" \]/, src), + "tools/gate.sh no longer handles the all-zeros sentinel as its own case" + end + + # Runs the gate. The assertions above read BYTES; the tests that call `run_gate/1` read + # BEHAVIOUR, and those are the ones that cannot be satisfied by a file that merely looks + # right. + # + # No count of them appears here, deliberately. Three drafts of this line carried one -- "the + # only one of the four", then a tally of tests, then another -- and each was wrong, because + # every disposition that added a behavioural test left the number behind. A comment that + # miscounts the file it sits in is this slice's own subject; the cure is to state the + # property and let `grep -c '^ test ' apps/hacktui_core/test/ci_baseline_guard_test.exs` + # do the counting. No tally of this file appears in this file + # + # Both reviewers of round 1 broke the byte-only version, independently and by different + # routes. Parking `[ -z "$ref" ] && { note ...; return 0; }` one line ABOVE the inspected + # branch reinstates the pre-16k defect exactly -- measured, `rc=0` and "no previous ref + # (new branch); nothing to compare" -- while leaving the inspected branch byte-identical, so + # the text assertions stayed green. Returning 0 as `rc=0; return "$rc"` defeats the + # `refute contains "return 0"` the same way. An unconditional bypass anywhere earlier in the + # function is invisible to a regex scoped to one branch. + # + # That is CLAUDE.md 4b's own class -- "a gate reporting a verdict without running its + # probe" -- committed inside the test written to prevent it. It is the reason this test + # shells out rather than reads. + # + # `System.cmd/3` needs an ABSOLUTE path: a relative "./tools/gate.sh" with `cd:` raises + # :enoent. LOGDIR is passed so gate.sh does not mktemp -d a directory nothing removes. + # + # Deliberately NOT asserted here: a real ref such as HEAD~1. `actions/checkout@v4` defaults + # to fetch-depth 1 and the `Gate - test ratchet` job does not override it, so HEAD~1 does + # not exist in the job this test runs in. Both cases below are answered before git is + # consulted at all. + defp run_gate(args) do + tmp = Path.join(System.tmp_dir!(), "hacktui-16k-#{System.unique_integer([:positive])}") + File.mkdir_p!(tmp) + + try do + System.cmd(Path.join(@root, "tools/gate.sh"), args, + cd: @root, + env: [{"LOGDIR", tmp}], + stderr_to_stdout: true + ) + after + File.rm_rf!(tmp) + end + end + + test "running the gate with an empty ref exits non-zero" do + {out, rc} = run_gate(["baseline", ""]) + + assert rc != 0, + """ + `tools/gate.sh baseline ""` exited #{rc}. A required status check just reported a + pass having compared zero baselines, which is the whole defect of slice 16k. + Output: + + #{out} + """ + + assert out =~ "empty ref", + "expected the refusal to name the empty ref; got:\n#{out}" + end + + # The POSITIVE control, and it closes a gap review found rather than a defect review + # measured. The tests beside this one assert that the gate REFUSES an empty ref and that the + # sentinel PASSES; neither asserts that it ever actually compares anything. Round 6 pointed out that an + # unconditional `return 0` placed after the sentinel branch leaves both of them green while + # `baseline ` returns 0 having compared nothing -- the slice's own defect, moved + # a few lines down the function. + # + # `HEAD`, not `HEAD~1`: `actions/checkout@v4` defaults to fetch-depth 1 and the + # `Gate - test ratchet` job does not override it, so `HEAD~1` does not exist in CI. `HEAD` + # always does, and comparing the committed baseline against the working tree's is a real + # comparison of both keys. The assertion is that both keys are NAMED, not that they hold any + # particular value: the values are the ratchet's and move without this test's leave. + test "the gate actually compares when handed a real ref" do + {out, rc} = run_gate(["baseline", "HEAD"]) + + assert rc == 0, "`tools/gate.sh baseline HEAD` exited #{rc}:\n#{out}" + + for key <- ["credo_issues", "dialyzer_warnings"] do + assert out =~ key, + """ + `tools/gate.sh baseline HEAD` did not compare #{key}. Output: + + #{out} + A gate that returns 0 without naming what it compared is the defect this slice + exists to close, and the tests beside this one cannot see it, because they only + exercise the paths that return before any comparison happens. + """ + end + end + + test "the all-zeros sentinel still passes, and says which case it took" do + {out, rc} = run_gate(["baseline", @zeros]) + + assert rc == 0, + "the all-zeros sentinel is a real value with a real meaning (a push that creates " <> + "a branch) and must keep passing; got rc=#{rc}:\n#{out}" + + assert out =~ "sentinel", + """ + the sentinel path passed but did not identify itself as the sentinel. Before 16k + one message -- "no previous ref (new branch)" -- covered BOTH this and the empty + ref, which is how two different situations shared one verdict for as long as they + did. Output: + + #{out} + """ + end +end diff --git a/tools/gate.sh b/tools/gate.sh index 42e2ddc..01d42ad 100755 --- a/tools/gate.sh +++ b/tools/gate.sh @@ -161,17 +161,63 @@ ratchet() { # A's raise is never compared, and no later run revisits it. On a slice branch a # multi-commit push is the normal case, so that hole is the common path, not the corner. # -# Fails closed on zero comparisons. Previously the PR path could exit 0 having compared -# nothing and printed nothing, because `git show` had no failure branch and an empty -# `old` made every key skip. +# Fails closed on zero comparisons, with EXACTLY ONE exception: the all-zeros sentinel +# branch below, which returns 0 having compared nothing. That carve-out is stated here, in the same +# sentence as the rule, and not two paragraphs below it -- because the line this replaces +# said "Fails closed on zero comparisons" as an unqualified universal and was read that way +# for as long as it stood, while the empty-ref path passed on zero comparisons. Slice 16k +# closed that path; review then caught the replacement making the same structural mistake in +# weaker form. It is not the first correction of this kind in this function -- see the +# "too strong and is corrected here" note further down, and the two-wrong-mechanisms note +# near the top of the file. (An ordinal stood here and was dropped: nobody can check "the +# third", and the count moves every time another one is written.) +# A headline universal whose exception lives further down is how the first one survived. +# +# What changed in 16k: an empty string matched the same branch as the sentinel and returned +# 0. `github.event.before` is absent from the schedule and workflow_dispatch payloads, so CI +# took that path on two of its four triggers; run 34028977996 (schedule, main, 778accb8) is +# green on zero comparisons, printing "no previous ref (new branch)" about a branch that is +# `main`. +# +# Empty is not a ref. It is never a value a caller means; it only arrives from a payload +# field that was not populated, and a gate handed one has measured nothing and must say so. +# The all-zeros SENTINEL is different -- a real value with a real meaning, a push that +# creates a branch -- and still returns 0 in that branch. +# +# No PRODUCTION caller reaches it: .githooks/pre-commit:247 passes a literal HEAD, and +# ci.yml's `elif` intercepts all-zeros and routes it to `sha~1`. One caller does -- +# apps/hacktui_core/test/ci_baseline_guard_test.exs, which pins this branch by running it. +# (Two earlier versions of this paragraph were wrong and both were caught by review: the +# first called the pass "a local fallback" when no local caller existed, and the second said +# NO caller in the tree reaches it, in the same commit that added one.) +# +# It is kept because the sentinel is a real git value a future caller may legitimately hand +# over, and because deleting it would answer that caller with a message about the wrong +# thing. Measured, with the branch deleted in a throwaway copy: +# +# baseline FAIL -- .claude/gate-baseline.json absent at 0000...0000; refusing to pass unmeasured +# +# (the forty zeros are abbreviated; nothing else is trimmed) +# +# -- a report that a file is missing at a ref, when the ref is a sentinel meaning "there is +# no previous ref". An earlier version of this sentence guessed the caller would land on the +# empty-ref refusal instead. It would not; the guess was measured and was wrong. +# +# Previously the PR path could exit 0 having compared nothing and printed nothing, because +# `git show` had no failure branch and an empty `old` made every key skip. # --------------------------------------------------------------------------- baseline_gate() { local ref="${1:-}" prev="$LOGDIR/base.prev" compared=0 raised=0 [ -r "$BASELINE" ] || { note baseline "FAIL -- $BASELINE missing or unreadable"; return 1; } - if [ -z "$ref" ] || [ "$ref" = "0000000000000000000000000000000000000000" ]; then - note baseline "no previous ref (new branch); nothing to compare" + if [ -z "$ref" ]; then + note baseline "FAIL -- empty ref; refusing to pass unmeasured (an absent event payload field, not a ref)" + return 1 + fi + + if [ "$ref" = "0000000000000000000000000000000000000000" ]; then + note baseline "all-zeros sentinel (branch created by this push); nothing to compare" return 0 fi diff --git a/tools/mutants/c5.tsv b/tools/mutants/c5.tsv index 3dadeb0..f355620 100644 --- a/tools/mutants/c5.tsv +++ b/tools/mutants/c5.tsv @@ -16,3 +16,19 @@ sensor_field_cap apps/hacktui_sensor/lib/hacktui_sensor/collectors/network.ex @m sensor_buffer_bound apps/hacktui_sensor/lib/hacktui_sensor/collectors/network.ex @max_buffer_bytes 64 * 1024 @max_buffer_bytes 1_073_741_824 apps/hacktui_sensor/test/collectors/ sensor_injection_floor apps/hacktui_sensor/lib/hacktui_sensor/collectors/network.ex do: floor_severity(classify_traffic(proto, event.info, service, site, event)), do: classify_traffic(proto, event.info, service, site, event), apps/hacktui_sensor/test/collectors/ journald_summary_cap apps/hacktui_sensor/lib/hacktui_sensor.ex summary = Text.ingest_or_nil(raw_summary, max_bytes: 200) || "journal event" summary = raw_summary apps/hacktui_sensor/test/collectors/ +ci_baseline_guard_route .github/workflows/ci.yml ./tools/gate.sh baseline "${{ github.sha }}~1" ./tools/gate.sh baseline "" apps/hacktui_core/test/ci_baseline_guard_test.exs +ci_baseline_guard_condition .github/workflows/ci.yml || [ -z "${{ github.event.before }}" ]; then\n # Same guard the attestation job carries above ; then\n # Same guard the attestation job carries above apps/hacktui_core/test/ci_baseline_guard_test.exs +gate_baseline_empty_ref_passes tools/gate.sh note baseline "FAIL -- empty ref; refusing to pass unmeasured (an absent event payload field, not a ref)"\n return 1 note baseline "no previous ref (new branch); nothing to compare"\n return 0 apps/hacktui_core/test/ci_baseline_guard_test.exs +gate_baseline_early_bypass tools/gate.sh if [ -z "$ref" ]; then [ -z "$ref" ] && { note baseline "no previous ref (new branch); nothing to compare"; return 0; }\n if [ -z "$ref" ]; then apps/hacktui_core/test/ci_baseline_guard_test.exs +ci_baseline_second_guard .github/workflows/ci.yml elif [ "${{ github.event.before }}" = "0000000000000000000000000000000000000000" ] || [ -z "${{ github.event.before }}" ]; then\n # Same guard the attestation job carries above elif [ "${{ github.event_name }}" = "schedule" ] || [ "${{ github.event_name }}" = "workflow_dispatch" ]; then\n ./tools/gate.sh baseline "0000000000000000000000000000000000000000"\n elif [ "${{ github.event.before }}" = "0000000000000000000000000000000000000000" ] || [ -z "${{ github.event.before }}" ]; then\n # Same guard the attestation job carries above apps/hacktui_core/test/ci_baseline_guard_test.exs +ci_baseline_computed_sentinel .github/workflows/ci.yml elif [ "${{ github.event.before }}" = "0000000000000000000000000000000000000000" ] || [ -z "${{ github.event.before }}" ]; then\n # Same guard the attestation job carries above elif [ "${{ github.event_name }}" = "schedule" ] || [ "${{ github.event_name }}" = "workflow_dispatch" ]; then\n ./tools/gate.sh baseline "$(printf '0%.0s' $(seq 40))"\n elif [ "${{ github.event.before }}" = "0000000000000000000000000000000000000000" ] || [ -z "${{ github.event.before }}" ]; then\n # Same guard the attestation job carries above apps/hacktui_core/test/ci_baseline_guard_test.exs +ci_baseline_bash_prefixed .github/workflows/ci.yml elif [ "${{ github.event.before }}" = "0000000000000000000000000000000000000000" ] || [ -z "${{ github.event.before }}" ]; then\n # Same guard the attestation job carries above elif [ "${{ github.event_name }}" = "schedule" ]; then\n bash tools/gate.sh baseline "$(printf '0%.0s' $(seq 40))"\n elif [ "${{ github.event.before }}" = "0000000000000000000000000000000000000000" ] || [ -z "${{ github.event.before }}" ]; then\n # Same guard the attestation job carries above apps/hacktui_core/test/ci_baseline_guard_test.exs +ci_baseline_extra_space .github/workflows/ci.yml elif [ "${{ github.event.before }}" = "0000000000000000000000000000000000000000" ] || [ -z "${{ github.event.before }}" ]; then\n # Same guard the attestation job carries above elif [ "${{ github.event_name }}" = "workflow_dispatch" ]; then\n ./tools/gate.sh baseline "$(printf '0%.0s' $(seq 40))"\n elif [ "${{ github.event.before }}" = "0000000000000000000000000000000000000000" ] || [ -z "${{ github.event.before }}" ]; then\n # Same guard the attestation job carries above apps/hacktui_core/test/ci_baseline_guard_test.exs +ci_baseline_duplicate_job_key .github/workflows/ci.yml baseline:\n name: Gate - baseline may only decrease baseline:\n name: Gate - decoy\n runs-on: ubuntu-latest\n steps:\n - run: ./tools/gate.sh baseline "$(printf '0%.0s' $(seq 40))"\n\n baseline:\n name: Gate - baseline may only decrease apps/hacktui_core/test/ci_baseline_guard_test.exs +ci_baseline_glob_spelling .github/workflows/ci.yml ./tools/gate.sh baseline "${{ github.sha }}~1" bash tools/gate.s? baseline "$(printf '0%.0s' $(seq 40))" apps/hacktui_core/test/ci_baseline_guard_test.exs +ci_baseline_trailing_true .github/workflows/ci.yml ./tools/gate.sh baseline "${{ github.event.before }}"\n fi ./tools/gate.sh baseline "${{ github.event.before }}"\n fi\n true apps/hacktui_core/test/ci_baseline_guard_test.exs +ci_baseline_continue_on_error .github/workflows/ci.yml name: Gate - baseline may only decrease\n runs-on: ubuntu-latest\n timeout-minutes: 10 name: Gate - baseline may only decrease\n runs-on: ubuntu-latest\n timeout-minutes: 10\n continue-on-error: true apps/hacktui_core/test/ci_baseline_guard_test.exs +ci_baseline_job_if_skip .github/workflows/ci.yml name: Gate - baseline may only decrease\n runs-on: ubuntu-latest\n timeout-minutes: 10 name: Gate - baseline may only decrease\n runs-on: ubuntu-latest\n timeout-minutes: 10\n if: github.event_name != 'schedule' apps/hacktui_core/test/ci_baseline_guard_test.exs +ci_baseline_job_renamed .github/workflows/ci.yml name: Gate - baseline may only decrease name: Gate - baseline decrease check apps/hacktui_core/test/ci_baseline_guard_test.exs +gate_baseline_post_sentinel_bypass tools/gate.sh if ! git cat-file -e "$ref:$BASELINE" 2>/dev/null; then note baseline "nothing to compare"; return 0\n if ! git cat-file -e "$ref:$BASELINE" 2>/dev/null; then apps/hacktui_core/test/ci_baseline_guard_test.exs +ci_baseline_cr_smuggled_line .github/workflows/ci.yml ./tools/gate.sh baseline "origin/${{ github.base_ref }}" ./tools/gate.sh baseline "origin/${{ github.base_ref }}"\n # harmless note exit 0 apps/hacktui_core/test/ci_baseline_guard_test.exs