From b095cfb5b0782f61c16d42d73f16bb8bc17a3849 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Topi=20J=C3=A4rvinen?= Date: Sun, 2 Aug 2026 13:11:56 +0300 Subject: [PATCH] =?UTF-8?q?docs:=20give=20the=20panel's=20filing=20rule=20?= =?UTF-8?q?a=20recovery=20path=20=E2=80=94=20withdraw,=20don't=20patch=20a?= =?UTF-8?q?gain?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The doctrine already says a new mechanism gets filed "however squarely a finding prompted it". That is prevention, and PR #225 measured it failing: a mechanism a round-1 finding prompted was built rather than filed, and every later round found a HIGH inside that mechanism while the rest of the change drew nothing. Rounds 2-5 reviewed the fixes, not the work. What was missing is the recovery. Once the mechanism is in the diff and drawing HIGHs, the existing rule has nothing to say, and the default — patch it — is what turns two rounds into five. The rule added names a signal that is not a round count: a fix that itself draws a HIGH, twice, in the same mechanism. Past that, each patch is a fresh guess at a problem the previous guess misread. Withdraw the mechanism, file it with what defeated it, and let the removing round be the last — it deletes surface rather than adding more for the next round to find. The companion measurement is the counterfactual: filed at round 1 as the rule already required, #225 is two rounds instead of five. Second half is operational. The threshold has to be declared BEFORE a round's findings, beside the stopping criterion — named in advance it is a decision; reached afterwards it is indistinguishable from fatigue, and no reviewer can tell which it was. #225 declared it at round 4 and honoured it. Placement: the rule goes in the paragraph that already carries the filing rule rather than opening a section, and the measurement goes to the companion — #213 stays open on this file's length, so an addition here has to earn its lines. The executable half is 14 lines; the evidence is 40 and lands in the file #213's split created for exactly that. Both files are KIT_OWNED, so kit-manifest.json is regenerated. Verified: the addition is outside `## The contract every lens gets`, which is the only section panel_prompt.py parses — `contract()` still returns all 13 items, so no launch prompt changes and #167's stable numbering is untouched. make test -> 699 passed. kit_doctor -> 32 unchanged, 0 differ, 0 missing, 0 unknown, exit 0. #213, #228 and #163's missing cost measurement all stay open. --- .../fallback-review-panel-evidence.md | 41 +++++++++++++++++++ docs/agentic-dev-kit/fallback-review-panel.md | 16 ++++++++ kit-manifest.json | 4 +- 3 files changed, 59 insertions(+), 2 deletions(-) diff --git a/docs/agentic-dev-kit/fallback-review-panel-evidence.md b/docs/agentic-dev-kit/fallback-review-panel-evidence.md index f0b3808..8b0ca3f 100644 --- a/docs/agentic-dev-kit/fallback-review-panel-evidence.md +++ b/docs/agentic-dev-kit/fallback-review-panel-evidence.md @@ -122,6 +122,47 @@ clause is `safety-critical-changes.md` rule 3's own wording, quoted from the sibling doctrine file rather than from a tracker issue — a review lens went looking for it on the tracker and could not source it.) +### The session that measured withdrawal + +PR `#225` (`#41`, the required/optional manifest axis). Five rounds: + +| Round | What it reviewed | HIGH | +|---|---|---| +| 1 | the change | 2 | +| 2 | the round-1 fix | 2 | +| 3 | the round-2 fix | 2 | +| 4 | the round-3 fix | 1 | +| 5 | the withdrawal | 0 | + +Only round 1 reviewed the change. Its second HIGH was a real fail-open — the +derived graph missed `lib/repo_root.sh`, which two shell engines `source`, so +`kit_doctor` called a broken tree clean at exit 0 — and the fix was a bash +`source` scanner: a new mechanism, prompted by a finding, which the filing rule +says to file rather than build. It was built. + +Every HIGH in rounds 2–4 was inside that scanner, and each was a defect in the +previous round's fix for it: + +- **round 2** — the scan missed guarded `source` (`[ -f "$L" ] && source "$L"`), + and its exit-code test was confounded by its own fixture, so deleting the + mechanism under test left the suite green +- **round 3** — the widened anchor matched `source` inside a quoted string, so + `echo "run this; source lib/dep.sh"` became a real edge; and the heredoc + tracker read `<<<` and arithmetic `<<` as openers, blinding whole files +- **round 4** — the heredoc tracker recognised fewer opener forms than bash + accepts (`cmd <