Wave 9: sibling-call sweep, guarddog step, Actions allow-list rule, pipeline hub fixes, perf A/B and load smoke

.claude/skills/INDEX.md

@@ -148,9 +148,10 @@ Run `python3 <skill>/scripts/<name>` (`bash` for `.sh`). `name: purpose — argu
 - review_checks.sh: run local, read-only analyzers over the files changed since… — [--base REF] [--tests] [--history]…
 - score_review.py: Score a review's findings against a ground-truth file: recall… — GROUND_TRUTH.json FINDINGS.json…
 - security_verify.py: verify-or-drop triage of security findings. — [findings.json|-] [override-file]
+- sibling_sweep.py: leads for an incomplete fix: a call changed here, its… — --repo DIR --base REF [--head REF]
 - tenant_scope_check.py: heuristic lead-finder for cross-tenant authz holes in Prisma… — error. Reads files only; writes nothing.
 - token_differ.py: the tier-1 design<->app design-token VALUE parity gate. — --design <file> --app <file> [--map…
-- No documented CLI: app_pentest.sh, binaries_gate.py, ci_cost_lint.py, closes_lint.py, identity_gate.py, invisible_chars.py, reaper_lint.py, review_feedback.py, source_scan_tests.py, supply_chain_check.sh, validate_status_claims.py
+- No documented CLI: app_pentest.sh, binaries_gate.py, ci_cost_lint.py, closes_lint.py, identity_gate.py, invisible_chars.py, perf_compare.sh, reaper_lint.py, review_feedback.py, source_scan_tests.py, supply_chain_check.sh, validate_status_claims.py
 
 ### growth-analytics
 - readout_check.py: deterministic pass/fail checks on a finished two-arm A/B… — READOUT.json

.claude/skills/agentic-delivery/scripts/heavy_gate.py

@@ -8,7 +8,7 @@
 
 What it does: if the command starts a test runner, a build, a whole-repo security scanner (semgrep scan|ci|--config, trivy
 fs|repo|image|config|rootfs, osv-scanner, grype, checkov, gitleaks detect|git|dir, trufflehog git|filesystem|github), a JS dependency
-install (npm|pnpm|yarn|bun ci|i|install) or a Perun scan/CI entrypoint (review_checks.sh, run-ci-tests.sh, app_pentest.sh, run as `bash PATH`
+install (npm|pnpm|yarn|bun ci|i|install) or a Perun scan/CI entrypoint (review_checks.sh, run-ci-tests.sh, app_pentest.sh, perf_compare.sh, run as `bash PATH`
 or directly) (DEFAULT_PATTERNS, or the newline-separated regexes in $PERUN_HEAVY_PATTERNS, each matched at the start of a shell segment,
 after runner wrappers are stripped), it denies the call when host_probe.decide(lane_type="heavy") says HOLD (load1 > cores or low free
 RAM; swap is not read, one sample shows no trend) or when the machine-wide heavy leases already fill the
@@ -50,7 +50,7 @@
     r"semgrep( .*)? (scan|ci|--config)\b", r"trivy( .*)? (fs|repo|image|config|rootfs)\b", r"(osv-scanner|checkov) \S",
     r"grype (?!(db|version|help)(\s|$))\S",
     r"gitleaks( .*)? (detect|git|dir)\b", r"trufflehog( .*)? (git|filesystem|github)\b", r"(npm|pnpm|yarn|bun) (ci|i|install)\b",
-    r"((ba|z)?sh )?(\S*/)?(review_checks|run-ci-tests|app_pentest)\.sh\b",
+    r"((ba|z)?sh )?(\S*/)?(review_checks|run-ci-tests|app_pentest|perf_compare)\.sh\b",
 )
 PUNCT = ";()<>|&\n"
 # Joins backslash-newline (group 1 = the escaped pairs kept: an odd run of backslashes before the newline, an even run is escaped backslashes) and

.claude/skills/agentic-delivery/scripts/pipeline_hub.py

@@ -26,10 +26,15 @@
 
 Exit codes: 0 ok, 2 usage, an unreadable or unwritable --data, a --root that is not a checkout of --repo, or no data.
 Side effects: `gh api` calls (read-only); `--data` writes that file.
-Limits: durations are job wall time (started to completed); a duplicate is the same check suite key in two enforced CI
-steps (a PR or merge-group gate; a build, install or generate step is never flagged), so a matrix or OS split also
-shows up: each recommendation says to check that first. Jobs match on workflow and displayed name, with a matrix
-suffix ignored. Duplicates are read from the workflow files under --root, so --root must be a checkout of --repo.
+Limits: durations are job wall time (started to completed); a duplicate is the same check command (script and
+arguments) in two enforced CI steps (a PR or merge-group gate; a build, install or generate step is never flagged), so a
+matrix or OS split also shows up: each recommendation says to check that first. The same script with different
+arguments is an advisory (check one covers the other before dropping), never a drop, and is not priced. The API times
+steps, not suites: a saving is priced only for a step that runs that one command; for any other step the suite's own
+time is unmeasured and the recommendation says to time it locally. Jobs match on workflow and displayed name, with a
+matrix suffix ignored. Duplicates are read from the workflow files under --root, so --root must be a checkout of --repo.
+A run whose jobs or attempts call fails is named in the notes and its job data stays missing until a later run with
+the same --data retries it; a failed runs-list or pulls call stops collection and says so.
 """
 from __future__ import annotations
 
@@ -59,19 +64,37 @@ def is_check(key):
     return key.split()[0] in pp._RUN or bool(CHECK.search(key.lower()))
 
 
+def api_error(err):
+    """One line for a failed `gh api` call's stderr: gh's own `<message> (HTTP nnn)` line, else the JSON body's
+    "message" (what the curl fallback keeps, after `HTTP nnn:`) with that status, else the last line. Never the bare
+    `}` that ends a multi-line body. Blank stderr gives "gh api failed"."""
+    err = (err or "").strip() or "gh api failed"
+    m = re.search(r"^gh: (.+ \(HTTP \d+\))$", err, re.M)
+    if m:
+        return m.group(1)[:300]
+    m = re.search(r'"message":\s*"((?:[^"\\]|\\.)*)"', err)
+    if m:
+        st = re.match(r"HTTP (\d+)", err)
+        return (m.group(1) + (" (HTTP %s)" % st.group(1) if st else ""))[:300]
+    return err.splitlines()[-1][:300]
+
+
 def gh_get(path, params=None):
-    """GET one REST page through bare `gh api` (curl fallback on gh's macOS-sandbox TLS failure) -> parsed JSON."""
+    """GET one REST page through bare `gh api` (curl fallback on gh's macOS-sandbox TLS failure) -> parsed JSON.
+    A failure raises OSError with `api_error`'s one-line reason."""
     args = ["gh", "api", path + ("?" + urllib.parse.urlencode(params) if params else "")]
     p = subprocess.run(args, capture_output=True, text=True)
     rc, out, err = gh_rest.fallback(args, p.returncode, p.stdout, p.stderr)
     if rc:
-        raise OSError((err or "gh api failed").strip().splitlines()[-1][:300])
+        raise OSError(api_error(err))
     return json.loads(out)
 
 
 def collect(repo, bundle, n_runs=30, n_pulls=30, get=gh_get):
-    """Fill `bundle` {runs, jobs, prior, pulls, limits} from the API, reusing what is there. A failed call stops
-    collection and is recorded in bundle['limits'] (rerun with the same --data to resume); it never raises."""
+    """Fill `bundle` {runs, jobs, prior, pulls, limits} from the API, reusing what is there. A failed jobs or attempts
+    call skips that run and is recorded once in bundle['limits'] with the run id and the API message (a later run
+    with the same --data retries it); a failed runs-list or pulls call stops collection and is recorded there too
+    (rerun with the same --data to resume). It never raises."""
     for k, v in (("runs", []), ("jobs", {}), ("prior", {}), ("pulls", [])):
         bundle.setdefault(k, v)
     bundle["limits"] = []
@@ -83,13 +106,17 @@ def collect(repo, bundle, n_runs=30, n_pulls=30, get=gh_get):
             bundle["runs"] = [pp._run_row(r) for r in got.get("workflow_runs", [])][:n_runs]
         for r in bundle["runs"]:
             rid = str(r["id"])
-            if rid not in bundle["jobs"]:
-                got = get("%s/actions/runs/%s/jobs" % (base, rid), {"per_page": 100, "filter": "latest"})
-                bundle["jobs"][rid] = [pp._job_row(j) for j in got.get("jobs", [])]
-            for a in range(1, (r.get("run_attempt") or 1)):  # earlier attempts: which job failed before the re-run
-                if "%s/%d" % (rid, a) not in bundle["prior"]:
-                    got = get("%s/actions/runs/%s/attempts/%d/jobs" % (base, rid, a), {"per_page": 100})
-                    bundle["prior"]["%s/%d" % (rid, a)] = [pp._job_row(j) for j in got.get("jobs", [])]
+            try:  # one run's endpoint can fail every time (an HTTP 502 on its /jobs): name it, go on to the next run
+                if rid not in bundle["jobs"]:
+                    got = get("%s/actions/runs/%s/jobs" % (base, rid), {"per_page": 100, "filter": "latest"})
+                    bundle["jobs"][rid] = [pp._job_row(j) for j in got.get("jobs", [])]
+                for a in range(1, (r.get("run_attempt") or 1)):  # earlier attempts: which job failed before the re-run
+                    if "%s/%d" % (rid, a) not in bundle["prior"]:
+                        got = get("%s/actions/runs/%s/attempts/%d/jobs" % (base, rid, a), {"per_page": 100})
+                        bundle["prior"]["%s/%d" % (rid, a)] = [pp._job_row(j) for j in got.get("jobs", [])]
+            except (OSError, ValueError) as e:
+                # ponytail: an outage adds one note per remaining run; add a consecutive-failure stop if that shows up
+                bundle["limits"].append("run %s skipped, its jobs or attempts call failed: %s" % (rid, e))
         for page in range(1, (n_pulls + 9) // 10 + 1):  # a PR object is large: small pages survive a body-cutting proxy
             if len(bundle["pulls"]) >= n_pulls or bundle.get("pulls_pages", 0) >= page:
                 continue
@@ -192,11 +219,16 @@ def oss_jobs(repo, root, run=subprocess.run):
 
 def dup_jobs(root, jobs):
     """Check suites (suite keys from `pipeline_profile.commands`, `is_check` only) run by two or more enforced CI steps
-    (a PR or merge-group gate: a push-to-main run is the post-merge copy, not a duplicate). Keep the copy in the job
-    with the lower p50 (red sooner); drop the others and price each from its step's median, split evenly over the
-    suites that step runs (an even split, not a measurement). Job rows match on workflow and job name."""
+    (a PR or merge-group gate: a push-to-main run is the post-merge copy, not a duplicate). The same normalized command
+    (script and arguments, as `commands` prints it) in two steps is a duplicate (`identical` True): keep the copy in the
+    job with the lower p50 (red sooner) and drop the others. A suite whose steps all run it with different arguments is
+    only an advisory (`identical` False, `saved_s` None): one may not cover the other. (Beside a repeated command, a
+    copy with other arguments is left out: only the repeat is reported.) A step's median prices a copy only when that
+    step runs that one command; the API times whole steps, never one suite, so for any other step
+    `step_p50_s` is None (unmeasured) rather than an even split of the step; `saved_s` sums the measured copies (None
+    when none is). Job rows match on workflow and job name."""
     wdir = os.path.join(root, ".github", "workflows")
-    seen, nkeys, names = {}, {}, {}
+    seen, ncmd, names = {}, {}, {}  # seen: suite key -> normalized command -> {step location}
     for fn in sorted(os.listdir(wdir)) if os.path.isdir(wdir) else []:
         if not fn.endswith((".yml", ".yaml")):
             continue
@@ -208,31 +240,37 @@ def dup_jobs(root, jobs):
             for st in job["steps"]:
                 if st["run"] and pp._wf_stage(wf, st, job)[1]:
                     loc = (fn, job["name"], pp.step_label(st))
-                    for _, key, _b in pp.commands(st["run"]):
-                        nkeys.setdefault(loc, set()).add(key)  # the split counts every suite the step runs
+                    for cmd, key, _b in pp.commands(st["run"]):
+                        ncmd[loc] = ncmd.get(loc, 0) + 1  # every suite command counts: the step median covers them all
                         if is_check(key):
-                            seen.setdefault(key, {})[loc] = 1
+                            seen.setdefault(key, {}).setdefault(cmd, {})[loc] = 1
 
     def find(fn, name):
         return [j for j in jobs if j["workflow"] in names[fn]
                 and (j["job"] == name or j["job"].startswith(name + " ("))]
 
+    groups = []  # (suite key, identical, {step location}), one per command that repeats
+    for key, by_cmd in sorted(seen.items()):
+        # no command repeats: the same suite with different arguments in each step is only an advisory
+        groups += [(key, True, locs) for locs in by_cmd.values() if len(locs) > 1] or [
+            (key, False, dict.fromkeys(loc for locs in by_cmd.values() for loc in locs))]
     out = []
-    for key, locs in sorted(seen.items()):
+    for key, identical, locs in groups:
         if len(locs) < 2:
             continue
         rows = []
         for fn, job, step in locs:
             m = find(fn, job)
-            sec = [s / len(nkeys[(fn, job, step)]) for j in m for s in [j["step_p50"].get(step)] if s is not None]
+            sec = [s for j in m for s in [j["step_p50"].get(step)] if s is not None and ncmd[(fn, job, step)] == 1]
             rows.append({"workflow_file": fn, "job": job, "step": step,
                          "job_p50_s": max((j["p50_s"] or 0 for j in m), default=None) if m else None,
                          "step_p50_s": max(sec) if sec else None})
         order = sorted(range(len(rows)), key=lambda i: (rows[i]["job_p50_s"] is None, rows[i]["job_p50_s"] or 0, i))
         keep = rows[order[0]]
         drop = [rows[i] for i in order[1:]]
-        out.append({"suite": key, "keep": keep, "drop": drop,
-                    "saved_s": sum(r["step_p50_s"] or 0 for r in drop) if any(r["step_p50_s"] for r in drop) else None})
+        out.append({"suite": key, "identical": identical, "keep": keep, "drop": drop,
+                    "saved_s": sum(r["step_p50_s"] or 0 for r in drop)
+                    if identical and any(r["step_p50_s"] for r in drop) else None})
     return out
 
 
@@ -253,12 +291,15 @@ def recommend(wfs, jobs, dups, lead, flake_jobs=None):
         if all(x["step_p50_s"] is not None and x["step_p50_s"] < DUP_FLOOR_S for x in drops):
             continue
         for x in drops:
+            if d["identical"]:
+                tail = (". Run it once: keep the first, drop the second, unless the copies are a matrix or OS split; %s."
+                        % ("saves at most about %s per run (its step's median)" % f(x["step_p50_s"]) if x["step_p50_s"]
+                           else "the suite's own time is unmeasured: time it locally to price the saving"))
+            else:
+                tail = "; same suite, different arguments: check one covers the other before dropping."
             rec.append({"kind": "duplicate-job", "target": x["job"],
-                        "text": "suite `%s` runs in job `%s` (step `%s`) and again in job `%s` (step `%s`, %s). Run it "
-                                "once: keep the first, drop the second, unless the copies are a matrix or OS split%s."
-                                % (d["suite"], k["job"], k["step"], x["job"], x["step"], x["workflow_file"],
-                                   "; saves about %s per run (even split over its step's suites)" % f(x["step_p50_s"])
-                                   if x["step_p50_s"] else "")})
+                        "text": "suite `%s` runs in job `%s` (step `%s`) and again in job `%s` (step `%s`, %s)%s"
+                                % (d["suite"], k["job"], k["step"], x["job"], x["step"], x["workflow_file"], tail)})
     if jobs:
         j = jobs[0]
         rec.append({"kind": "slowest-job", "target": j["job"],
@@ -309,8 +350,10 @@ def render(rep):
                                                 "n/a" if j["flakes"] is None else j["flakes"])
           for j in rep["slowest_jobs"]]
     L += ["", "DUPLICATE CHECKS (same suite in 2+ CI steps)"]
-    L += ["  %s: keep in `%s`; also in %s" % (d["suite"], d["keep"]["job"],
-                                              ", ".join("`%s`" % x["job"] for x in d["drop"])) for d in rep["duplicates"]]
+    names = lambda xs: ", ".join("`%s`" % x["job"] for x in xs)  # noqa: E731
+    L += ["  %s: keep in `%s`; also in %s" % (d["suite"], d["keep"]["job"], names(d["drop"])) if d["identical"] else
+          "  %s: different arguments in %s (advisory: check one covers the other)"
+          % (d["suite"], names([d["keep"]] + d["drop"])) for d in rep["duplicates"]]
     L += ["  none found"] * (not rep["duplicates"])
     ld = rep["lead"]
     L += ["", "PR LEAD TIME  prs=%d  open-to-merge p50=%s p90=%s  push-to-merge=%s  post-green wait=%s  merged/day=%s" % (

.claude/skills/deep-code-review/INDEX-scripts.md

@@ -28,13 +28,15 @@ Generated by `scripts/skill_index.py` (do not hand-edit). The script half of `IN
 | `scripts/outline.py` | cheap symbol map (line ranges) so a lane reads only the ranges it needs, not the whole file. | outline.py FILE... |
 | `scripts/parity_differ.py` | the two-sided design<->app parity gate: sections first, then the element inventory of every matched… | parity_differ.py --design <file> --app <file> [--accept <tsv> [--accept-rev REV]] [--min-pairs N] [--style [--style-min… |
 | `scripts/partition_diff.py` | split a unified diff into whole-file review chunks, deterministically. | partition_diff.py [--diff FILE \| --base REF] [--max-lines N] [--split-hunks] [--skip-glob GLOB ...] [--out-dir DIR] par… |
+| `scripts/perf_compare.sh` | perf_compare.sh [--runs N] [--base REF] [--json FILE] -- CMD...: hyperfine A/B of CMD in a detached… | — |
 | `scripts/post_review.sh` | post review findings to a PR (opt-in). | post_review.sh <PR-number> <findings.json> [--post] |
 | `scripts/provision_tools.sh` | make review_checks.sh scanners actually run. | provision_tools.sh [--print] [DIR] |
 | `scripts/reaper_lint.py` | a heuristic, opt-in lint that flags a bulk/fleet-wide process-reaper pattern in a cleanup script be… | — |
 | `scripts/review_checks.sh` | run local, read-only analyzers over the files changed since REF (default origin/main, compared as R… | review_checks.sh [--base REF] [--tests] [--history] [--full] [--timeout SECS] |
 | `scripts/review_feedback.py` | repo-local accept/dismiss ledger for review findings. | — |
 | `scripts/score_review.py` | Score a review's findings against a ground-truth file: recall and precision under an FP budget. | score_review.py GROUND_TRUTH.json FINDINGS.json [--max-fp N] |
 | `scripts/security_verify.py` | verify-or-drop triage of security findings. | security_verify.py [findings.json\|-] [override-file] Input: JSON list of findings on stdin or a file, or a semgrep --js… |
+| `scripts/sibling_sweep.py` | leads for an incomplete fix: a call changed here, its unchanged siblings there. | sibling_sweep.py --repo DIR --base REF [--head REF] |
 | `scripts/source_scan_tests.py` | flag a "test" that reads UI source as TEXT instead of rendering it, the class of test that cannot o… | — |
 | `scripts/supply_chain_check.sh` | supply_chain_check.sh [--online] [--fail-on critical\|high\|moderate\|low] [--scorecard owner/repo] [-… | — |
 | `scripts/tenant_scope_check.py` | heuristic lead-finder for cross-tenant authz holes in Prisma server code (report-only, stdlib only). | error. Reads files only; writes nothing. ponytail: text scan, not an AST -- regex literals with quotes or a backtick in… |