From 1dcc88a3a5e0350045a5516474fb26f40a184247 Mon Sep 17 00:00:00 2001 From: Remi Antczak <140408771+remigiusz-antczak@users.noreply.github.com> Date: Fri, 30 Oct 2026 09:03:53 +0100 Subject: [PATCH 1/2] Wave 7: schemathesis API fuzzing, offline Scorecard for a git DIR, committed-personal-data check, CI liveness rule, comms_lint audit-pr area budget MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Review hub: - New committed-data step in review_checks.sh (after invisible-chars): committed_personal_data.py counts the distinct email-shaped strings per tracked file, binary data included, and reports a file at or above 10 by path and count only, never an address. gitleaks, trufflehog and semgrep look for secrets, so a committed export of real people's addresses passed the hub clean. Changed files in diff mode, every tracked file under --full. - A repository-total row (path ".", --repo-threshold, default 50) sums the addresses of files that stay below the file threshold, so a leak spread one address per file is not silent. - Compressed and columnar files the scan cannot read (parquet, Arrow, ORC, zip, gzip, zstd, xz, 7z, lz4 frame, bzip2, Avro, PDF, zlib) get a not-checked row (path and format only) instead of a silent pass. Parquet, Arrow and ORC need their footer magic as well, and a bare magic is not a container. - Reserved example and test names, asset-scale names such as icon@2x.png and scp-style git@ remotes are not counted. --max-bytes is range-checked. method-situational.md names the step and its rows. Supply chain: - supply_chain_check.sh scores a git DIR with OpenSSF Scorecard offline when --scorecard owner/repo is not given, instead of printing SKIPPED. It scans a git archive export of the repo root's HEAD (tracked files only, so an untracked node_modules cannot score Binary-Artifacts 0) and runs only the checks that work offline: Binary-Artifacts, Dangerous-Workflow, Dependency-Update-Tool, License, Packaging, Pinned-Dependencies, SAST, Security-Policy and Token-Permissions. Vulnerabilities is never run (it reported a false 10 while api.osv.dev was unreachable) and Fuzzing errors offline. Scores stay informational; the ok line says the view is partial and not comparable with a public score. - The export is removed by an EXIT trap set right after mktemp, so SIGTERM and SIGHUP no longer leave it in $TMPDIR. git archive runs with GIT_NO_LAZY_FETCH=1, so a partial clone is SKIPPED instead of fetching blobs. A subdirectory DIR scores the whole repo root, and the output says so. Communication structure: - comms_lint --kind audit-pr: the 1200-word, 160-line budget counts everything outside "## Changes by area"; inside it each "### " area fails above 200 words and the whole body fails above 3000 words. A body written to the required structure with many areas no longer fails for its area text. Eval audit-pr-multi-area-body-budget added. - The placeholder rule no longer flags a mktemp template (3 or more uppercase X inside a single-backtick span on one line); TBD, TODO and a lone XXX in prose still fail, and , and now fail. Review method: - domain-k.md: a workflow file is not CI coverage until it runs. Before claiming CI coverage or recommending a CI-only fix, a review confirms workflows fire (gh api repos///actions/permissions --jq .enabled, gh run list -L 5). If Actions is off or unknown it says so and aims the fix at the gate that does run, or flags re-enabling Actions as an owner decision. A real review recommended a CI step for a repo whose Actions was disabled, so the step would never have run. Eval ci-only-fix-where-actions-is-disabled-confirm-workflows-run added. Delivery checks: - test_destructive_guard_w21.py waits on the guard with one 60 s ceiling instead of 20 s (it failed once at a load average near 13). A new test pins the ceiling, that a hang still fails, and that the guard's own watchdog stays at most 20 s. Live app testing: - app_pentest.sh --openapi SPEC fuzzes the validated target's API with schemathesis under the same rails as ZAP and nuclei. SPEC is a local file or an http(s) URL; a URL must pass every target rule and is read once (no redirects, no env proxies, 10 s timeout, 10 MB cap), and schemathesis gets that copy. Passive mode sends only GET and HEAD; --active drops the method filter and runs only for a listed loopback or private host. --origin pins requests to the target, the coverage phase is off and an empty config file is pinned. - Egress block: schemathesis runs with every proxy variable pointed at a closed loopback port and no_proxy set to the target host, so a remote $ref or externalValue that the early text check misses cannot be fetched from another host, target traffic goes direct, and the caller's own proxy cannot divert it. Residual: the same host on another port, its subdomains, and a fetcher that ignores proxy variables. - An unreadable spec, one over 10 MB, or one with a remote $ref or externalValue in its text exits 2 before any tool starts; a spec schemathesis cannot load ends as error. The JSON adds openapi.tested and openapi.errored. Each failure is one lead per code and operation: medium for 5xx, 5xx on an undeclared Content-Type, unenforced authentication and a server that stopped answering; low otherwise. Tests: 59/59 affected suites (run-ci-tests.sh --affected 6611ca89b on the assembled tree 39d5d4b40); diff --check, fragments, tests-wired (485 tests, 473 run by CI, 12 allowlisted) and the size ratchet pass; test-ci-gates.sh is left to CI. size-budget-raise: .claude/skills/communication-structure/references/artifact-templates.md 9234→9443 the audit-pr paragraph states the budget scope and the two area and body caps size-budget-raise: .claude/skills/deep-code-review/references/supply-chain-check.md 3421→3646 the scorecard table row states the offline git-DIR mode, its whole-repo scope and the partial offline view size-budget-raise: .claude/skills/deep-code-review/references/method-situational.md 43864→44018 the analyzer list names the committed-data step and its rows size-budget-raise: .claude/skills/deep-code-review/references/domain-k.md 5188→5674 the liveness rule (one checklist bullet plus one red-flag clause) size-budget-raise: .claude/skills/deep-code-review/references/app-pentest.md 3033→6344 the --openapi safety rules, the egress block, the schemathesis flags with their reasons, the status semantics and the severity map Co-Authored-By: Claude Sonnet 5.5 Co-Authored-By: Claude Opus 5.5 --- .claude/skills/INDEX.md | 1 + .../skills/communication-structure/INDEX.md | 2 +- .../communication-structure/evals/evals.json | 13 + .../references/artifact-templates.md | 2 +- .../scripts/comms_lint.py | 59 +++- .../skills/deep-code-review/INDEX-scripts.md | 3 +- .claude/skills/deep-code-review/INDEX.md | 10 +- .../skills/deep-code-review/evals/evals.json | 11 + .../references/app-pentest.md | 23 +- .../deep-code-review/references/domain-k.md | 9 +- .../references/method-situational.md | 2 +- .../references/supply-chain-check.md | 2 +- .../deep-code-review/scripts/app_pentest.sh | 140 ++++++-- .../scripts/committed_personal_data.py | 248 +++++++++++++ .../deep-code-review/scripts/review_checks.sh | 16 + .../scripts/supply_chain_check.sh | 31 +- SHA256SUMS | 29 +- changelog.d/app-pentest-openapi.md | 3 + changelog.d/ci-liveness-before-ci-only-fix.md | 3 + changelog.d/committed-personal-data-check.md | 8 + changelog.d/supply-chain-scorecard-local.md | 3 + changelog.d/w7a-comms-lint-guard-wait.md | 5 + scripts/ci-tests.d/audit-pr-budget.sh | 2 + scripts/ci-tests.d/committed-personal-data.sh | 3 + .../comms-lint-placeholder-template.sh | 2 + .../ci-tests.d/destructive-guard-w21-wait.sh | 2 + scripts/size-budgets.tsv | 16 +- scripts/test-review-checks-committed-data.sh | 73 ++++ scripts/test-supply-chain-check.sh | 84 ++++- scripts/test_app_pentest.sh | 233 ++++++++++++- scripts/test_audit_pr_budget.py | 123 +++++++ scripts/test_committed_personal_data.py | 325 ++++++++++++++++++ .../test_comms_lint_placeholder_template.py | 45 +++ scripts/test_destructive_guard_w21.py | 12 +- scripts/test_destructive_guard_w21_wait.py | 69 ++++ 35 files changed, 1517 insertions(+), 95 deletions(-) create mode 100644 .claude/skills/deep-code-review/scripts/committed_personal_data.py create mode 100644 changelog.d/app-pentest-openapi.md create mode 100644 changelog.d/ci-liveness-before-ci-only-fix.md create mode 100644 changelog.d/committed-personal-data-check.md create mode 100644 changelog.d/supply-chain-scorecard-local.md create mode 100644 changelog.d/w7a-comms-lint-guard-wait.md create mode 100644 scripts/ci-tests.d/audit-pr-budget.sh create mode 100644 scripts/ci-tests.d/committed-personal-data.sh create mode 100644 scripts/ci-tests.d/comms-lint-placeholder-template.sh create mode 100644 scripts/ci-tests.d/destructive-guard-w21-wait.sh create mode 100644 scripts/test-review-checks-committed-data.sh create mode 100644 scripts/test_audit_pr_budget.py create mode 100644 scripts/test_committed_personal_data.py create mode 100644 scripts/test_comms_lint_placeholder_template.py create mode 100644 scripts/test_destructive_guard_w21_wait.py diff --git a/.claude/skills/INDEX.md b/.claude/skills/INDEX.md index fa90de686..ea59a9ed9 100644 --- a/.claude/skills/INDEX.md +++ b/.claude/skills/INDEX.md @@ -126,6 +126,7 @@ Run `python3 /scripts/` (`bash` for `.sh`). `name: purpose — argu ### deep-code-review - ci_cost_advisor.py: estimate a repo's GitHub Actions minutes and dollars, and… — [ROOT] [--visibility… +- committed_personal_data.py: count distinct email addresses per tracked file and report the… — [--threshold N] [--repo-threshold N]… - context_pack.py: bounded context pack for a DIFF review. — [--base REF] [--repo DIR] [--intent… - coverage_ledger.py: completeness check for a machine-report coverage map. — [--domains SPEC] - derived_numbers.py: bind a doc number to the counter that derives it. — [--fix] [--selftest] [ROOT] diff --git a/.claude/skills/communication-structure/INDEX.md b/.claude/skills/communication-structure/INDEX.md index c26d00bab..7dcb40a29 100644 --- a/.claude/skills/communication-structure/INDEX.md +++ b/.claude/skills/communication-structure/INDEX.md @@ -6,7 +6,7 @@ Generated by `scripts/skill_index.py` (do not hand-edit; rerun it after changing | File | Read this when... | Tok | |---|---|---| -| `references/artifact-templates.md` | you draft a status update, PR body, release note, decision memo or executive summary, incident summ… | 2308 | +| `references/artifact-templates.md` | you draft a status update, PR body, release note, decision memo or executive summary, incident summ… | 2360 | | `references/leadership-status-update.md` | you write any status update a non-technical leader will read: a tracker project update, an issue st… | 500 | ## Scripts diff --git a/.claude/skills/communication-structure/evals/evals.json b/.claude/skills/communication-structure/evals/evals.json index 5cade3798..890318804 100644 --- a/.claude/skills/communication-structure/evals/evals.json +++ b/.claude/skills/communication-structure/evals/evals.json @@ -231,6 +231,19 @@ "One 'Key figure:' line states 1,840 documents exactly, near the top", "No invented number, date, owner or recommendation" ] + }, + { + "id": "audit-pr-multi-area-body-budget", + "prompt": "Write the PR body for a security-hardening PR on the Acme API. Eight areas, each with its directory and test: auth (src/auth/, test/auth.test.ts), uploads (src/upload/, test/upload.test.ts), exports (src/export/, test/export.test.ts), webhooks (src/webhooks/, test/webhooks.test.ts), logging (src/log/, test/log.test.ts), rate limits (src/limits/, test/limits.test.ts), CORS (src/cors/, test/cors.test.ts), dependencies (package.json, test/deps.test.ts). Exports and uploads now need sign-in. 14 findings fixed; 3 not fixed (1 high, 2 medium), tracked privately. npm test on this head: 212/212 pass.", + "expected_output": "An audit-pr body with Summary, What changes for reviewers and users, Changes by area, How to verify and Not fixed. Changes by area has one bullet or one '### ' heading per area, each naming its directory and test, and no '###' area is over 200 words. Everything outside Changes by area fits 1200 words and 160 lines, and the whole body stays under 3000 words. Not fixed gives counts only.", + "expectations": [ + "All five required headings are present: Summary, What changes for reviewers and users, Changes by area, How to verify, Not fixed", + "Each of the eight areas is its own bullet or '### ' heading under Changes by area, naming its directory (or package.json) and its test", + "No '###' area runs past 200 words; a long area is cut or split, not moved into the Summary", + "Everything outside Changes by area is within 1200 words and 160 lines, and the whole body is under 3000 words", + "What changes for reviewers and users lists the sign-in now needed for exports and uploads", + "Not fixed gives counts by severity only (1 high, 2 medium) with no file, route or exploit detail, and How to verify claims only the 212/212 npm test result" + ] } ] } diff --git a/.claude/skills/communication-structure/references/artifact-templates.md b/.claude/skills/communication-structure/references/artifact-templates.md index 2882dddef..106f2cb0c 100644 --- a/.claude/skills/communication-structure/references/artifact-templates.md +++ b/.claude/skills/communication-structure/references/artifact-templates.md @@ -43,7 +43,7 @@ verified and command. Fenced code does not count toward the budget. empty (items, or `None`), or `Not fixed` carries a file:line, route, URL or file path (counts only). A closing footer (`---` or `🤖 Generated`, then only blank, `Generated with` and `Co-Authored-By:` lines, any of them in ``, ``, ``, `*` or `_` markup) is not checked; other text after a `---` is. With `--range BASE..HEAD` it also fails when a changed file is not named under `Changes by area`, by full path -or by a containing directory with a trailing slash. The guard hook picks this kind when the `gh pr create|edit` title says audit, security +or by a containing directory with a trailing slash. The 1200-word, 160-line budget counts everything outside `Changes by area`, where an area may be a bullet or a `### ` heading; each `### ` area fails above 200 words and the whole body above 3000 words. The guard hook picks this kind when the `gh pr create|edit` title says audit, security or harden, and for `create` passes the diff against `--base` (default `origin/HEAD`). ## Templates diff --git a/.claude/skills/communication-structure/scripts/comms_lint.py b/.claude/skills/communication-structure/scripts/comms_lint.py index a3af9aac2..35522e7d0 100644 --- a/.claude/skills/communication-structure/scripts/comms_lint.py +++ b/.claude/skills/communication-structure/scripts/comms_lint.py @@ -5,7 +5,8 @@ (the one source of truth; `--list` prints it). Always checked: lead (line 1 is the answer: <=60 words, first sentence <=35, no greeting/background opener), budget (fails only above BUDGET_SLACK x the table value), dates (a weekday must fit its date, computed from the calendar), filler (AI-slop phrases), placeholder (TBD, [owner], and the -stand-ins "owner unassigned", "no deadline", "not stated"), vague (numberless "significantly", "on track"). Advisory +stand-ins "owner unassigned", "no deadline", "not stated"; a run of capital Xs inside a single-backtick span on one line, a mktemp +template such as `drain.XXXXXX`, is not one), vague (numberless "significantly", "on track"). Advisory WARNs that never put a draft below the bar: hedge, length (over the table value), markers (check/hourglass lines), numbers (a figure in the first 4 content lines), verified, timezone, and ask (runs only with --source, only when the source has an ask, and flags a missing owner or date only if the source gives one: nothing demands what the input @@ -19,7 +20,9 @@ `audit-pr` (the body of an audit, security or hardening PR) adds `sections`, a failing rule at any length: every AUDIT_SECTIONS heading is present, "What changes for reviewers and users" is not empty (items, or "None"), "Not fixed" holds counts only (no file:line, route or URL), and, with --range BASE..HEAD, every file the range changes is named -under "Changes by area" by its full path or by a containing directory with a trailing slash. A trailing attribution footer +under "Changes by area" by its full path or by a containing directory with a trailing slash. Its budget counts everything +outside "Changes by area", whose `### ` areas fail above AREA_WORDS words each and the whole body above TOTAL_WORDS (a hard +cap, no BUDGET_SLACK); the reported words and lines are the ones the budget counts. A trailing attribution footer (a `---` rule or `🤖 Generated` line, then only blank, `---`, `Generated with/by` and `Co-Authored-By:` lines; those lines may sit in ``, ``, ``, `*` or `_` markup) is not part of the last section; any other text after a `---` is still checked. @@ -52,6 +55,7 @@ BUDGET_SLACK = 1.5 # budget fails only above this multiple of the table value; between 1x and the slack it is a WARN (length) AUDIT_SECTIONS = ("Summary", "What changes for reviewers and users", "Changes by area", "How to verify", "Not fixed") # the one home of the audit-pr heading names; the template in artifact-templates.md spells them out _, S_BEHAVIOR, S_AREAS, _, S_NOT_FIXED = AUDIT_SECTIONS +AREA_WORDS, TOTAL_WORDS = 200, 3000 # audit-pr: the most words in one `### ` area under `## Changes by area` and in the whole body; both fail above, with no slack. The table budget covers everything outside that section. AUDIT_TITLE = re.compile(r"\b(?:audit\w*|security|harden\w*)\b", re.I) # a PR title the guard hook checks as `audit-pr` @@ -82,7 +86,7 @@ def load_kinds(path=REFERENCE): r"(?:vast|great) majority|majority of|mostly|nearly|almost all|on track|going well|(?:good|great|strong) progress|" r"a lot of|lots of|numerous|huge|massive|(?:major|large|big) improvements?|(?|\bowner unassigned\b|\bunassigned owner\b|" + r"<(?:owner|date|name|link|number|who|when|fill(?: in| me)?)>|\bowner unassigned\b|\bunassigned owner\b|" r"\bno deadline(?: given| set| stated)?\b|\bnot (?:stated|specified)\b|\bnone given\b", re.I) OPENER = re.compile(r"^(?:hi|hello|hey|dear|thanks?|thank you|hope|i (?:just |wanted|am writing|['’]m writing)|just\b|(?:a )?quick\b|" r"as (?:discussed|per|promised|requested)|following up|per our|so[, ]|" @@ -146,6 +150,13 @@ def _words(s): return len(s.split()) +def _no_templates(ln): + """`ln` with each run of 3+ capital Xs inside a single-backtick span on this line blanked: a mktemp template such as + `drain.XXXXXX` is not a placeholder. Anywhere else (prose, a double-backtick span, a span split across lines, an unclosed + backtick) the line is unchanged, so a lone XXX in prose still is one. One linear pass (no nested quantifier).""" + return re.sub(r"`[^`]*`", lambda m: re.sub(r"X{3,}", "_", m.group()), ln) + + def _skip(ln): """True for a line that carries no answer (heading, rule, greeting, subject, bare label).""" s = ln.strip() @@ -289,6 +300,24 @@ def _sections(text): return out +def _areas(text, prose): + """(inside, areas) for the audit-pr budget. `inside` = line numbers in the `## Changes by area` section; `areas` = + [(heading line, heading text, words)] for each `### ` heading in it; the words are those under it up to the next heading of + level 1-3 (prose only: the heading, blank and fenced lines count 0). Empty when the body has no such section, so the whole + body is then counted.""" + body = _sections(text).get(S_AREAS.lower(), (None, []))[1] + words, areas, fence, cur = dict(prose), [], False, None + for n, ln in body: + fence ^= ln.lstrip().startswith("```") + m = None if fence else HEADING.match(ln) + if m and len(m.group(1)) <= 3: + cur = [n, m.group(2).strip(), 0] if len(m.group(1)) == 3 else None + areas += [cur] if cur else [] + elif cur: + cur[2] += _words(words.get(n, "")) + return {n for n, _ in body}, [tuple(a) for a in areas] + + def _covers(area, path): """True when `area` names repo path `path` as a whole path (not part of a longer one) or a directory holding it, written with a trailing slash. Whole = not glued to a path character (a word character or one of @ + ~ / . -) on the left (a `./` @@ -346,7 +375,8 @@ def changed_files(rng, cwd=None): def lint(kind, text, source=None, files=None): """Score `text` as artifact `kind` (see load_kinds). Returns {kind, ok, score:[passed, total], words, lines, - budget:[words, lines], rules:[{rule, ok, advisory, hits:[{line, problem, fix}]}]}. Pure apart from reading the + budget:[words, lines], rules:[{rule, ok, advisory, hits:[{line, problem, fix}]}]}; `words` and `lines` are what the + budget counts: the whole body, but for audit-pr everything outside `## Changes by area` (see _areas). Pure apart from reading the table. `status` text with LONG_FORM_LABELS or more leadership labels is checked as `leadership`. `files` (paths changed by the PR, from changed_files) turns on the audit-pr coverage check; None skips it, [] has nothing to cover. Raises KeyError for a kind outside the table.""" @@ -389,15 +419,24 @@ def lint(kind, text, source=None, files=None): if _words(ln) > LEAD_WORDS or _words(first) > SENTENCE_WORDS: res["lead"].append(_hit(n, f"lead is {_words(ln)} words (first sentence {_words(first)}); the answer is buried", f"rewrite line {n} as one sentence of {SENTENCE_WORDS} words or fewer: outcome + number + ask; move the rest below it")) - over = f"{words} words / {len(prose)} lines against {max_words} / {max_lines} for {kind}" - cut = f"cut {max(words - max_words, 0)} words and {max(len(prose) - max_lines, 0)} lines: drop process narration and background, move detail behind one link; keep every fact the reader needs" - if words > max_words * BUDGET_SLACK or len(prose) > max_lines * BUDGET_SLACK: + inside, areas = _areas(text, prose) if "sections" in res else (set(), []) # audit-pr: the budget skips `## Changes by area`, which has caps of its own + counted = [(n, t) for n, t in prose if n not in inside] + cwords = sum(_words(t) for _, t in counted) + where = f" outside `## {S_AREAS}`" if inside else "" + over = f"{cwords} words / {len(counted)} lines{where} against {max_words} / {max_lines} for {kind}" + cut = f"cut {max(cwords - max_words, 0)} words and {max(len(counted) - max_lines, 0)} lines: drop process narration and background, move detail behind one link; keep every fact the reader needs" + if cwords > max_words * BUDGET_SLACK or len(counted) > max_lines * BUDGET_SLACK: res["budget"].append(_hit(None, over, cut)) - elif words > max_words or len(prose) > max_lines: + elif cwords > max_words or len(counted) > max_lines: res["length"] = [_hit(None, over, cut)] # advisory: a ceiling at BUDGET_SLACK x, so a needed fact is never cut for the target + res["budget"] += [_hit(n, f"area '{h}' is {w} words against {AREA_WORDS} for a `###` area", f"cut {w - AREA_WORDS} words, or split it into two `###` areas; move detail behind one link") + for n, h, w in areas if w > AREA_WORDS] + if "sections" in res and words > TOTAL_WORDS: + res["budget"].append(_hit(None, f"{words} words against the {TOTAL_WORDS} hard cap for {kind}", f"cut {words - TOTAL_WORDS} words: move detail behind one link")) res["dates"] = [_hit(n, f"'{m}' is a {day}", "check the date: fix the weekday or the date, or ask the author") for n, m, day in _bad_weekdays(prose)] + plain = [(n, _no_templates(ln)) for n, ln in prose] for rule, rx, fix in (("filler", FILLER, "delete the phrase"), ("placeholder", PLACEHOLDER, "delete it, or fill it from the source; leave an unknown part out instead of writing a stand-in")): - res[rule] = [_hit(n, f"'{rx.search(ln).group(0)}'", fix) for n, ln in prose if rx.search(ln)] + res[rule] = [_hit(n, f"'{rx.search(ln).group(0)}'", fix) for n, ln in plain if rx.search(ln)] for n, ln in prose: for sent in re.split(r"(?<=[.!?;])\s+", ln): m = VAGUE.search(sent) @@ -457,7 +496,7 @@ def lint(kind, text, source=None, files=None): "placeholder", "vague", "grounded", "hedge") rules = [{"rule": r, "ok": not res[r], "advisory": r in ADVISORY, "hits": res[r]} for r in order if r in res] return {"kind": kind, "ok": all(r["ok"] or r["advisory"] for r in rules), "score": [sum(r["ok"] for r in rules), len(rules)], - "words": words, "lines": len(prose), "budget": [max_words, max_lines], "rules": rules} + "words": cwords, "lines": len(counted), "budget": [max_words, max_lines], "rules": rules} def render(res): diff --git a/.claude/skills/deep-code-review/INDEX-scripts.md b/.claude/skills/deep-code-review/INDEX-scripts.md index 0c70b4663..42d5eb86b 100644 --- a/.claude/skills/deep-code-review/INDEX-scripts.md +++ b/.claude/skills/deep-code-review/INDEX-scripts.md @@ -4,11 +4,12 @@ Generated by `scripts/skill_index.py` (do not hand-edit). The script half of `IN | File | Trigger | CLI (— = open the header) | |---|---|---| -| `scripts/app_pentest.sh` | app_pentest.sh [--active] URL: ZAP baseline + nuclei over a RUNNING app you own | — | +| `scripts/app_pentest.sh` | app_pentest.sh [--active] [--openapi SPEC] URL: ZAP, nuclei, schemathesis over a RUNNING app you own | — | | `scripts/binaries_gate.py` | no git-tracked file at a banned image/media/archive/ build-output extension. | — | | `scripts/ci_cost_advisor.py` | estimate a repo's GitHub Actions minutes and dollars, and print a local-first plan with a ready pat… | ci_cost_advisor.py [ROOT] [--visibility auto\|public\|private] [--plan free\|pro\|team\|enterprise] [--required FILE[,FILE]\|… | | `scripts/ci_cost_lint.py` | flag GitHub Actions workflows that waste CI runner minutes. | — | | `scripts/closes_lint.py` | a closing keyword may only reference an issue/PR this change owns. | — | +| `scripts/committed_personal_data.py` | count distinct email addresses per tracked file and report the path and the count, never an address. | committed_personal_data.py [--threshold N] [--repo-threshold N] [--include-reserved] [--strict] [--stdin] [--max-bytes… | | `scripts/context_pack.py` | bounded context pack for a DIFF review. | context_pack.py [--base REF] [--repo DIR] [--intent FILE\|-] [--cap N] [--max-bytes N] | | `scripts/coverage_ledger.py` | completeness check for a machine-report coverage map. | coverage_ledger.py [--domains SPEC] | | `scripts/derived_numbers.py` | bind a doc number to the counter that derives it. | python3 derived_numbers.py [--fix] [--selftest] [ROOT] Exit codes: 0 ok (or fixed); 1 drift or bad marker; 2 usage or s… | diff --git a/.claude/skills/deep-code-review/INDEX.md b/.claude/skills/deep-code-review/INDEX.md index 8310c1aab..34c29abb0 100644 --- a/.claude/skills/deep-code-review/INDEX.md +++ b/.claude/skills/deep-code-review/INDEX.md @@ -13,7 +13,7 @@ Generated by `scripts/skill_index.py` (do not hand-edit; rerun it after changing | `references/a11y-live.md` | the target or diff renders a loading / skeleton / spinner region (a shared loader or a hand-rolled… | 1722 | | `references/api-contracts-streaming.md` | the target serves or consumes a WebSocket, SSE or gRPC-streaming endpoint (server, client or SDK wr… | 1852 | | `references/api-contracts.md` | reviewing public HTTP/GraphQL/RPC surfaces, webhooks, message/ queue payloads, SDK boundaries, or a… | 5364 | -| `references/app-pentest.md` | the target app is running (local or an owned staging host) and you want a black-box pass with prove… | 758 | +| `references/app-pentest.md` | the target app is running (local or an owned staging host) and you want a black-box pass with prove… | 1586 | | `references/appsec-approvals.md` | the target or diff performs a consequential or high-value action: a payout, wire transfer, payout-d… | 976 | | `references/appsec-corroboration.md` | a finding claims that input reaches a sink, that a secret is live, that a dependency version is vul… | 650 | | `references/appsec-crypto.md` | the target or diff encrypts or decrypts (cipher mode, IV, nonce), compares a signature, token, API… | 2675 | @@ -61,7 +61,7 @@ Generated by `scripts/skill_index.py` (do not hand-edit; rerun it after changing | `references/domain-h.md` | domain H (Tech debt, dead code & maintainability) is applicable in Phase 2 — the coverage ledger ma… | 1951 | | `references/domain-i.md` | domain I (API, interface, contracts & integration) is applicable in Phase 2 — the coverage ledger m… | 364 | | `references/domain-j.md` | domain J (Testing & evaluation) is applicable in Phase 2 — the coverage ledger marks it, or a DIFF… | 705 | -| `references/domain-k.md` | domain K (Build, CI/CD, supply chain & release) is applicable in Phase 2 — the coverage ledger mark… | 1297 | +| `references/domain-k.md` | domain K (Build, CI/CD, supply chain & release) is applicable in Phase 2 — the coverage ledger mark… | 1418 | | `references/domain-l.md` | domain L (Infrastructure as code, containers & cloud) is applicable in Phase 2 — the coverage ledge… | 295 | | `references/domain-m.md` | domain M (Observability: logs, metrics, traces) is applicable in Phase 2 — the coverage ledger mark… | 298 | | `references/domain-n.md` | domain N (Configuration, secrets & environments) is applicable in Phase 2 — the coverage ledger mar… | 214 | @@ -98,7 +98,7 @@ Generated by `scripts/skill_index.py` (do not hand-edit; rerun it after changing | `references/method-diff.md` | the review scope is a DIFF or FILE (a large or blast-radius-flagged DIFF, a fix that is itself the… | 1968 | | `references/method-lenses.md` | method.md routes here: a call site may bypass or re-derive a shared helper, component or predicate,… | 1935 | | `references/method-report.md` | writing the Phase 5 report (delivery shape, out-of-tree vs in-repo, ledger and claim reconciliation… | 2292 | -| `references/method-situational.md` | method.md routes here (its "Situational Phase 1 checks" bullet and its perceived-absent-premise par… | 10966 | +| `references/method-situational.md` | method.md routes here (its "Situational Phase 1 checks" bullet and its perceived-absent-premise par… | 11004 | | `references/method.md` | executing a review: after stating SCOPE / START_SHA / COVERAGE_LEDGER, work the phases in order. | 7484 | | `references/migration-parity.md` | the reference you are matching is a prototype / mockup / design export (a static page, a design-too… | 7610 | | `references/mobile-appsec.md` | the target is (or ships) a native mobile client — an iOS (Swift / Objective-C) or Android (Kotlin /… | 6478 | @@ -130,7 +130,7 @@ Generated by `scripts/skill_index.py` (do not hand-edit; rerun it after changing | `references/skill-authoring-and-size.md` | the target repository ships or installs agent skills (/SKILL.md, .claude/skills/, .cursor/skills/,… | 991 | | `references/small-diff-read-budget.md` | reviewing a small DIFF (the fast path in SKILL.md), before the first read or command. | 131 | | `references/standards-index.md` | citing a standard URL or verification date from an installed copy of the skill. | 117 | -| `references/supply-chain-check.md` | gating a deploy on dependency advisories, hardening a repo's GitHub Actions, or adding code scannin… | 855 | +| `references/supply-chain-check.md` | gating a deploy on dependency advisories, hardening a repo's GitHub Actions, or adding code scannin… | 911 | | `references/testing-ai-evals.md` | the target calls an LLM, retrieves for generation (RAG), or runs a tool-using agent. | 2564 | | `references/testing-and-evals.md` | Read this to judge whether the target is *actually* verified — not whether a coverage number is hig… | 3667 | | `references/testing-ml.md` | the target trains or serves a classical ML model, makes consequential model decisions about people,… | 2743 | @@ -152,4 +152,4 @@ Generated by `scripts/skill_index.py` (do not hand-edit; rerun it after changing ## Scripts -35 scripts. Read `INDEX-scripts.md` when you need a script (a trigger and CLI per script). `../INDEX.md`, `### deep-code-review`, has only each one-line purpose and arguments. +36 scripts. Read `INDEX-scripts.md` when you need a script (a trigger and CLI per script). `../INDEX.md`, `### deep-code-review`, has only each one-line purpose and arguments. diff --git a/.claude/skills/deep-code-review/evals/evals.json b/.claude/skills/deep-code-review/evals/evals.json index 16577feba..43434a305 100644 --- a/.claude/skills/deep-code-review/evals/evals.json +++ b/.claude/skills/deep-code-review/evals/evals.json @@ -8430,6 +8430,17 @@ "Treats a skipped mode as unreviewed surface, never as a clean result", "Puts the coverage line in the hand-back itself, not only in a long report" ] + }, + { + "id": "ci-only-fix-where-actions-is-disabled-confirm-workflows-run", + "prompt": "A review finds that nothing in the gate scans for committed secrets. The repo has .github/workflows/ci.yml (lint and unit tests on pull_request) and a .githooks/pre-commit that runs the same lint and tests. The obvious fix is a new gitleaks step in ci.yml. Before you file it you notice that the last ten merged PRs show no checks at all, and the repository settings say Actions is disabled for this repository. What do you recommend, and what do you say about the existing CI?", + "expected_output": "Does not file 'add a gitleaks step to ci.yml' as the fix. A workflow file is not coverage until it runs, so it confirms liveness instead of inferring it from the file: `gh api repos///actions/permissions --jq .enabled` (expect false) and `gh run list -L 5` (expect empty), and reports that Actions is off: ci.yml never runs, so its lint and tests are not CI coverage and the pre-commit hook is the real gate. Aims the secret scan at the gate that does run (a step in .githooks/pre-commit), or flags re-enabling Actions as an owner decision (shared repo state, runner cost, policy). A ci.yml step may be offered only as one that runs once Actions is enabled, never as a running gate. Rule: references/domain-k.md.", + "expectations": [ + "Confirms workflows run (gh api repos///actions/permissions --jq .enabled, gh run list -L 5) instead of inferring CI coverage from the ci.yml file", + "Says Actions is off so ci.yml never runs, and does not count its lint and tests as CI coverage", + "Aims the fix at the gate that does run (the pre-commit hook) or flags re-enabling Actions as an owner decision", + "Does not present the ci.yml step as a running gate; at most as one that runs once Actions is enabled" + ] } ] } diff --git a/.claude/skills/deep-code-review/references/app-pentest.md b/.claude/skills/deep-code-review/references/app-pentest.md index 052d9b5e1..b87036edd 100644 --- a/.claude/skills/deep-code-review/references/app-pentest.md +++ b/.claude/skills/deep-code-review/references/app-pentest.md @@ -2,7 +2,7 @@ Read this when the target app is running (local or an owned staging host) and yo ## Run it -`bash scripts/app_pentest.sh [--active] URL` (beside the skill's other scripts; the flag comes first). It runs OWASP ZAP baseline (`zap-baseline.py`, passive spider) and nuclei (default templates). A tool that is not installed is reported `skipped`, a failed one `error`. Install nothing on the host for this: use the project's image or CI, or report the skip. ZAP must be a native or in-image install on PATH: the script passes an absolute report path under its own temp dir, which a docker wrapper that only mounts `/zap/wrk` would not see. +`bash scripts/app_pentest.sh [--active] [--openapi SPEC] URL` (beside the skill's other scripts; the flags come first). It runs OWASP ZAP baseline (`zap-baseline.py`, passive spider) and nuclei (default templates); `--openapi` adds schemathesis API fuzzing (see below). A tool that is not installed is reported `skipped`, a failed one `error`. Install nothing on the host for this: use the project's image or CI, or report the skip. ZAP must be a native or in-image install on PATH: the script passes an absolute report path under its own temp dir, which a docker wrapper that only mounts `/zap/wrk` would not see. ## Safety rules (the script enforces them; do not route around them) @@ -10,13 +10,30 @@ Read this when the target app is running (local or an owned staging host) and yo - The resolved addresses are printed in the JSON (`ips`). Check them: an allowlisted name can point at a machine that is not yours. - Passive by default. nuclei runs `-rl 10 -c 5`, without interactsh (`-ni`, no callbacks to third-party servers), following redirects to the same host only (`-fhr`), with dos/fuzz/bruteforce/intrusive tags excluded. - `--active` swaps in `zap-full-scan.py` (throttled: `-T 10`, `threadPerHost=2`, `delayInMs=100`; confirm those keys against your ZAP version) and lifts the nuclei tag exclusion. It requires the host in `APP_PENTEST_ALLOW` even for localhost, and every resolved address loopback or private. Active scanning writes data and can break a service: use a disposable instance, never one holding real data. nuclei keeps its `-rl 10` cap. +- `--openapi SPEC` is a local file or an http(s) URL; a URL must pass every target rule above and its addresses print as `openapi.ips`. The spec is read once, before any tool starts (no redirects, no env proxies, 10 s timeout, 10 MB cap); schemathesis gets that copy, never the URL, because its own fetch follows redirects. An unreadable or over-size spec, or one with an absolute or scheme-relative `$ref` or `externalValue`, exits 2: inline it. That check is a text match that JSON escapes and YAML tags get past; the egress block below is the real guard. A spec schemathesis cannot load still ends as `error`. - An allowlist entry is the owner's statement that the host is theirs to test. Never point this at a third-party or production system. +## API fuzzing (`--openapi`) + +Needs schemathesis 4.x as `schemathesis` on PATH or `~/.local/bin/schemathesis`; no other name is run. Flags follow `schemathesis run --help` (v4 renamed several): re-read it after an upgrade. + +- `--origin` pins requests to the validated target origin; the schema's base path (`servers` or `basePath`) is appended, the target URL's path is not. +- Both modes: `--phases examples,fuzzing --rate-limit 5/s -n 10 -w 1 --no-shrink --max-redirects 0 --request-timeout 10`. The default coverage phase ignores `-n` and probes undeclared methods such as TRACE, so it is off; `--max-time` is unused because fuzzing then repeats until the time is spent. Run time grows with the operation count (about 5 requests per second). +- An empty `--config-file` is pinned: a `schemathesis.toml` above the working directory cannot change the run. +- Egress block: every proxy variable (`http_proxy`, `https_proxy`, `all_proxy`, both cases) is set to a closed loopback port and `no_proxy` to the target host alone. Any other host a spec points at (a `$ref` or `externalValue` the text match missed) cannot be fetched, target traffic goes direct, and the caller's own proxy cannot divert it. Residual: the same host on another port, its subdomains, and any fetcher that ignores proxy variables. +- Passive adds `--include-method GET --include-method HEAD`: nothing that writes is sent. `--active` drops the filter (same rate and budget) and runs only for a listed loopback or private host. + +Status: `skipped` (no `--openapi` or no schemathesis), `ran` (the report parses, is complete and tested at least one operation; `openapi.errored` above 0 is partial coverage), `error` (anything else: unreachable target, untrusted TLS certificate, unreadable report; rerun `schemathesis run` by hand to see why). Exit code 1 only means failures were found. + +Each failure becomes one lead per code and operation (`id` is the code, `title` is `: `). Severity: medium for `ST101` server error (5xx), `ST102` 5xx on an undeclared Content-Type, `ST401` authentication not enforced, and `server_unavailable` (not a schemathesis code: the server stopped answering mid-run); low for every other code (contract violations such as `ST103` undocumented status code, and codes a later release adds). + +Limits: requests are unauthenticated (no `--auth` or `--header`), TLS verification stays on (an https target needs a certificate this machine trusts), and the stateful phase is not run. + ## Leads, not findings -Output is JSON: `{"target","ips","active","tools":{"zap","nuclei"},"leads":[{"tool","id","title","severity","url","verified":false}],"note"}`. Every lead is a hint. Before it reaches a report: +Output is JSON: `{"target","ips","active","tools":{"zap","nuclei","schemathesis"},"openapi":null|{"spec","ips","tested","errored"},"leads":[{"tool","id","title","severity","url","verified":false}],"note"}`. Every lead is a hint. Before it reaches a report: 1. Reproduce it by hand against the same instance (request, response, observed impact). A header or version banner matched by a template is not an exploit. 2. Map it to source (`path:line`) and apply `appsec-verify.md` section 1: no source-to-sink path, no finding; an unreproduced lead becomes `❓`. 3. Drop template noise (informational banners, missing optional headers on a dev server) under `appsec-verify.md` section 2. -4. A clean scan proves nothing: baseline mode only sees what a passive spider reaches. Say so; do not write "no vulnerabilities found". +4. A clean scan proves nothing: baseline mode only sees what a passive spider reaches, and schemathesis only what the spec declares, unauthenticated. Say so; do not write "no vulnerabilities found". diff --git a/.claude/skills/deep-code-review/references/domain-k.md b/.claude/skills/deep-code-review/references/domain-k.md index c30aa6dd8..875644ff8 100644 --- a/.claude/skills/deep-code-review/references/domain-k.md +++ b/.claude/skills/deep-code-review/references/domain-k.md @@ -8,6 +8,12 @@ lifecycle, canary/blue-green claims vs actual config, DORA-or-`UNMEASURED` — `references/release-engineering.md`. - One-command reproducible build; lockfiles committed and honored; CI gates merge on lint + format + type + tests + security/dependency scan. +- **A workflow file is not coverage until it runs.** Before claiming CI + coverage or recommending a CI-only fix, confirm workflows fire: + `gh api repos///actions/permissions --jq .enabled`, + `gh run list -L 5` (`method.md` "Read the host CI"). Off or unknown: say so; + aim the fix at the gate that does run (a local hook) or flag re-enabling + Actions as an owner decision. Never present a CI-only fix as running. - **A new or changed gate ships with its measured false-positive rate on HEAD** in the PR description (target ≤10%), not only a description of what it detects. Sound-looking detection logic can still fire mostly on non-defects @@ -66,4 +72,5 @@ lifecycle, canary/blue-green claims vs actual config, DORA-or-`UNMEASURED` — dependency scan, non-reproducible build, deploy without ownership check, a known-vulnerable or **EOL** dependency/runtime/base image shipping, a single "update all dependencies" commit with no per-dep test evidence, no update-bot - config (`dependabot.yml`/`renovate.json`) beside a long tail of outdated deps. + config (`dependabot.yml`/`renovate.json`) beside a long tail of outdated deps, + a CI-only fix where Actions is off or no run exists. diff --git a/.claude/skills/deep-code-review/references/method-situational.md b/.claude/skills/deep-code-review/references/method-situational.md index 8ffb057e3..012e72aed 100644 --- a/.claude/skills/deep-code-review/references/method-situational.md +++ b/.claude/skills/deep-code-review/references/method-situational.md @@ -5,7 +5,7 @@ Read this when `method.md` routes here (its "Situational Phase 1 checks" bullet **Phase 1 — gate disputes and target-shape preflights.** - **On a `DIFF`, run `scripts/review_checks.sh --base ` first.** It runs only the installed local - analyzers (shellcheck, `bash -n`, py_compile, ruff/pyflakes, `node --check`, tsc, `go vet`, jq, `invisible-chars` (NUL/C0, bidi, zero-width), plus + analyzers (shellcheck, `bash -n`, py_compile, ruff/pyflakes, `node --check`, tsc, `go vet`, jq, `invisible-chars` (NUL/C0, bidi, zero-width), `committed-data` (10+ distinct non-reserved emails in a file, count only; a `.` row for 50+ spread thin; `not-checked` for compressed or columnar files), plus gitleaks, semgrep with the repo's rules or else the bundled taint rules plus p/default (`REVIEW_SEMGREP_REGISTRY=0` drops the registry pack), trivy and osv-scanner offline, osv online with `REVIEW_OSV=1`) on the changed files and prints findings JSON; `--history` scans commit history, `--tests` adds the declared `REVIEW_TEST_CMD`; stderr ends with a one-line ran/skipped summary. Its findings are leads: verify each against the code before reporting, report a `not run` line or `not-checked` row as unchecked surface, and never read an empty diff --git a/.claude/skills/deep-code-review/references/supply-chain-check.md b/.claude/skills/deep-code-review/references/supply-chain-check.md index 74667aec6..e881de785 100644 --- a/.claude/skills/deep-code-review/references/supply-chain-check.md +++ b/.claude/skills/deep-code-review/references/supply-chain-check.md @@ -17,7 +17,7 @@ silent pass: it prints `SKIPPED : ` and the run exits 3 (INCOMPLET | Dependency advisories on lockfiles | `osv-scanner` | **Opt-in** (`--online`): queries `api.osv.dev` | fails at `--fail-on` (default `high`); prints counts and fixed versions | | Workflow security (injection, unpinned or over-permissioned actions) | `zizmor` | none | any finding fails | | Workflow syntax and shell | `actionlint` | none | any finding fails | -| Repo posture (branch protection, pinned deps, review) | `scorecard` (OpenSSF) | queries GitHub; `--scorecard owner/repo`, public repos | informational scores, no threshold | +| Repo posture (branch protection, pinned deps, review) | `scorecard` (OpenSSF) | none for a git DIR: offline subset on an export of HEAD, the whole repo even for a subdirectory DIR (tracked files only; Vulnerabilities and Fuzzing never run); `--scorecard owner/repo` queries GitHub, public repos | informational scores, no threshold; the offline view is partial, not comparable with a public score | Pre-deploy: `--online --fail-on high`; block the release on any non-zero exit. osv-scanner exit 128 (no package sources found) is reported as a failure, not confirmed as "nothing to scan". diff --git a/.claude/skills/deep-code-review/scripts/app_pentest.sh b/.claude/skills/deep-code-review/scripts/app_pentest.sh index 6ec3c583f..289043ca6 100755 --- a/.claude/skills/deep-code-review/scripts/app_pentest.sh +++ b/.claude/skills/deep-code-review/scripts/app_pentest.sh @@ -1,31 +1,42 @@ #!/usr/bin/env bash -# app_pentest.sh [--active] URL: ZAP baseline + nuclei over a RUNNING app you own; prints JSON leads. +# app_pentest.sh [--active] [--openapi SPEC] URL: ZAP, nuclei, schemathesis over a RUNNING app you own; prints JSON leads. # Invoke with bash. Safety rules, flags and how to treat the output: references/app-pentest.md (single source). # Exit: 0 ok, 2 refused/usage. set -u -active=0 -[ "${1:-}" = "--active" ] && { active=1; shift; } -url="${1:-}" refuse() { echo "REFUSED: $*" >&2; exit 2; } -[ $# -eq 1 ] || refuse "usage: app_pentest.sh [--active] URL" -case "$url" in http://*|https://*) ;; *) refuse "URL must start with http:// or https://" ;; esac -auth="${url#*://}"; auth="${auth%%[/?#]*}" -case "$auth" in *@*) refuse "userinfo (@) in URL" ;; esac -host="${auth%%:*}"; port="" -case "$auth" in *:*) port="${auth#*:}" ;; esac -host="$(printf '%s' "$host" | tr 'A-Z' 'a-z')" -[[ "$host" =~ ^[a-z0-9]([a-z0-9.-]*[a-z0-9])?$ ]] || refuse "bad host '$host'" -[[ "$port" =~ ^([0-9]{1,5})?$ ]] || refuse "bad port '$port'" -listed=0 -IFS=',' read -r -a allow <<< "${APP_PENTEST_ALLOW:-}" -for h in ${allow[@]+"${allow[@]}"}; do - h="$(printf '%s' "$h" | tr -d ' ' | tr 'A-Z' 'a-z')" - [ -n "$h" ] && [ "$h" = "$host" ] && listed=1 +active=0; spec="" +while :; do + case "${1:-}" in + --active) active=1; shift ;; + --openapi) [ -n "${2:-}" ] || refuse "--openapi needs a spec file or URL"; spec="$2"; shift 2 ;; + *) break ;; + esac done -[ "$host" = localhost ] || [ "$host" = 127.0.0.1 ] || [ $listed -eq 1 ] || refuse "host '$host' is not localhost, 127.0.0.1 or in APP_PENTEST_ALLOW" -[ $active -eq 0 ] || [ $listed -eq 1 ] || refuse "--active needs host '$host' listed in APP_PENTEST_ALLOW" -# resolve; fail closed. Prints "ip ip ..." and exits 1 if --active and any address is not loopback/private. -ips="$(python3 -I - "$host" "$active" <<'PY' +url="${1:-}" +[ $# -eq 1 ] || refuse "usage: app_pentest.sh [--active] [--openapi SPEC] URL" + +# check_url LABEL URL: the allow rules for one URL (the target, and a spec URL with LABEL "--openapi: "). Sets c_ips +# (resolved addresses), c_origin (scheme://host[:port]) and c_host. Refuses (exit 2) on any violation. +check_url() { + local l="$1" u="$2" auth host port h rc listed=0 + local -a allow + case "$u" in http://*|https://*) ;; *) refuse "${l}URL must start with http:// or https://" ;; esac + auth="${u#*://}"; auth="${auth%%[/?#]*}" + case "$auth" in *@*) refuse "${l}userinfo (@) in URL" ;; esac + host="${auth%%:*}"; port="" + case "$auth" in *:*) port="${auth#*:}" ;; esac + host="$(printf '%s' "$host" | tr 'A-Z' 'a-z')" + [[ "$host" =~ ^[a-z0-9]([a-z0-9.-]*[a-z0-9])?$ ]] || refuse "${l}bad host '$host'" + [[ "$port" =~ ^([0-9]{1,5})?$ ]] || refuse "${l}bad port '$port'" + IFS=',' read -r -a allow <<< "${APP_PENTEST_ALLOW:-}" + for h in ${allow[@]+"${allow[@]}"}; do + h="$(printf '%s' "$h" | tr -d ' ' | tr 'A-Z' 'a-z')" + [ -n "$h" ] && [ "$h" = "$host" ] && listed=1 + done + [ "$host" = localhost ] || [ "$host" = 127.0.0.1 ] || [ $listed -eq 1 ] || refuse "${l}host '$host' is not localhost, 127.0.0.1 or in APP_PENTEST_ALLOW" + [ $active -eq 0 ] || [ $listed -eq 1 ] || refuse "${l}--active needs host '$host' listed in APP_PENTEST_ALLOW" + # resolve; fail closed. Prints "ip ip ..." and exits 1 if --active and any address is not loopback/private. + c_ips="$(python3 -I - "$host" "$active" <<'PY' import ipaddress, socket, sys try: ips = sorted({a[4][0] for a in socket.getaddrinfo(sys.argv[1], None)}) @@ -36,10 +47,47 @@ if sys.argv[2] == "1" and not all(ipaddress.ip_address(i.split("%")[0]).is_loopb sys.exit(1) PY )"; rc=$? -[ $rc -eq 0 ] || { [ $rc -eq 1 ] && refuse "--active needs loopback/private addresses, '$host' resolves to $ips"; refuse "cannot resolve '$host'"; } + [ $rc -eq 0 ] || { [ $rc -eq 1 ] && refuse "${l}--active needs loopback/private addresses, '$host' resolves to $c_ips"; refuse "${l}cannot resolve '$host'"; } + c_origin="${u%%://*}://$host${port:+:$port}"; c_host="$host" +} +check_url "" "$url"; ips="$c_ips"; origin="$c_origin"; thost="$c_host" +sips=""; loc="" +if [ -n "$spec" ]; then + case "$spec" in + http://*|https://*) check_url "--openapi: " "$spec"; sips="$c_ips" ;; + *://*) refuse "--openapi: a spec URL must start with http:// or https://" ;; + /*) [ -f "$spec" ] || refuse "--openapi: '$spec' is not a file"; loc="$spec" ;; + *) [ -f "$spec" ] || refuse "--openapi: '$spec' is not a file"; loc="$PWD/$spec" ;; + esac +fi out="$(mktemp -d "${TMPDIR:-/tmp}/ap.XXXXXX")" || exit 1; trap 'rm -rf "$out"' EXIT -st_zap=skipped; st_nuc=skipped +if [ -n "$spec" ]; then + # Read the spec before any tool starts: an unreadable or over-size spec, or one with a remote $ref or externalValue + # (an early text match; JSON escapes and YAML tags get past it, the egress block on the run is the real guard), + # refuses here. A spec schemathesis cannot load still ends as status error. A spec URL is fetched once, without + # redirects or env proxies, and schemathesis gets the copy: its own fetch follows redirects. + python3 -I - "${loc:-$spec}" "$out/spec" >"$out/msg" <<'PY' +import re, sys, urllib.request +src, dst = sys.argv[1:3] +class NoRedirect(urllib.request.HTTPRedirectHandler): + def redirect_request(self, *a): return None +def die(m): print(m); sys.exit(1) +url = src.startswith(("http://", "https://")) +try: + if url: + with urllib.request.build_opener(NoRedirect, urllib.request.ProxyHandler({})).open(src, timeout=10) as r: data = r.read(10**7 + 1) + else: + with open(src, "rb") as f: data = f.read(10**7 + 1) +except Exception as e: die("cannot read spec: %s" % e) +if len(data) > 10**7: die("spec is over 10 MB") +if re.search(rb'(?:\$ref|externalValue)["\']?\s*:\s*["\']?(//|[A-Za-z][A-Za-z0-9+.-]*:)', data): die("spec has a remote $ref or externalValue; inline it") +if url: open(dst, "wb").write(data) +PY + [ $? -eq 0 ] || refuse "--openapi: $(cat "$out/msg")" + [ -n "$loc" ] || loc="$out/spec" +fi +st_zap=skipped; st_nuc=skipped; st_st=skipped zap=zap-baseline.py; zargs=(-t "$url" -I -J "$out/zap.json") if [ $active -eq 1 ]; then zap=zap-full-scan.py @@ -53,9 +101,26 @@ if command -v nuclei >/dev/null 2>&1; then [ $active -eq 1 ] || nargs+=(-etags dos,fuzz,bruteforce,intrusive) if nuclei "${nargs[@]}" >"$out/nuclei.log" 2>&1; then st_nuc=ran; else st_nuc=error; fi fi -python3 -I - "$url" "$active" "$st_zap" "$st_nuc" "$out" "$ips" <<'PY' +if [ -n "$spec" ]; then + stbin="$(command -v schemathesis 2>/dev/null || command -v "${HOME:-}/.local/bin/schemathesis" 2>/dev/null)" + if [ -n "$stbin" ]; then + # Empty config pinned: schemathesis would otherwise load any schemathesis.toml found above the cwd. + : >"$out/st.toml" + sargs=(--config-file "$out/st.toml" run "$loc" --origin "$origin" --phases examples,fuzzing --rate-limit 5/s -n 10 -w 1 + --no-shrink --max-redirects 0 --request-timeout 10 --report json --report-json-path "$out/st.json") + [ $active -eq 1 ] || sargs+=(--include-method GET --include-method HEAD) + # Egress block: every proxy variable is a closed loopback port and no_proxy is the target host alone, so a fetch of + # any other host (a remote $ref or externalValue the text match missed) fails, target traffic goes direct, and the + # caller's own proxy cannot divert it. Residual: the same host on another port, its subdomains, and a fetcher that ignores proxies. + dead=http://127.0.0.1:1 + (cd "$out" && env http_proxy=$dead HTTP_PROXY=$dead https_proxy=$dead HTTPS_PROXY=$dead all_proxy=$dead ALL_PROXY=$dead \ + no_proxy="$thost" NO_PROXY="$thost" "$stbin" "${sargs[@]}" >st.log 2>&1) + st_st=attempted # exit 1 means failures found; the report below decides ran or error + fi +fi +python3 -I - "$url" "$active" "$st_zap" "$st_nuc" "$out" "$ips" "$st_st" "$spec" "$sips" <<'PY' import json, os, sys -url, active, zs, ns, out, ips = sys.argv[1:7] +url, active, zs, ns, out, ips, ss, spec, sips = sys.argv[1:10] leads = [] def load(name): try: return open(os.path.join(out, name)).read() @@ -76,6 +141,27 @@ for line in load("nuclei.jsonl").splitlines(): i = r.get("info") or {} leads.append({"tool": "nuclei", "id": r.get("template-id", ""), "title": i.get("name", ""), "severity": i.get("severity", "unknown"), "url": r.get("matched-at", url), "verified": False}) -print(json.dumps({"target": url, "ips": ips.split(), "active": active == "1", "tools": {"zap": zs, "nuclei": ns}, "leads": leads, +# schemathesis: one lead per failure code and operation. 5xx and unenforced auth are medium, everything else +# (schema/contract/validation codes, and any code added later) is low. Ran = the report parses, is complete and +# tested at least one operation; an unreachable target or an unreadable spec tests nothing and is an error. +tested = errored = None +if ss == "attempted": + ss = "error" + try: + r = json.loads(load("st.json")) + for f in r.get("failures") or []: + for op in f.get("operations") or [""]: + leads.append({"tool": "schemathesis", "id": f["code"], "title": f["title"] + (": " + op if op else ""), + "severity": "medium" if f["code"] in ("ST101", "ST102", "ST401") else "low", "url": url, "verified": False}) + if r.get("stop_reason") == "server_unavailable": + leads.append({"tool": "schemathesis", "id": "server_unavailable", "title": "Server stopped responding during the run", + "severity": "medium", "url": url, "verified": False}) + tested, errored = r["operations"]["tested"], r["operations"].get("errored") + if r["complete"] and tested >= 1: ss = "ran" + except (ValueError, KeyError, TypeError, AttributeError): + pass +print(json.dumps({"target": url, "ips": ips.split(), "active": active == "1", + "tools": {"zap": zs, "nuclei": ns, "schemathesis": ss}, + "openapi": {"spec": spec, "ips": sips.split(), "tested": tested, "errored": errored} if spec else None, "leads": leads, "note": "LEADS only: reproduce each by hand (request, response, impact) before reporting."}, indent=2)) PY diff --git a/.claude/skills/deep-code-review/scripts/committed_personal_data.py b/.claude/skills/deep-code-review/scripts/committed_personal_data.py new file mode 100644 index 000000000..1fa5631b1 --- /dev/null +++ b/.claude/skills/deep-code-review/scripts/committed_personal_data.py @@ -0,0 +1,248 @@ +#!/usr/bin/env python3 +"""committed_personal_data.py [--threshold N] [--repo-threshold N] [--include-reserved] [--strict] [--stdin] [--max-bytes N] [ROOT] — count distinct email addresses per tracked file and report the path and the count, never an address. + +WHY THIS EXISTS +--------------- +gitleaks, trufflehog and semgrep look for secrets. A committed export of real people's data (an arrow, parquet or csv report holding +hundreds of customer addresses) holds no secret, so every one of them passes it clean. This check counts instead of matching rules: +for each tracked file it counts the distinct email-shaped strings whose domain is not a reserved example or test name, and reports a +file whose count reaches a threshold. It is a lead, not a verdict: open the file locally and decide whether the people published the +data on purpose. + +WHAT COUNTS +----------- +Files: every path of `git -C ROOT ls-files -z` (default ROOT is `.`), or with `--stdin` the newline-separated paths on stdin, relative +to ROOT (review_checks.sh passes the files it scans). Symlinks, directories (submodules) and paths missing on disk (a sparse +checkout) are skipped. + +Bytes: up to `--max-bytes` (default 32 MiB, at most 1 GiB) of each file is read raw, with no decoding, so a text export and the plain +string bytes of a binary one (sqlite, pickles) are scanned the same way. A larger file is scanned over its first `--max-bytes` only: its +row says so when it is flagged, and when it is not it gets a `not-checked` row, so it never reads as a clean file. A file in a +recognised compressed or columnar format (see KNOWN CEILINGS) that stays under the threshold gets a `not-checked` row too. + +Vendored paths: this script scans every path it is given and skips none. review_checks.sh drops vendored, build and worktree +directories (node_modules, vendor, dist, build, third_party, Pods, bower_components and so on) and *.min.js before it calls the script, +so run on its own it also reads those. + +Address: ASCII `local@domain`; the local part is up to 64 characters of letters, digits and `._%+-`, the domain is up to 256 bytes of +dot-joined labels ending in a 2-24 letter label. Distinct means case-insensitive. Counting stops at 100000 distinct addresses per file. + +Dropped unless `--include-reserved`: the reserved example and test names (example.com, example.net and example.org with their +subdomains, and the .test, .example, .invalid and .localhost top-level names), noreply forms (the word `noreply` or `donotreply`, +also written `no-reply`, `no_reply`, `do.not.reply` and so on, as a whole word of the local part or of a domain label, such as +users.noreply.github.com), the ssh account `git@` of git hosts (`git@github.com:org/repo.git`, `ssh://git@host/org/repo`: a local part of +exactly `git`) and image asset-scale names (`icon@2x.png`, `icon@1.5x.png`, `icon@2x-dark.png`: `@`, digits with an optional decimal +part, `x`, an optional `-word`, then one of png, jpg, jpeg, gif, webp, avif, svg, pdf, ico, bmp, tif, tiff, heic, heif or apng). +`--include-reserved` counts them too, so tests can plant fictional example.com fixtures. + +Threshold (default 10, `--threshold N`): a file is reported at or above N distinct addresses. Prose and manifests (README, SECURITY, +CODEOWNERS, package.json) seldom name ten different mailboxes, and a data export names dozens to thousands. Random or compressed +bytes are no noise at that level: about 0.05 email-shaped strings per MB were measured (the unit tests scan 24 MiB of random bytes). A +contributor list (AUTHORS, MAINTAINERS, .mailmap) reaches 10 by design and is reported as a lead too; whether those people agreed to +publish is the reviewer's call. + +Repository total (default 50, `--repo-threshold N`): addresses spread thin, one or two in each of dozens of per-person or per-record +files, never reach the file threshold. So the distinct addresses of all the files that stay below it are also counted together, and the +total is reported when it reaches N. A file already reported is left out, so no address is counted twice, and an address that appears in +many files counts once. 50 is five times the file threshold: a leak of per-record files runs to hundreds of addresses (the audit behind +this check found about 850), while a small repository seldom scatters 50 different mailboxes through its docs and headers. A large one +may, and then gets one lead row. `--threshold 1` lists the files behind the total. + +OUTPUT +------ +One TSV row per file on stdout, flushed per row: `path0rulemessage`, the row shape invisible_chars.py prints, so +review_checks.sh adds each as a finding. Rule `email-addresses` carries the count ("12 distinct email addresses (threshold 10)"), and +once for the repository total under the path `.` ("69 distinct email addresses across 75 file(s) ..."); rule `not-checked` is surface +this scan did not cover (an unreadable file, one over the size cap, or a compressed/columnar container). Never an address, a domain or +a line of a file: only the path and the count (or the container format). Silent when there is nothing to report. Read-only; stdlib only. + +EXIT CODES (fail closed) +------------------------ + 0 advisory: finished, whatever it found (with `--strict`, nothing was flagged). + 1 `--strict` and at least one file, or the repository total, was flagged (`not-checked` rows alone do not count). + 2 bad usage, or `git ls-files` failed (not a repository): never a silent pass. + +KNOWN CEILINGS +-------------- +Addresses inside compressed streams are not reliably seen, and a parquet or Arrow file also stores its values in pages and buffers, so a +neighbouring value's tail can stick to an address (a reserved domain may then read as a real one, and a repeated value may count more +than once): a low count proves nothing about such a file. A file in one of the formats below that stays under the threshold therefore +gets a `not-checked` row (the format, never a value) instead of passing silently; one at or above the threshold is reported as usual, +as a lower bound. The formats are recognised from their magic bytes (CONTAINERS in this file), and only these are: + parquet `PAR1` at the start and the end, or `PARE` at both (encrypted footer) arrow `ARROW1` at the start and the end (IPC file) + orc `ORC` at the start and before the last byte (ORC 0.11 files lack the end magic and pass as plain bytes) + zip `PK\\x03\\x04` (also docx, xlsx, pptx, jar, epub) gzip `\\x1f\\x8b` zstd `\\x28\\xb5\\x2f\\xfd` xz `\\xfd7zXZ\\x00` + 7z `7z\\xbc\\xaf\\x27\\x1c` lz4 frame `\\x04\\x22\\x4d\\x18` bzip2 `BZh`, a digit 1-9, `1AY&SY` avro `Obj\\x01` pdf `%PDF-` + zlib a valid RFC 1950 header followed by data that decodes (a bare zlib stream) +A magic without its footer (a text file that opens with "PAR1" or "ORC") is not a parquet or ORC file and gets no row. Every other +compressed or columnar format (brotli, snappy, rar, an Arrow IPC stream, raw deflate, a file whose magic was cut off) is read as plain +bytes and can pass clean. UTF-16 text, non-ASCII addresses and obfuscated forms (`[at]`, `%40`, `\\u0040`) are not seen. Names are not +detected. +""" +from __future__ import annotations + +import argparse +import os +import re +import subprocess +import sys +import zlib + +THRESHOLD = 10 # distinct addresses at or above which a file is reported +REPO_THRESHOLD = 50 # distinct addresses, over the files below THRESHOLD, at or above which the repository total is reported +MAX_READ = 32 << 20 # bytes read per file; the rest is reported as not scanned +MAX_READ_CAP = 1 << 30 # --max-bytes ceiling: read(n) sizes its buffer up front, so a huge n is a traceback (exit 1, which --strict reads as a finding) +MAX_COUNT = 100_000 # distinct addresses counted per file; the scan of that file stops there (bounds memory) +# (start, end, kind) byte regexes: a file that opens with `start` and ends with `end` holds compressed or columnar pages, where an address is +# not reliably seen, so it must not pass silently. Parquet (PAR1, or PARE with an encrypted footer) and Arrow repeat their magic as the +# footer, and an ORC postscript ends with "ORC" and one length byte: requiring that, and that the two magics do not overlap, keeps a text +# file that merely opens with "PAR1" or "ORC", or is only "PAR1", out. +CONTAINERS = ( + (rb"PAR1", rb"PAR1", "parquet"), (rb"PARE", rb"PARE", "parquet"), (rb"ARROW1", rb"ARROW1", "arrow"), (rb"ORC", rb"ORC.", "orc"), + (rb"PK\x03\x04", b"", "zip"), (rb"\x1f\x8b", b"", "gzip"), (rb"\x28\xb5\x2f\xfd", b"", "zstd"), (rb"\xfd7zXZ\x00", b"", "xz"), + (rb"7z\xbc\xaf\x27\x1c", b"", "7z"), (rb"\x04\x22\x4d\x18", b"", "lz4"), (rb"BZh[1-9]1AY&SY", b"", "bzip2"), + (rb"Obj\x01", b"", "avro"), (rb"%PDF-", b"", "pdf"), +) + +# The scan starts at each '@': the domain is matched within 256 bytes after it and the local part is read leftwards from the 64 bytes before +# it. One regex over the whole buffer would retry every start inside a long alphanumeric run (base64, hex, minified code), which is +# quadratic, and an unbounded domain match on `@a.a.a.` repeated for megabytes grows the regex stack to gigabytes. +DOMAIN = re.compile(rb"(?:[A-Za-z0-9-]{1,63}\.)+[A-Za-z]{2,24}") +LOCAL = re.compile(rb"[A-Za-z0-9._%+-]+") # matched on the (reversed) 64 bytes before the '@' +# Run on one lower-cased address: example.com/net/org (and subdomains), the .test/.example/.invalid/.localhost names, noreply forms, +# image asset-scale names (icon@2x.png is a file name, not a mailbox) and the ssh account of git hosts (git@github.com:org/repo.git). +RESERVED = re.compile( + rb"@(?:[a-z0-9-]+\.)*example\.(?:com|net|org)\Z" + rb"|\.(?:test|example|invalid|localhost)\Z" + rb"|(? set[bytes]: + """The distinct lower-cased email-shaped strings in `buf` (at most MAX_COUNT: the scan stops there). + + Reserved example/test/localhost, noreply, `git@` and asset-scale (`icon@2x.png`) names are left out unless `include_reserved`. + Edge cases: an empty buffer gives an empty set; a '@' with no letters, digits or `._%+-` before it, or with no dotted domain after + it, is not an address; case variants of one address count once. Pure; no I/O. + """ + found: set[bytes] = set() + i = buf.find(b"@") + while i != -1: + dom = DOMAIN.match(buf, i + 1, i + 257) + loc = LOCAL.match(buf[max(0, i - 64):i][::-1]) if dom else None + if loc: + addr = (loc.group()[::-1] + b"@" + dom.group()).lower() + if include_reserved or not RESERVED.search(addr): + found.add(addr) + if len(found) >= MAX_COUNT: + break + i = buf.find(b"@", i + 1) + return found + + +def zlib_stream(head: bytes) -> bool: + """True when `head` opens a zlib stream (RFC 1950): a deflate header whose check bits add up, and data that decodes. + + One text file in 31 that opens with a fitting pair of characters (`8` then `0`) passes the 2-byte header check by chance, so the + first block must also decode: it must end inside `head` or run for 64 bytes without an error. Truncated input is fine; a stream + that needs a preset dictionary is not recognised. Pure; the decoder reads at most 4 KiB and returns at most 4 KiB. + """ + if len(head) < 2 or head[0] & 0x8F != 8 or (head[0] << 8 | head[1]) % 31: + return False + d = zlib.decompressobj() + try: + d.decompress(head[:4096], 4096) + except zlib.error: + return False + return d.eof or len(head) >= 64 + + +def container(buf: bytes, tail: bytes, size: int) -> str | None: + """The compressed or columnar format a file is in, or None: `buf` is its first bytes, `tail` its last 8, `size` its length in bytes. + + Edge cases: an empty file, a magic without its footer (a text file that opens with "PAR1") and a file too short to hold both magics + (a bare "PAR1" is one magic read as both ends) are None. `size` is the real length, not len(buf), which a small --max-bytes cuts. Pure; no I/O. + """ + for start, end, kind in CONTAINERS: + head, foot = re.match(start, buf), re.search(end + rb"\Z", tail, re.S) + if head and foot and head.end() + foot.end() - foot.start() <= size: + return kind + return "zlib" if zlib_stream(buf[:4096]) else None + + +def tracked(root: str) -> list[str]: + """The paths of `git -C root ls-files -z`. Exits 2 with git's own message when it fails (not a repository): never a silent clean.""" + r = subprocess.run(["git", "-C", root, "ls-files", "-z"], capture_output=True) + if r.returncode: + print("committed_personal_data: git ls-files failed in %s: %s" % (root, r.stderr.decode("utf-8", "replace").strip()), file=sys.stderr) + sys.exit(2) + return [os.fsdecode(p) for p in r.stdout.split(b"\0") if p] + + +def row(rel: str, rule: str, msg: str) -> None: + """Print one TSV finding row, flushed. Tab and newline in the path become spaces so a hostile file name cannot split the row.""" + print("%s\t0\t%s\t%s" % (rel.replace("\t", " ").replace("\n", " "), rule, msg), flush=True) + + +def main(argv: list[str] | None = None) -> int: + """Print one TSV row per flagged or unchecked file (see the module docstring); return the exit code. Read-only.""" + ap = argparse.ArgumentParser(prog="committed_personal_data.py", description="Count distinct email addresses per tracked file; never print one.") + ap.add_argument("root", nargs="?", default=".", help="repository root (default .)") + ap.add_argument("--threshold", type=int, default=THRESHOLD, help="report a file at or above this many distinct addresses (default %d)" % THRESHOLD) + ap.add_argument("--repo-threshold", type=int, default=REPO_THRESHOLD, help="report the repository total at or above this many distinct addresses over the files below --threshold (default %d)" % REPO_THRESHOLD) + ap.add_argument("--include-reserved", action="store_true", help="also count the reserved example/test names, noreply forms, git@ and asset-scale names (for fictional fixtures)") + ap.add_argument("--strict", action="store_true", help="exit 1 when a file or the repository total is flagged (default: always exit 0)") + ap.add_argument("--stdin", action="store_true", help="scan the newline-separated paths on stdin instead of every tracked file") + ap.add_argument("--max-bytes", type=int, default=MAX_READ, help="bytes read per file (default %d, at most %d)" % (MAX_READ, MAX_READ_CAP)) + a = ap.parse_args(argv) + if not 1 <= a.threshold <= MAX_COUNT: + ap.error("--threshold must be between 1 and %d" % MAX_COUNT) + if not 1 <= a.repo_threshold <= MAX_COUNT: + ap.error("--repo-threshold must be between 1 and %d" % MAX_COUNT) + if not 1 <= a.max_bytes <= MAX_READ_CAP: + ap.error("--max-bytes must be between 1 and %d" % MAX_READ_CAP) + for stream in (sys.stdin, sys.stdout): # a non-UTF-8 file name must not crash the scan: that would hide every file + stream.reconfigure(errors="surrogateescape") + paths = [p for p in sys.stdin.read().split("\n") if p] if a.stdin else tracked(a.root) + flagged = holders = 0 + spread: set[bytes] = set() # the distinct addresses of the files that stay under --threshold, for the repository total + for rel in paths: + p = os.path.join(a.root, rel) + if os.path.islink(p) or not os.path.isfile(p): + continue + try: + with open(p, "rb") as fh: + buf = fh.read(a.max_bytes + 1) + size = os.fstat(fh.fileno()).st_size + fh.seek(max(0, size - 8)) + tail = fh.read(8) # the footer of a parquet, Arrow or ORC file; past `buf` when the file is over the cap + except OSError as e: + row(rel, "not-checked", "unreadable: %s" % e.strerror) + continue + big = len(buf) > a.max_bytes + found = addresses(buf[:a.max_bytes], a.include_reserved) + n = len(found) + partial = "; only the first %d bytes were read" % a.max_bytes if big else "" + kind = container(buf, tail, size) + if n >= a.threshold: + flagged += 1 + row(rel, "email-addresses", "%d%s distinct email addresses (threshold %d)%s" % (n, "+" * (n >= MAX_COUNT), a.threshold, partial)) + continue + holders += bool(found) + if len(spread) < MAX_COUNT: # bounds memory: the total stops growing there + spread |= found + if kind: + row(rel, "not-checked", "compressed/columnar container (%s): addresses inside its pages may not be seen, so this is not a clean result%s" % (kind, partial)) + elif big: + row(rel, "not-checked", "partially scanned: file over %d bytes, only the first %d were read" % (a.max_bytes, a.max_bytes)) + if len(spread) >= a.repo_threshold: + flagged += 1 + row(".", "email-addresses", "%d%s distinct email addresses across %d file(s) that each stay below the file threshold %d (repository threshold %d; --threshold 1 lists the files)" % ( + len(spread), "+" * (len(spread) >= MAX_COUNT), holders, a.threshold, a.repo_threshold)) + return 1 if a.strict and flagged else 0 + + +if __name__ == "__main__": + sys.exit(main()) diff --git a/.claude/skills/deep-code-review/scripts/review_checks.sh b/.claude/skills/deep-code-review/scripts/review_checks.sh index 5396f8abd..395ba7088 100755 --- a/.claude/skills/deep-code-review/scripts/review_checks.sh +++ b/.claude/skills/deep-code-review/scripts/review_checks.sh @@ -16,6 +16,10 @@ # (`hub=`); a matching changed file with the tool missing prints "NOT CHECKED: (install: )" on stderr, never a silent clean. # Built-in, always on, no install: invisible_chars.py flags NUL/C0 control characters and bidi/zero-width/tag characters in changed text files (tool `invisible-chars`, with line and col; # rows under fixture/test paths carry `[likely-fixture]`; third_party/, Pods/, bower_components/ and *.min.js are skipped; a `not-checked` row is unchecked surface). +# Built-in too: committed_personal_data.py counts the distinct non-reserved email addresses in each changed file (third_party/, Pods/, bower_components/ and *.min.js are skipped, as for invisible-chars), binary data included, +# and reports a file with 10 or more (tool `committed-data`, rule `email-addresses`; the row carries the count only, never an address). Addresses spread thin are summed over the files that stay below 10: 50 or more +# distinct give one `email-addresses` row with the path `.`. `not-checked` = unreadable, over the 32 MiB read cap, or, under the threshold, a file in a format whose pages can hide addresses (parquet, arrow, orc, zip, gzip, +# zstd, xz, 7z, lz4, bzip2, avro, pdf, zlib; the list and its ceilings are in committed_personal_data.py). gitleaks looks for secrets, not for exported personal data. # `--full` scans every tracked file (base = git's empty tree) instead of a diff; it also runs tenant_scope_check.py (SKIPPED without a *.prisma). gitleaks runs once over the working tree. gitleaks drops # fingerprints listed in the repo's .gitleaksignore (as of BASE, HEAD under --full). # gitleaks generic-api-key hits tagged likely-fixture collapse to one summary row per top fixture dir when it has >3 (every other rule, and name-only matches, stay rows); REVIEW_FIXTURE_ROWS=all lists every hit. @@ -637,6 +641,18 @@ if [ -n "$cf" ]; then fi fi +# Built-in, no tool to install: distinct non-reserved email addresses per changed file, binary data files included (committed_personal_data.py), over the same $cf as invisible-chars (vendored paths skipped). Cheap, so +# unleased: it reads each file's bytes once (capped at 32 MiB). Rows: tool committed-data, rule email-addresses|not-checked (the row with path `.` is the repository total over files below the file threshold); the message has the count or the container format, never a value. Not tagged likely-fixture: a real address in a fixture is still a leak. +pd="$here/committed_personal_data.py" +if [ -n "$cf" ]; then + if [ ! -f "$pd" ]; then skip committed-data "committed_personal_data.py missing" + else + pdok=0; run_one committed-data python3 "$pd" --stdin <<<"$cf" && scan_ok committed-data && pdok=1 + # flushed per row, as above: a timeout still yields the findings made so far + if [ "$pdok" = 1 ] || [ "$rc" = 124 ] || [ "$rc" = 142 ]; then while IFS=$'\t' read -r f l r m; do add committed-data "$f" "$l" "$r" "$m"; done <"$out"; fi + fi +fi + if [ "$tests" = 1 ]; then if [ -z "${REVIEW_TEST_CMD:-}" ]; then skip tests "REVIEW_TEST_CMD not set" elif run_one tests bash -c "$REVIEW_TEST_CMD"; then diff --git a/.claude/skills/deep-code-review/scripts/supply_chain_check.sh b/.claude/skills/deep-code-review/scripts/supply_chain_check.sh index 4d5f897a0..55007dc12 100644 --- a/.claude/skills/deep-code-review/scripts/supply_chain_check.sh +++ b/.claude/skills/deep-code-review/scripts/supply_chain_check.sh @@ -4,13 +4,19 @@ # osv-scanner lockfile advisories. Opt-in: needs --online (or SUPPLY_CHAIN_ONLINE=1) because it queries # api.osv.dev. Prints counts by severity + fixed versions; fails on >= --fail-on (default high). # zizmor, actionlint static checks on DIR/.github/workflows (local, no network). -# scorecard OpenSSF Scorecard for a PUBLIC repo; only with --scorecard owner/repo (queries GitHub). +# scorecard OpenSSF Scorecard, informational scores, no threshold. --scorecard owner/repo scores a PUBLIC repo remotely +# (queries GitHub). Without it, a git DIR is scanned offline from an export of HEAD (tracked files only, so an +# untracked node_modules never scores) with the checks that work offline; Vulnerabilities (a false 10 while +# api.osv.dev is unreachable) and Fuzzing (errors) never run there. That is a partial view, not comparable with +# a public Scorecard score: SAST sees workflow files only, and export-ignore paths are not scored. A subdirectory +# DIR scores the whole repo root (the ok line says so). A non-git DIR, or a partial clone missing objects (lazy +# fetch stays off, so this never touches the network), is SKIPPED. # Severity bands: advisory database_specific.severity, else the CVSS score OSV reports (>=9 critical, >=7 high, # >=4 moderate, else low). An advisory with neither (e.g. a CVSS vector string) is "unknown" and counts as at or # above ANY threshold (fail closed). # osv-scanner CLI crash (rc>1) retries via the OSV API with curl and prints "osv: api-fallback"; python3+curl needed. # Exit: 0 every check ran clean; 1 finding at/above threshold or a tool failed; 2 usage; -# 3 INCOMPLETE: a check was skipped (tool or python3 missing, osv/scorecard not requested), nothing failed. +# 3 INCOMPLETE: a check was skipped (tool or python3 missing, osv not requested, scorecard with no exportable git HEAD), nothing failed. # Reach: an npm osv row no source file imports prints "(reach: dev-only|transitive)"; only dev-only skips the gate # unless SUPPLY_CHAIN_STRICT=1 (transitive may be reached via a direct dependency); unknown reach is gated. # --allow-skip turns a skip-only run into exit 0. A finding (1) wins over INCOMPLETE (3). Needs python3 for osv. @@ -178,9 +184,24 @@ for t in zizmor actionlint; do else echo "ok $t"; fi done -# --- OpenSSF Scorecard (public repos) --- -if [ -z "$sc_repo" ]; then skip scorecard "pass --scorecard owner/repo (public repos; queries GitHub)" -elif ! have scorecard; then skip scorecard "not installed" +# --- OpenSSF Scorecard --- +# Local mode scans the repo root's HEAD export, like the remote mode scores the whole repo whatever DIR is. +# ponytail: git archive honors export-ignore, so such paths are not scored; use git checkout-index --prefix if that bites. +# GIT_NO_LAZY_FETCH=1 stops a partial clone fetching missing blobs (network); a git that ignores it may still do so. +sc_local() { + local top sub= + top=$(git -C "$dir" rev-parse --show-toplevel 2>/dev/null) || { skip scorecard "DIR is not a git repo (or git is missing); pass --scorecard owner/repo (public repos; queries GitHub)"; return; } + [ -n "$(git -C "$dir" rev-parse --show-prefix)" ] && sub=", DIR is a subdirectory so the whole repo is scored" + sc_ex=$(mktemp -d "${TMPDIR:-/tmp}/supply-chain-sc.XXXXXX") || { skip scorecard "mktemp failed"; return; } + trap 'rm -rf "$sc_ex"' EXIT # also on SIGTERM/SIGHUP/SIGINT: the export is a full copy of the tracked files + if ! (set -o pipefail; GIT_NO_LAZY_FETCH=1 git -C "$top" archive HEAD | tar -x -C "$sc_ex"); then skip scorecard "git archive HEAD failed (no commits, or objects missing in a partial clone: lazy fetch is off to stay offline)" + elif scorecard --local="$sc_ex" --show-details \ + --checks=Binary-Artifacts,Dangerous-Workflow,Dependency-Update-Tool,License,Packaging,Pinned-Dependencies,SAST,Security-Policy,Token-Permissions + then echo "ok scorecard (local export of HEAD in $top$sub; partial view, not comparable with a public Scorecard score; no pass/fail threshold)" + else echo "FAIL scorecard"; rc=1; fi +} +if ! have scorecard; then skip scorecard "not installed" +elif [ -z "$sc_repo" ]; then sc_local elif scorecard --repo="github.com/$sc_repo"; then echo "ok scorecard (review scores above; no pass/fail threshold)" else echo "FAIL scorecard"; rc=1; fi if [ "$rc" = 0 ] && [ -n "$skipped" ] && [ "$allow_skip" != 1 ]; then diff --git a/SHA256SUMS b/SHA256SUMS index 3649cc121..36e9cfa71 100644 --- a/SHA256SUMS +++ b/SHA256SUMS @@ -179,15 +179,15 @@ b8eaccc017ff43654798f4781f3e6c804ca3306d1365c2ab7870e9e1f855e0ba .claude/skills 6967542c52536ecea3dcaf1374a2ea78928e19d3b4595aacb09b025cc46bfe08 .claude/skills/business-ops/references/full-map.md 8be21023d9bc5b2eb1d10bf95c45de8526b212afbd46eb429f9856fc2620a86f .claude/skills/business-ops/references/regulated-domain-triage.md 66b3862368182ef01649d0db8f99913b518bdce78fce4a9282da247d77c7ea3a .claude/skills/business-ops/scripts/bizops_check.py -476d2b728eef9a96d144e78dc618f080e9533118efb26a6937dba696eb51bfb0 .claude/skills/communication-structure/INDEX.md +348dc6748344e458f9f6d1e3900956f10129288ce5c3ffccf731a48a7312de3c .claude/skills/communication-structure/INDEX.md 0e5afb32996c1f721bbfe709a78a21b6feaa1f347fdb705be1ed6a102b0db719 .claude/skills/communication-structure/SKILL.md 563a0b4eec3c77bbc34ac630d4fa5214f5c6726ae01468404fd704473110084e .claude/skills/communication-structure/VERSION -c90009107b343eff57af6565f8a7bec56626688402eb113bc4315760f0fdbdb7 .claude/skills/communication-structure/evals/evals.json +f1832c344f02562376e6e6254524d0051e61fd7843fc59b767b19b4fbe8271aa .claude/skills/communication-structure/evals/evals.json 7024c68d4964f2b247f5b9cc05d6827611c362664d055f137cb00a80f5d48e70 .claude/skills/communication-structure/evals/reader-checks.json 29020bf7db659b0e2019587f6857d34794d05dc1f693f909543b20dd7f0873b3 .claude/skills/communication-structure/evals/triggers.json -72564c2554208142c5d87122939a3771e8f2b7251cce36b60fc817c24eb7c6a5 .claude/skills/communication-structure/references/artifact-templates.md +07413f998402d3910cd71931ae1a1b0d635aa6b09952c3bc4d0ba7d04b6dd91a .claude/skills/communication-structure/references/artifact-templates.md b9e8c4a71aa0102503226c0180379bc3bc25f13485ac35232521da88b88fc843 .claude/skills/communication-structure/references/leadership-status-update.md -ef7df98af7a1db3e2ac62dd18d9b858f11c5908b6a3a8e6eb258791e57ea996b .claude/skills/communication-structure/scripts/comms_lint.py +5bc888cdb3fa30e9492b5a4fd9b58f10a6861209d6ffc6b52d184fb21e50e486 .claude/skills/communication-structure/scripts/comms_lint.py aee1b377e135c8dcdb808cb02b29d7bd8ca9ab3038023ee4afd616f5ffc74d6f .claude/skills/communication-structure/scripts/reader_check.py 4195df9b36e2a1f73fd6ffa397fca22e95e86465064110fdfaea65c19f130d9d .claude/skills/communication-structure/scripts/status_update_lint.py 2524bf62b627d362d888535ec9ef6527642cd7eeccbfd518454ed68f30d21344 .claude/skills/communication-structure/scripts/status_vocab_lint.py @@ -202,11 +202,11 @@ d98f3e4380da3496061ba63be7beebe75017ab4ee1734eda9a3e798e19007b0e .claude/skills 6108f7f8d13cefb9a23ca3960cc3bc73200f8ef88fe39c9d1a9b49b9ec7222f1 .claude/skills/contribution/scripts/learning_to_pr.py 92d22cc5d0a60ba6bdb1f4a9b597823dec5b8412cec09a5be6cf93b3973f4226 .claude/skills/contribution/scripts/prefile_check.sh 18f20db537ab784460ad33cf218f7a01ac0a172fc38413f14c0e19e1ce6821cc .claude/skills/contribution/scripts/share_learning.py -e2c5a8031dd78e97703fa9cf9b946f650721c6d4b56f57ae5726dd1d1d96260d .claude/skills/deep-code-review/INDEX-scripts.md -533bb3d58e6319f9b2a33e6b0f13212fc320c82b0164dd19612c32d274746ca5 .claude/skills/deep-code-review/INDEX.md +17958281dce6f131591dcdba808153213372b8ecdc1005c6ff6fdf6cead34da0 .claude/skills/deep-code-review/INDEX-scripts.md +9067b2ebf9b7fc44db387ec27b73a921f7ed3cc0cabf738b8f2442c4d5706f21 .claude/skills/deep-code-review/INDEX.md f7b6bb4c7b22d75df66ac9c63285a148f790f4d28e28e5e762448a2dcaf84df4 .claude/skills/deep-code-review/SKILL.md 11e3c4a20b52c9b6ff67b62f689ad64790a55e4bb9743260025179320c23aa9b .claude/skills/deep-code-review/VERSION -dffc2f49183ea895a27d503ecf0538bfa5dd18a60824f86e886b512bc6282d92 .claude/skills/deep-code-review/evals/evals.json +b97d95bc93cc23a8bc689e2e23db261a138367bcbc69d2f8a0452d5c2c6dfb68 .claude/skills/deep-code-review/evals/evals.json 13dbb52e4bf61b1093302e094151def167371d26e7cf67a026f3beeb09d34378 .claude/skills/deep-code-review/evals/triggers.json b84e3a2b88dc189f041784e76b97e68ad879915814e00f4b37cd27513746d6f1 .claude/skills/deep-code-review/references/a11y-aria.md 5a3e66353bcdabb0289ea926c7f192dbad750edca74ef0f0d8c0f6b71bcbf5ef .claude/skills/deep-code-review/references/a11y-color-motion.md @@ -215,7 +215,7 @@ de68ee1296de1b8ad6f855f9582cc2f382aab403c69b34047584e33fbf55c67d .claude/skills bc0c945b60a87d6f0b84caccfda7085b602d7921b803d4662c1d543b1229fc1e .claude/skills/deep-code-review/references/a11y-live.md 486fcb2070125cf04b603b820974fc7454a41ea2a2fbbb96c5c620c6fa995701 .claude/skills/deep-code-review/references/api-contracts-streaming.md 580dce392a2ee8401019812a2690d0691fe7ee8fae305b6dab0fe23fe1b35ce7 .claude/skills/deep-code-review/references/api-contracts.md -567091e6770713af556f27f59500d0e04e41cfc4dd38ae45794e9b198da8491d .claude/skills/deep-code-review/references/app-pentest.md +61c61d38fd2b27cb8c8781701899f149ae2f2defa4fe14204c55bc02e2df74b3 .claude/skills/deep-code-review/references/app-pentest.md 1b0f52a6249e129b4342acb40b3dc58615116ad1151b472293cb9fcf83f10c99 .claude/skills/deep-code-review/references/appsec-approvals.md d3dcddc61b815a8b50f908c6f9b646bacdd23902d715432185f5d1f1eced2bfc .claude/skills/deep-code-review/references/appsec-corroboration.md db83977c17ed42a158f5bbb2242f6dd91ff40392cdee823de37eec798bc64f89 .claude/skills/deep-code-review/references/appsec-crypto.md @@ -263,7 +263,7 @@ e22ea19809ee4b31eb218785b82a4a32b5731e0e358ab44acc014c53ee951588 .claude/skills 657077f914b63bdef5a54efc0771e1fd7a488c35e0227445ad34427138f3b611 .claude/skills/deep-code-review/references/domain-h.md ad2723dbfb3186c6c9624e2211fe169a97cb231468632ef1e2fa3a66b7ef26f6 .claude/skills/deep-code-review/references/domain-i.md c3b3bc5ae6563da5704e49f8c506111c7e6fe2c0f1e40078e8b0a40c1de8bab7 .claude/skills/deep-code-review/references/domain-j.md -006bfae4927758458d79a93a43afc9b13df38b98f928a6eaa66f577d3e443e8d .claude/skills/deep-code-review/references/domain-k.md +abc878658f0b1f4278511985b7520fb342c00ea2a094bece8317520c0fd8d40d .claude/skills/deep-code-review/references/domain-k.md 6d5480f980f4659b1b4e264f34520808be8296f62774d7dcc218d35ae152af2c .claude/skills/deep-code-review/references/domain-l.md 66a6ad6966ce90e7d352f153a7688b3610a5a189cd6353131349779699cb9a02 .claude/skills/deep-code-review/references/domain-m.md 6c928f2327737f8efe990a34574fea3d20e669def085df20ad2a20a9f2b3bbf3 .claude/skills/deep-code-review/references/domain-n.md @@ -300,7 +300,7 @@ cfdc3609c63b7669dd0ddcab3546dbae6036e5214668802acfb0cfc9117a8f25 .claude/skills 712ff4adad24308045101ee386340a61dd80230ddcbdedc30603d18b412378d3 .claude/skills/deep-code-review/references/method-diff.md 1c428659366bb21fad34ab66f42390410dbf5191d97c07ca40db8b0476081fa2 .claude/skills/deep-code-review/references/method-lenses.md 43ce8532a5f85d5784bcfd1d052ed6d7e832902602d2f7e5e8c4fafebef0b8e2 .claude/skills/deep-code-review/references/method-report.md -7e9666236ca18ae00e79d34af038d0dc7ae611783c7e5a73a5d5b5c7f404927d .claude/skills/deep-code-review/references/method-situational.md +b3cc64a91f2bca8ff8ba67aa225f36f3ee12ee0a3735d4240f5f7e897961805e .claude/skills/deep-code-review/references/method-situational.md fd354313f768db8a13a99e4012379a2c75f804a120c8e2de8556a8cac3981742 .claude/skills/deep-code-review/references/method.md 5fd086a47f860ab2b946d9409d1e1de03b1338cf832d5f1539fb3dfe63d85001 .claude/skills/deep-code-review/references/migration-parity.md 17838a58489145ecb8a466737ae5e12ac37e98260d335d160aff95d3d5354804 .claude/skills/deep-code-review/references/mobile-appsec.md @@ -332,7 +332,7 @@ d8075eb0c9482e05a96c8ded3f885f98b165a43cf19b52ba461fb90339b00cd7 .claude/skills e22c43592533cdeaa5dd6d6967d2a5519124dc2e5401b595caa182e7e0e68fb5 .claude/skills/deep-code-review/references/skill-authoring-and-size.md df698c4cbc9eb45aec1ffe62191801d713c9d6f923ed36914a703d3c9c3e71f1 .claude/skills/deep-code-review/references/small-diff-read-budget.md a0ed7ad061b7e60230601ee9b38a00400841d6ddf3af386edbd4b34a981c9798 .claude/skills/deep-code-review/references/standards-index.md -3aa290fc569016e1e433ffc4583606e63e1585feccc5306226b6e747770491bd .claude/skills/deep-code-review/references/supply-chain-check.md +0ab13691dcaddb5fe5fd80bde7b73410cf818c44b090e5f1a1d8d99a994a0b3e .claude/skills/deep-code-review/references/supply-chain-check.md 1d42030a463bd30ec43ed5a57af49551749d29ce3f3f4d0095ee59d3fafecd6f .claude/skills/deep-code-review/references/testing-ai-evals.md 65e11f80668f360d8516f3ddae7ea923df1c46f97edee08401024fb1c80bfd02 .claude/skills/deep-code-review/references/testing-and-evals.md 2463c659a2591017ee207a137da11ed263fee5e9b899c22873039e7c78fc40ce .claude/skills/deep-code-review/references/testing-ml.md @@ -351,11 +351,12 @@ b7b85757362758f393b6ddaaafed0f8a5e5075c9a74b604d3a80fca0301bdd71 .claude/skills 0de8627b98293ea5f2b2869d6aac8ec6fd2015336dc4c2e17cd3c5b8f251455b .claude/skills/deep-code-review/references/ux-writes.md 034c0fe0b35fc11bc46b4ae8d3651ff80f9ed28d43b8aa3373927ba37ea6b23f .claude/skills/deep-code-review/references/web-fetch.md b05798db39365a53e913dee3910c1193de2560b00ef2ed47c91ec1608ce2a96f .claude/skills/deep-code-review/references/web-render.md -693e0f16aebff6d729e19ff71a4c74cf4042b103a180143eb6fa92ba9bed3bae .claude/skills/deep-code-review/scripts/app_pentest.sh +8538d33f901a52d6d0084e52a265d7ff4d124c98ccdae1c1bd23a7fd5b3b57b6 .claude/skills/deep-code-review/scripts/app_pentest.sh bfa59b3b63638f8fc3a2b7f639c987c2ccebb0b13bd0cb04f82415e53c752e67 .claude/skills/deep-code-review/scripts/binaries_gate.py 59d4ba15b55581ef021a4866d64382660b5fbc6688e860102034ac14f897ba87 .claude/skills/deep-code-review/scripts/ci_cost_advisor.py a0301aaa1b602fd8ca44955eaf66b17893905f99aee8fe7d8db0be981bcc25c2 .claude/skills/deep-code-review/scripts/ci_cost_lint.py ed0f0c1eebfdca4dcbaafbf5c2f7b5cc93884de38715b57db894e0f210a811f9 .claude/skills/deep-code-review/scripts/closes_lint.py +8ca2cce4d6c745fd1a78c452d2a49961c7655ca78648b95e958db635cfabd9fc .claude/skills/deep-code-review/scripts/committed_personal_data.py 6632d4fa969875bb8a26fd8fd1677c3a5eaeeadbea00cb28c88a22d3d9dd4138 .claude/skills/deep-code-review/scripts/context_pack.py c5b40f3f0d3fb174e6c9490150c7b85723de33f95d3fab87d60214d4939401a8 .claude/skills/deep-code-review/scripts/coverage_ledger.py 36c1447a65135496c519397c04c6a74ef066bd01399d593907e598b8d2038498 .claude/skills/deep-code-review/scripts/derived_numbers.py @@ -393,12 +394,12 @@ cdacdfba4a9aff3c20344bfc8a4fe84b6ed24e85e9b5feefb2a08058f8858bda .claude/skills 5d6910164ddf6dfd4ed62dfc365a2f615f49dc07878ed3d1f881aa9518e481c2 .claude/skills/deep-code-review/scripts/post_review.sh 9b564750380e275e2bcbfd6961cd357001ff307dbf610b889cdfe8684e11b109 .claude/skills/deep-code-review/scripts/provision_tools.sh f1108e2700864195a239744b319b1d04953879589f8a2b83708bc45a080e3d86 .claude/skills/deep-code-review/scripts/reaper_lint.py -2916de43f608d5cf0b3311cb0e6d0b50921cde81f2aa0436827fba16cafbf6d3 .claude/skills/deep-code-review/scripts/review_checks.sh +b069c22172d06cc8b88088aee28450845a14a27c570bf3826ac01c2f59e17ea7 .claude/skills/deep-code-review/scripts/review_checks.sh 8c87c02a050a975e453b52ecacbaf01481c7134466a9c67feaf409d22e36d9f7 .claude/skills/deep-code-review/scripts/review_feedback.py 859c67d6a6a781c873f47d5662d403da2321da53e6994ca3d0ccdbf74afd39ac .claude/skills/deep-code-review/scripts/score_review.py e9dfa337ad16d0022a3b7a87e0ba49acb5c2c9c0a5ee24396ff80523f2f8fa1a .claude/skills/deep-code-review/scripts/security_verify.py 761ccb5730e297f41d865aeec9babd44b00b5cc029a8d4e7b2cb37b1122222be .claude/skills/deep-code-review/scripts/source_scan_tests.py -8da75dd5a0d483d6e5ecfb29d15210c4225eff118304dc09aee34f0c8d9506dc .claude/skills/deep-code-review/scripts/supply_chain_check.sh +b28b622d7e59fb00549141b439871f5a6c6faddcb83d2d76c7303d7efdbb2897 .claude/skills/deep-code-review/scripts/supply_chain_check.sh ef606126a1a40934671f8fea1d5caaeaf7957070033557445692eb249a643bb7 .claude/skills/deep-code-review/scripts/tenant_scope_check.py 01ce284157850e6f9b146023dbcf12a8ed014db578ee5f6f27ddf21a2a3a0b5e .claude/skills/deep-code-review/scripts/token_differ.py cbfd54a4fa86885f0c49b77327f034c2986170c3d74b4c39b2d9f2686ba87c40 .claude/skills/deep-code-review/scripts/validate_status_claims.py diff --git a/changelog.d/app-pentest-openapi.md b/changelog.d/app-pentest-openapi.md new file mode 100644 index 000000000..4ec1e61cb --- /dev/null +++ b/changelog.d/app-pentest-openapi.md @@ -0,0 +1,3 @@ +### Added +- `scripts/app_pentest.sh --openapi SPEC URL`: optional schemathesis API fuzzing of the validated target. SPEC is a local file or an http(s) URL. A URL must pass the same allowlist, userinfo, port and resolution rules as the target (and, with `--active`, the listed-and-private rule); it is fetched once, without redirects or env proxies (10 s, 10 MB), and schemathesis gets the local copy. A spec that is unreadable, over 10 MB, or has an absolute or scheme-relative `$ref` or `externalValue` in its text is refused (an early text match, not the guard). The binary is `schemathesis` on PATH, else `~/.local/bin/schemathesis`; no other name is run. Passive runs send only GET and HEAD; `--active` has no method filter and keeps the loopback/private-only rule. Both modes are rate-limited (`--rate-limit 5/s -n 10 -w 1`), pinned to the target origin with `--origin`, and run with an empty pinned config file. The run has an egress block: every proxy variable is a closed loopback port and `no_proxy` is the target host alone, so a remote `$ref` or `externalValue` that gets past the text match (a JSON unicode escape, a YAML tag) cannot be fetched and the caller's own proxy cannot divert target traffic. Residual, documented: the same host on another port, its subdomains, and a fetcher that ignores proxy variables. The JSON gains `tools.schemathesis` (`skipped`, `ran` or `error`; `ran` needs a complete report that tested at least one operation) and `openapi` (null without the flag, else `spec`, `ips` and the report's `tested` and `errored` operation counts, so `errored` above 0 is partial coverage). Each failure is a `{"tool":"schemathesis",...,"verified":false}` lead: medium for `ST101`, `ST102`, `ST401` and a `server_unavailable` crash, low for every other code. `references/app-pentest.md` documents the flags, the safety rules and the severity map. Test: `scripts/test_app_pentest.sh` (a fake schemathesis, a toy spec server, and, when the real binary is installed, wire-level runs that prove the passive and active methods, the egress block against hidden remote references and a user proxy, and the spec read limits). +- size-budget-raise: .claude/skills/deep-code-review/references/app-pentest.md 3033→6344 the --openapi safety rules, the egress block, the schemathesis flags with their reasons, the status semantics and the severity map diff --git a/changelog.d/ci-liveness-before-ci-only-fix.md b/changelog.d/ci-liveness-before-ci-only-fix.md new file mode 100644 index 000000000..5004ddecb --- /dev/null +++ b/changelog.d/ci-liveness-before-ci-only-fix.md @@ -0,0 +1,3 @@ +### Added +- `references/domain-k.md`: a workflow file is not CI coverage until it runs. Before claiming CI coverage or recommending a CI-only fix, a review confirms workflows fire (`gh api repos///actions/permissions --jq .enabled`, `gh run list -L 5`). If Actions is off or unknown it says so and aims the fix at the gate that does run, or flags re-enabling Actions as an owner decision. The red-flag line gains "a CI-only fix where Actions is off or no run exists". Eval: `ci-only-fix-where-actions-is-disabled-confirm-workflows-run`. +- size-budget-raise: .claude/skills/deep-code-review/references/domain-k.md 5188→5674 the liveness rule (one checklist bullet plus one red-flag clause) diff --git a/changelog.d/committed-personal-data-check.md b/changelog.d/committed-personal-data-check.md new file mode 100644 index 000000000..d2ae5b98f --- /dev/null +++ b/changelog.d/committed-personal-data-check.md @@ -0,0 +1,8 @@ +### Added +- Committed personal-data check. New built-in `committed_personal_data.py` (stdlib only) counts the distinct email-shaped strings in every tracked file, plain binary data (sqlite, pickles) included, and reports a file with 10 or more as one `email-addresses` row: path and count only, never an address, a domain or a line. gitleaks, trufflehog and semgrep look for secrets, so a committed export of real people's addresses passed all of them clean. Not counted: the reserved example and test names (example.com, .net and .org with their subdomains, `.test`, `.example`, `.invalid`, `.localhost`), noreply forms, the ssh account `git@` of git remotes (`git@host:org/repo.git`, `ssh://git@host/org/repo`) and image asset-scale names (`icon@2x.png`, `icon@1.5x.png`, `icon@2x-dark.png`, for png, jpg, jpeg, gif, webp, avif, svg, pdf, ico, bmp, tif, tiff, heic, heif and apng). `--include-reserved` counts them so tests can use fictional example.com fixtures. Flags: `--threshold N` and `--repo-threshold N` (each 1 to 100000), `--strict` (exit 1 when a file or the repository total is flagged; the default is advisory, exit 0), `--stdin` (scan the listed paths instead of every tracked file), `--max-bytes N` (1 to 1 GiB, default 32 MiB; a larger file is scanned over its first bytes and a not-checked row says so). It exits 2 when `git ls-files` fails or a flag is out of range, never a silent pass and never a traceback. +- Addresses spread thin are summed. A leak of one address per file never reaches the per-file threshold, so the distinct addresses of all the scanned files that stay below it are also counted together, and a total of 50 or more (`--repo-threshold`) gives one `email-addresses` row with the path `.` ("69 distinct email addresses across 75 file(s) ..."). A file already reported is left out of the total, an address repeated across files counts once, and in diff mode only the changed files are counted. `--threshold 1` lists the files behind the row. The default 50 is five times the file threshold. The audit behind this check found about 850 addresses; this repository holds 69 distinct ones in 75 files that each stay below 10 (60 of them under docs/bench), so its own `--full` run shows the row as a lead. A default of 100 would stay quiet here but would miss a spread of 50 to 99 addresses. +- Compressed and columnar files do not pass silently, for exactly these formats. A file that stays under the threshold gets a `not-checked` row (path and format, never a value) when it is parquet (`PAR1` at the start and the end, or `PARE` at both for an encrypted footer), an Arrow IPC file (`ARROW1` at both ends), ORC (`ORC` at the start and before the last byte), zip (also docx, xlsx, jar), gzip, zstd, xz, 7z, lz4 frame, bzip2, Avro, PDF, or a zlib stream (a valid header whose data decodes). Parquet, Arrow and ORC need their footer, so a text file that merely begins with "PAR1" or "ORC" gets no row, and neither does a file too short to hold both magics (a bare "PAR1") or text whose first two bytes pass the zlib header check by chance. Every other format (brotli, snappy, rar, Arrow IPC streams, raw deflate, ORC 0.11 files, which lack the end magic) is read as plain bytes and can pass clean, as can UTF-16 text and obfuscated forms (`[at]`, `%40`); names are not detected. A file at or above the threshold is reported as usual, as a lower bound. +- Wired into `review_checks.sh` as the built-in `committed-data` step, right after `invisible-chars`: it scans the changed files in diff mode and every tracked file under `--full`, skips the paths every step skips (node_modules, vendor, build dirs) plus third_party/, Pods/, bower_components/ and *.min.js, as `invisible-chars` does, and is listed under "analyzers ran". A missing script gives one not-checked row. Rows are not tagged `likely-fixture`, because a real address in a fixture is still a leak. The `method-situational.md` analyzer list names the step with its `.` and `not-checked` rows. +- Threshold 10, measured rather than guessed: 5 x 8 MiB of seeded random bytes held 2 email-shaped strings (0.05 per MB); across this repository's 93,711 tracked files at this commit the most distinct non-reserved addresses in one file is 6 (75 files hold any, none is a recognised container), so 10 flags nothing here while a threshold of 5 would flag 3 files (two bench outputs and this check's own unit test); a whole-tree run took 3 to 7 s. A data export holds dozens to thousands. Contributor lists (AUTHORS, .mailmap) reach 10 by design and show up as leads for the reviewer. `--threshold` tunes it. +- New tests `scripts/test_committed_personal_data.py` (31 unit tests, one skipped on macOS: they cover the container rows for every format above, real gzip, bzip2, xz, zlib and PDF data, footerless and overlapping magics and zlib lookalikes, the `git@` and asset-scale names, the repository total with its edges, the flag ranges and the regex memory bound on a hostile dotted run) and `scripts/test-review-checks-committed-data.sh` (12 checks, with vendor/ and node_modules/ files planted and left unflagged, and the repository-total row passing through the hub), registered in `scripts/ci-tests.d/committed-personal-data.sh`. +- size-budget-raise: .claude/skills/deep-code-review/references/method-situational.md 43864→44018 the analyzer list names the new committed-data step and its rows diff --git a/changelog.d/supply-chain-scorecard-local.md b/changelog.d/supply-chain-scorecard-local.md new file mode 100644 index 000000000..419ebd471 --- /dev/null +++ b/changelog.d/supply-chain-scorecard-local.md @@ -0,0 +1,3 @@ +### Added +- `supply_chain_check.sh` scans a git DIR with OpenSSF Scorecard offline when `--scorecard owner/repo` is not given; before, that run printed `SKIPPED scorecard`. It exports the repo root's HEAD (`git archive HEAD`, tracked files only) and runs `scorecard --local` on the export, so an untracked `node_modules` cannot score Binary-Artifacts 0. The export is removed afterwards, also when the run is stopped by SIGTERM, SIGHUP or SIGINT. Only the checks that work offline run: Binary-Artifacts, Dangerous-Workflow, Dependency-Update-Tool, License, Packaging, Pinned-Dependencies, SAST, Security-Policy and Token-Permissions. Vulnerabilities is never included because it reported a false 10 ("0 existing vulnerabilities") while api.osv.dev was unreachable, and Fuzzing errors offline. Scores stay informational with no pass/fail threshold, as in the remote mode, and the `ok scorecard` line says the offline view is partial and not comparable with a public Scorecard score. A subdirectory DIR scores the whole repo root, and the same line names the root and says so. `git archive` runs with `GIT_NO_LAZY_FETCH=1`, so a partial clone that is missing blobs is SKIPPED with a reason naming the partial clone instead of fetching over the network. A non-git DIR or a repo with no commit is also SKIPPED with the reason (exit 3 unless `--allow-skip`), and `--scorecard owner/repo` keeps the remote scan. The `supply-chain-check.md` table row and the script header say so. Test: `scripts/test-supply-chain-check.sh`, with a fake scorecard that records its arguments and the files it can see, and can hang so a test can signal the script. +- size-budget-raise: .claude/skills/deep-code-review/references/supply-chain-check.md 3421→3646 the scorecard table row now states the offline git-DIR mode, its whole-repo scope and its partial view diff --git a/changelog.d/w7a-comms-lint-guard-wait.md b/changelog.d/w7a-comms-lint-guard-wait.md new file mode 100644 index 000000000..d4074abe8 --- /dev/null +++ b/changelog.d/w7a-comms-lint-guard-wait.md @@ -0,0 +1,5 @@ +### Fixed +- `comms_lint.py` `audit-pr`: the budget (1200 words and 160 lines, a failure above 1.5 times either) now counts everything outside `## Changes by area`, so a body written to the template with many areas no longer fails for its area text (eight areas came to 2024 words and failed). Inside that section each `### ` area fails above 200 words and the whole body fails above 3000 words, with no slack. The per-area cap covers `### ` areas only: bullet areas, text before the first `### ` heading and a `#### ` section that follows no `### ` heading are bound by the 3000-word cap alone. The `words` and `lines` the lint reports, and its score line, are the counted ones. `artifact-templates.md` states the scope and both caps. Test: `scripts/test_audit_pr_budget.py`, plus the eval `audit-pr-multi-area-body-budget`. +- `comms_lint.py` placeholder rule: a run of capital X inside a single-backtick code span on one line (a `mktemp` template such as `drain.XXXXXX`) is no longer read as a placeholder; the same run in a double-backtick span or a span split across lines still is. TBD, TODO and a lone XXX in prose still fail, also on a line that carries a template, and so does `` (also `` and ``), which the rule did not catch before. Test: `scripts/test_comms_lint_placeholder_template.py`. +- `scripts/test_destructive_guard_w21.py`: its waits on the guard process (20 s for the wall-clock case, 10 s or 20 s elsewhere) are one `WAIT = 60` ceiling, so a loaded host (load average near 13) no longer fails the wall-clock case while a real hang still fails, at 60 s. Test: `scripts/test_destructive_guard_w21_wait.py`, which also pins the guard's own `TIMEOUT` at 20 s or less so raising its watchdog still fails a test. +- size-budget-raise: .claude/skills/communication-structure/references/artifact-templates.md 9234→9443 the audit-pr paragraph states the budget scope and the two area and body caps diff --git a/scripts/ci-tests.d/audit-pr-budget.sh b/scripts/ci-tests.d/audit-pr-budget.sh new file mode 100644 index 000000000..499663df4 --- /dev/null +++ b/scripts/ci-tests.d/audit-pr-budget.sh @@ -0,0 +1,2 @@ +# Registers scripts/test_audit_pr_budget.py (the audit-pr budget outside "Changes by area", the 200-word area cap and the 3000-word hard cap) with CI. +python3 scripts/test_audit_pr_budget.py diff --git a/scripts/ci-tests.d/committed-personal-data.sh b/scripts/ci-tests.d/committed-personal-data.sh new file mode 100644 index 000000000..438aca136 --- /dev/null +++ b/scripts/ci-tests.d/committed-personal-data.sh @@ -0,0 +1,3 @@ +# Registers the committed_personal_data.py tests: unit (CSV, parquet-like and Arrow-like fixtures flagged only with --include-reserved, count-only output, threshold edge, noreply, git@ and asset-scale names, the repository-total row for addresses spread thin, a not-checked row for every recognised container format with its footer and for real gzip, bzip2, xz, zlib and PDF data, zlib-lookalike text and magic-only text left alone, --strict, --stdin, size and count caps, fail-closed flag ranges, random-byte noise) and the review_checks.sh committed-data step (rows, vendored paths, repository total, diff vs --full, missing script). Offline. +python3 scripts/test_committed_personal_data.py +bash scripts/test-review-checks-committed-data.sh diff --git a/scripts/ci-tests.d/comms-lint-placeholder-template.sh b/scripts/ci-tests.d/comms-lint-placeholder-template.sh new file mode 100644 index 000000000..a8a4d3251 --- /dev/null +++ b/scripts/ci-tests.d/comms-lint-placeholder-template.sh @@ -0,0 +1,2 @@ +# Registers scripts/test_comms_lint_placeholder_template.py (a mktemp template in a code span is not a placeholder) with CI. +python3 scripts/test_comms_lint_placeholder_template.py diff --git a/scripts/ci-tests.d/destructive-guard-w21-wait.sh b/scripts/ci-tests.d/destructive-guard-w21-wait.sh new file mode 100644 index 000000000..14b0c1082 --- /dev/null +++ b/scripts/ci-tests.d/destructive-guard-w21-wait.sh @@ -0,0 +1,2 @@ +# Registers scripts/test_destructive_guard_w21_wait.py (the load-tolerant wait ceiling of the w21 wall-clock case) with CI. +python3 scripts/test_destructive_guard_w21_wait.py diff --git a/scripts/size-budgets.tsv b/scripts/size-budgets.tsv index eacc5c84b..c980910e4 100644 --- a/scripts/size-budgets.tsv +++ b/scripts/size-budgets.tsv @@ -112,7 +112,7 @@ .claude/skills/deep-code-review/references/domain-h.md 7805 .claude/skills/deep-code-review/references/domain-i.md 1456 .claude/skills/deep-code-review/references/domain-j.md 2823 -.claude/skills/deep-code-review/references/domain-k.md 5188 +.claude/skills/deep-code-review/references/domain-k.md 5674 .claude/skills/deep-code-review/references/domain-l.md 1181 .claude/skills/deep-code-review/references/domain-m.md 1193 .claude/skills/deep-code-review/references/domain-n.md 858 @@ -139,7 +139,7 @@ .claude/skills/deep-code-review/references/language-stack-redflags.md 10082 .claude/skills/deep-code-review/references/machine-report.md 13064 .claude/skills/deep-code-review/references/merge-operations.md 79430 -.claude/skills/deep-code-review/references/method-situational.md 43864 +.claude/skills/deep-code-review/references/method-situational.md 44018 .claude/skills/deep-code-review/references/method.md 29936 .claude/skills/deep-code-review/references/method-diff.md 7872 .claude/skills/deep-code-review/references/method-lenses.md 7743 @@ -208,14 +208,14 @@ README.md 16118 .claude/skills/product-discovery/references/full-map.md 12384 .claude/skills/growth-analytics/references/full-map.md 10661 .claude/skills/positioning/references/full-map.md 9520 -.claude/skills/INDEX.md 16151 +.claude/skills/INDEX.md 16291 .claude/skills/agentic-delivery/INDEX.md 4672 .claude/skills/agentic-delivery/INDEX-scripts.md 17223 -.claude/skills/deep-code-review/INDEX.md 20551 -.claude/skills/deep-code-review/INDEX-scripts.md 6045 +.claude/skills/deep-code-review/INDEX.md 20552 +.claude/skills/deep-code-review/INDEX-scripts.md 6334 .claude/skills/agentic-delivery/references/pipeline-optimizer.md 19857 .claude/skills/agentic-delivery/references/focus-kit.md 5235 -.claude/skills/communication-structure/references/artifact-templates.md 9234 +.claude/skills/communication-structure/references/artifact-templates.md 9443 .claude/skills/deep-code-review/references/code-structure.md 6449 .claude/skills/deep-code-review/references/compatibility-portability.md 5771 .claude/skills/deep-code-review/references/delivery-speed.md 7666 @@ -232,8 +232,8 @@ README.md 16118 .claude/skills/deep-code-review/references/api-contracts-streaming.md 7409 .claude/skills/deep-code-review/references/performance-latency-depth.md 13236 .claude/skills/deep-code-review/references/small-diff-read-budget.md 526 -.claude/skills/deep-code-review/references/app-pentest.md 3033 -.claude/skills/deep-code-review/references/supply-chain-check.md 3421 +.claude/skills/deep-code-review/references/app-pentest.md 6344 +.claude/skills/deep-code-review/references/supply-chain-check.md 3646 .claude/skills/deep-code-review/references/tool-hub.md 8731 .claude/skills/deep-code-review/references/desktop-native-appsec.md 4724 .claude/skills/deep-code-review/references/dynamic-probe.md 1890 diff --git a/scripts/test-review-checks-committed-data.sh b/scripts/test-review-checks-committed-data.sh new file mode 100644 index 000000000..a3bb99228 --- /dev/null +++ b/scripts/test-review-checks-committed-data.sh @@ -0,0 +1,73 @@ +#!/usr/bin/env bash +# Tests for review_checks.sh's built-in committed-data step (committed_personal_data.py): a CSV and a binary-ish file with 12 distinct +# made-up-domain addresses give one email-addresses row each (count only, never an address, not tagged likely-fixture); 9 addresses, +# reserved example.com names and vendor/, node_modules/, third_party/, Pods/ and *.min.js files give none; a small parquet-like file +# gives a not-checked row; six files of 9 addresses each, none reaching the threshold, give one repository-total row (path "."); +# diff mode flags only changed files and --full all tracked files; the step is listed under "analyzers ran"; a missing script gives +# one not-checked row. Hermetic: temp git repo, pruned PATH, fictional data. +set -euo pipefail +ROOT="$(cd "$(dirname "${BASH_SOURCE[0]}")/.." && pwd)" +RC="$ROOT/.claude/skills/deep-code-review/scripts/review_checks.sh" +T=$(mktemp -d "${TMPDIR:-/tmp}/dcr-test-rcpd.XXXXXX"); trap 'rm -rf "$T"' EXIT +fail=0 +ok() { printf 'ok %s\n' "$1"; } +bad() { printf 'FAIL %s\n' "$1"; fail=1; } + +mkdir "$T/bin" "$T/nopd" "$T/repo" +# Pruned PATH: no semgrep, trivy or linters, so the run is offline and only built-in steps fire. +for c in git python3 bash sed grep head tail mktemp rm cat dirname perl env tr wc sleep timeout; do ln -s "$(command -v "$c")" "$T/bin/" 2>/dev/null || true; done +cp "$RC" "$T/nopd/review_checks.sh" # no committed_personal_data.py beside it +# mk KIND DOMAIN N PATH: N distinct fictional addresses as a CSV or as a parquet-like binary (magic, length-prefixed values, NUL padding). +mk() { python3 - "$@" <<'PY' +import sys +kind, domain, n, path = sys.argv[1], sys.argv[2].encode(), int(sys.argv[3]), sys.argv[4] +a = [b"fixture-user-%04d@%s" % (i, domain) for i in range(1, n + 1)] +if kind == "csv": + data = b"id,name,email\n" + b"".join(b"%d,Jane Smith,%s\n" % (i, x) for i, x in enumerate(a)) +else: + data = b"PAR1" + b"".join(len(x).to_bytes(4, "little") + x for x in a) + b"\0" * 64 + b"PAR1" +open(path, "wb").write(data) +PY +} +cd "$T/repo" +git init -q . && git -c user.email=t@example.com -c user.name=t commit -q --allow-empty -m empty +mk csv acme.zz 12 old.csv +git add . && git -c user.email=t@example.com -c user.name=t commit -q -m old +git branch base +mkdir -p third_party Pods vendor node_modules spread lib tests/fixtures +mk csv acme.zz 12 report.csv +mk bin acme.zz 12 export.parquet +mk csv acme.zz 10 ten.csv +mk csv acme.zz 9 nine.csv +mk csv example.com 12 reserved.csv +mk csv acme.zz 12 third_party/vendored.csv +mk csv acme.zz 12 tests/fixtures/people.csv +mk csv acme.zz 12 Pods/people.csv +mk csv acme.zz 12 vendor/people.csv +mk csv acme.zz 12 node_modules/people.csv +mk csv acme.zz 12 lib/app.min.js +mk bin acme.zz 5 small.parquet +for i in 1 2 3 4 5 6; do mk csv "spread$i.zz" 9 "spread/s$i.csv"; done # 6 domains x 9 addresses: no file reaches 10 +git add . && git -c user.email=t@example.com -c user.name=t commit -q -m files +export PERUN_HEAVY_DIR="$T/heavy" +PATH="$T/bin" bash "$RC" --base base 2>"$T/err" >"$T/out" || true +rows() { jq -r '[.[] | select(.tool=="committed-data") | .file] | sort | join(" ")' "$1"; } + +[ "$(rows "$T/out")" = ". export.parquet report.csv small.parquet ten.csv tests/fixtures/people.csv" ] && ok "csv and binary flagged; 9 addresses, reserved names, vendored paths and unchanged files are not" || bad "row files: $(rows "$T/out")" +jq -e '[.[] | select(.tool=="committed-data" and .file=="report.csv")] | length == 1 and all(.[]; .rule=="email-addresses" and .line==0 and .message=="12 distinct email addresses (threshold 10)" and .text=="email-addresses: 12 distinct email addresses (threshold 10)")' "$T/out" >/dev/null && ok "row shape: rule, count, text" || bad "row shape" +jq -e 'any(.[]; .tool=="committed-data" and .file=="ten.csv" and (.message|startswith("10 distinct")))' "$T/out" >/dev/null && ok "threshold edge: 10 flagged" || bad "ten.csv row" +jq -e '[.[] | select(.tool=="committed-data")] | length == 6 and all(.[]; (has("hint")|not) and (.text|contains("likely-fixture")|not))' "$T/out" >/dev/null && ok "fixture-path row not tagged likely-fixture" || bad "fixture tag" +jq -e '[.[] | select(.tool=="committed-data" and .file==".")] | length == 1 and .[0].rule=="email-addresses" and .[0].line==0 and .[0].message=="63 distinct email addresses across 8 file(s) that each stay below the file threshold 10 (repository threshold 50; --threshold 1 lists the files)"' "$T/out" >/dev/null && ok "addresses spread thin give one repository-total row" || bad "repository-total row" +jq -e '[.[] | select(.tool=="committed-data" and .file=="small.parquet")] | length == 1 and .[0].rule=="not-checked" and (.[0].message|contains("compressed/columnar container (parquet)"))' "$T/out" >/dev/null && ok "small parquet under the threshold is a not-checked row, not a clean pass" || bad "container row" +jq -e 'any(.[]; .tool=="committed-data")' "$T/out" >/dev/null && ! grep -Eq '@|fixture-user|acme|\.zz' "$T/out" "$T/err" && ok "no address, local part or domain in stdout or stderr" || bad "value leaked, or no committed-data rows to check" +grep -Eq 'analyzers ran:.* committed-data' "$T/err" && ok "listed under analyzers ran" || bad "analyzers ran" +jq -e 'any(.[]; .tool=="committed-data") and all(.[]; has("file") and has("line") and has("rule") and has("message") and has("text") and has("tool"))' "$T/out" >/dev/null && ok "existing row shape" || bad "row shape (all rows)" + +PATH="$T/bin" bash "$RC" --base base --full 2>"$T/err1" >"$T/out1" || true +[ "$(rows "$T/out1")" = ". export.parquet old.csv report.csv small.parquet ten.csv tests/fixtures/people.csv" ] && ok "--full also flags the file committed before base" || bad "full row files: $(rows "$T/out1")" + +PATH="$T/bin" bash "$T/nopd/review_checks.sh" --base base 2>"$T/err2" >"$T/out2" || true +jq -e '[.[] | select(.tool=="committed-data")] | length == 1 and .[0].rule=="not-checked" and .[0].file=="" and (.[0].message|contains("committed_personal_data.py missing"))' "$T/out2" >/dev/null && ok "script missing gives one not-checked row" || bad "missing-script row" +grep -q 'not run: committed-data' "$T/err2" && ! grep -Eq 'analyzers ran:.* committed-data' "$T/err2" && ok "missing script reported on stderr, not listed as ran" || bad "missing-script stderr" + +[ "$fail" = 0 ] diff --git a/scripts/test-supply-chain-check.sh b/scripts/test-supply-chain-check.sh index edb5eff3f..cf003f238 100644 --- a/scripts/test-supply-chain-check.sh +++ b/scripts/test-supply-chain-check.sh @@ -14,11 +14,27 @@ expect() { local n=$1 w=$2 p=$3 o r; shift 4; o=$("$@" 2>&1) && r=0 || r=$? mkdir -p "$T/bin" "$T/repo/.github/workflows" "$T/empty" "$T/sys" "$T/nopy" "$T/nowf/.github/workflows" : >"$T/repo/.github/workflows/ci.yml" +# git fixtures: a repo with tracked files plus a never-tracked one (stands in for node_modules), and a repo with no commit. +# No git repo above $T may leak in (GIT_CEILING_DIRECTORIES), and no hook env (GIT_DIR...) may redirect git. +g() { git -c user.name=Test -c user.email=test@example.com -c commit.gpgsign=false -c core.hooksPath=/dev/null "$@"; } +export GIT_CEILING_DIRECTORIES="$T"; unset GIT_DIR GIT_WORK_TREE GIT_INDEX_FILE +mkdir -p "$T/gitrepo/.github/workflows" "$T/gitrepo/node_modules" "$T/gitrepo/sub" "$T/nocommit" "$T/xtmp" +: >"$T/gitrepo/.github/workflows/ci.yml"; echo tracked >"$T/gitrepo/tracked.txt"; echo inner >"$T/gitrepo/sub/inner.txt" +g -C "$T/gitrepo" init -q; g -C "$T/gitrepo" add -A; g -C "$T/gitrepo" commit -qm fixture +: >"$T/gitrepo/node_modules/untracked.bin"; g -C "$T/nocommit" init -q for t in cat grep python3 dirname env find head; do ln -s "$(command -v $t)" "$T/sys/$t"; done # PATH without the fake tools for t in cat grep dirname env bash find head; do ln -s "$(command -v $t)" "$T/nopy/$t"; done # ... and without python3 -for t in zizmor actionlint scorecard; do +for t in zizmor actionlint; do printf '#!/usr/bin/env bash\necho "%s $*" >>"$FAKE_LOG"\n[ "${FAKE_FAIL:-}" = %s ] && exit 1\nexit 0\n' "$t" "$t" >"$T/bin/$t" done +cat >"$T/bin/scorecard" <<'S' +#!/usr/bin/env bash +echo "scorecard $*" >>"$FAKE_LOG" +for a in "$@"; do [ "${a#--local=}" != "$a" ] && (cd "${a#--local=}" && find . -type f | sort | sed 's/^/scorecard-sees /' >>"$FAKE_LOG"); done +[ -n "${FAKE_HANG:-}" ] && { echo $$ >"$FAKE_HANG"; exec sleep 20; } +[ "${FAKE_FAIL:-}" = scorecard ] && exit 1 +exit 0 +S cat >"$T/bin/osv-scanner" <<'S' #!/usr/bin/env bash echo "osv-scanner $*" >>"$FAKE_LOG" @@ -60,10 +76,74 @@ bash "$SC" "$T/nowf" 2>&1 | grep -q '^INCOMPLETE:.*actionlint' && bad "no workfl expect "zizmor failure beats INCOMPLETE" 1 'FAIL zizmor' -- env PATH="$P" FAKE_FAIL=zizmor bash "$SC" "$T/repo" expect "actionlint failure fails the gate" 1 'FAIL actionlint' -- env PATH="$P" FAKE_FAIL=actionlint bash "$SC" "$T/repo" expect "scorecard failure fails the gate" 1 'FAIL scorecard' -- env PATH="$P" FAKE_FAIL=scorecard bash "$SC" --scorecard acme/widget "$T/repo" -expect "scorecard only with --scorecard" 3 'SKIPPED scorecard: pass --scorecard' -- env PATH="$P" bash "$SC" "$T/repo" +expect "scorecard on a non-git DIR without --scorecard is SKIPPED with the reason" 3 'SKIPPED scorecard: DIR is not a git repo' -- env PATH="$P" bash "$SC" "$T/repo" +: >"$FAKE_LOG"; env PATH="$P" bash "$SC" --allow-skip "$T/repo" >/dev/null 2>&1 +grep -q '^scorecard' "$FAKE_LOG" && bad "scorecard ran on a non-git DIR" || ok "scorecard not invoked on a non-git DIR" : >"$FAKE_LOG" expect "scorecard runs for a named repo" 0 'ok scorecard' -- env PATH="$P" bash "$SC" --scorecard acme/widget --allow-skip "$T/repo" grep -q 'scorecard --repo=github.com/acme/widget' "$FAKE_LOG" && ok "scorecard got repo arg" || bad "scorecard arg" grep -q '^actionlint' "$FAKE_LOG" && grep -q '^zizmor' "$FAKE_LOG" && ok "zizmor+actionlint ran" || bad "ci tools ran" expect "fully run, all clean: exit 0" 0 'ok scorecard' -- env PATH="$P" OSV_JSON='{"results":[]}' OSV_RC=0 bash "$SC" --online --scorecard acme/widget "$T/repo" + +# --- scorecard local mode: no --scorecard and DIR is a git repo -> offline subset on an export of HEAD --- +ROOTP=$(cd "$T/gitrepo" && pwd -P) +CHECKS=Binary-Artifacts,Dangerous-Workflow,Dependency-Update-Tool,License,Packaging,Pinned-Dependencies,SAST,Security-Policy,Token-Permissions +: >"$FAKE_LOG" +o=$(env PATH="$P" TMPDIR="$T/xtmp" bash "$SC" --allow-skip "$T/gitrepo" 2>&1) && r=0 || r=$? +[ "$r" = 0 ] && printf '%s' "$o" | grep -q 'ok scorecard (local export of HEAD in' && ok "git DIR without --scorecard: scorecard runs locally" || bad "local run (rc=$r): $o" +printf '%s' "$o" | grep -q "ok scorecard (local export of HEAD in $ROOTP;" && ok "root DIR: ok line names the repo root, no subdirectory note" || bad "root scope line: $o" +printf '%s' "$o" | grep -q 'partial view, not comparable with a public Scorecard score' && ok "ok line says local mode is a partial view" || bad "partial view wording: $o" +sl=$(grep '^scorecard ' "$FAKE_LOG" || true) +[ "$(printf '%s' "$sl" | sed -n 's/.*--checks=\([^ ]*\).*/\1/p')" = "$CHECKS" ] && ok "local mode passes exactly the offline check list" || bad "local check list: $sl" +printf '%s' "$sl" | grep -q 'Vulnerabilities\|Fuzzing' && bad "local mode asked for Vulnerabilities or Fuzzing" || ok "local mode never asks for Vulnerabilities or Fuzzing" +printf '%s' "$sl" | grep -q -- '--show-details' && ok "local mode shows details" || bad "no --show-details" +printf '%s' "$sl" | grep -q -- '--repo' && bad "local mode used the remote --repo" || ok "local mode makes no remote --repo call" +printf '%s' "$sl" | grep -q -- "--local=$T/xtmp/supply-chain-sc\." && ok "export lives under TMPDIR" || bad "export dir: $sl" +grep -qx 'scorecard-sees ./tracked.txt' "$FAKE_LOG" && ok "export holds tracked files" || bad "tracked file missing from export" +grep -q 'untracked.bin' "$FAKE_LOG" && bad "export leaked an untracked file" || ok "export has no untracked file" +[ -z "$(ls -A "$T/xtmp")" ] && ok "export removed after the run" || bad "export left behind" +: >"$FAKE_LOG" +expect "local scorecard failure fails the gate" 1 'FAIL scorecard' -- env PATH="$P" TMPDIR="$T/xtmp" FAKE_FAIL=scorecard bash "$SC" --allow-skip "$T/gitrepo" +[ -z "$(ls -A "$T/xtmp")" ] && ok "export removed after a failed run" || bad "export left behind after failure" +: >"$FAKE_LOG" +expect "subdirectory DIR scores the repo root and says so" 0 "ok scorecard (local export of HEAD in $ROOTP, DIR is a subdirectory so the whole repo is scored;" -- env PATH="$P" bash "$SC" --allow-skip "$T/gitrepo/sub" +grep -qx 'scorecard-sees ./tracked.txt' "$FAKE_LOG" && grep -qx 'scorecard-sees ./sub/inner.txt' "$FAKE_LOG" && ok "subdirectory DIR exports the whole repo" || bad "subdirectory export" +: >"$FAKE_LOG" +expect "git repo with no commit: scorecard SKIPPED" 3 'SKIPPED scorecard: git archive HEAD failed' -- env PATH="$P" TMPDIR="$T/xtmp" bash "$SC" "$T/nocommit" +grep -q '^scorecard' "$FAKE_LOG" && bad "scorecard ran on an empty export" || ok "scorecard not invoked without a commit" +[ -z "$(ls -A "$T/xtmp")" ] && ok "export removed after a skipped run" || bad "export left behind after skip" +expect "unwritable TMPDIR: scorecard SKIPPED, not run" 3 'SKIPPED scorecard: mktemp failed' -- env PATH="$P" TMPDIR="$T/no-such-dir" bash "$SC" "$T/gitrepo" +grep -q '^scorecard' "$FAKE_LOG" && bad "scorecard ran without an export" || ok "scorecard not invoked without an export dir" +expect "git DIR, scorecard not installed: SKIPPED not installed" 3 'SKIPPED scorecard: not installed' -- env PATH="$T/sys" /bin/bash "$SC" "$T/gitrepo" +: >"$FAKE_LOG" +expect "--scorecard in a git DIR stays remote" 0 'ok scorecard' -- env PATH="$P" bash "$SC" --scorecard acme/widget --allow-skip "$T/gitrepo" +grep -q -- '--repo=github.com/acme/widget' "$FAKE_LOG" && ! grep -q -- '--local' "$FAKE_LOG" && ok "--scorecard keeps the remote scan" || bad "remote scan in a git DIR" + +# --- interrupted scan: the export dir is a full copy of the tracked files, so a signal must not leave it behind --- +# The fake scorecard hangs after recording its pid; the script is signalled first, then the fake is stopped (our own pid only). +# SIGINT is not testable here: a non-interactive shell starts background jobs with it ignored. +for sig in TERM HUP; do + env PATH="$P" TMPDIR="$T/xtmp" FAKE_HANG="$T/hang.$sig" bash "$SC" --allow-skip "$T/gitrepo" >/dev/null 2>&1 & spid=$! + for _ in $(seq 50); do [ -s "$T/hang.$sig" ] && break; sleep 0.1; done + [ -n "$(ls -A "$T/xtmp")" ] || bad "SIG$sig: no export dir during the scan (fixture broke)" + kill -"$sig" "$spid" 2>/dev/null || true + kill "$(cat "$T/hang.$sig")" 2>/dev/null || true + wait "$spid" 2>/dev/null || true + [ -z "$(ls -A "$T/xtmp")" ] && ok "export removed after SIG$sig mid-scan" || bad "export left behind after SIG$sig" +done + +# --- partial (blobless) clone: git archive would lazy-fetch missing blobs from the promisor remote (network) --- +# The promisor here is a local file:// repo, so old code succeeds by fetching; the fixed script must refuse and skip instead. +mkdir "$T/psrc"; echo data >"$T/psrc/f.txt"; g -C "$T/psrc" init -q; g -C "$T/psrc" add -A; g -C "$T/psrc" commit -qm src +g -C "$T/psrc" config uploadpack.allowFilter true; g -C "$T/psrc" config uploadpack.allowAnySHA1InWant true +g clone -q --no-checkout --filter=blob:none "file://$T/psrc" "$T/partial" +if GIT_NO_LAZY_FETCH=1 git -C "$T/partial" cat-file -e HEAD:f.txt 2>/dev/null; then + printf 'skip partial-clone tests: this git ignores GIT_NO_LAZY_FETCH\n' +else + : >"$FAKE_LOG" + expect "partial clone missing blobs: scorecard SKIPPED, names the partial clone" 3 'SKIPPED scorecard: .*partial clone' -- env PATH="$P" TMPDIR="$T/xtmp" bash "$SC" "$T/partial" + grep -q '^scorecard' "$FAKE_LOG" && bad "scorecard ran on a partial export" || ok "scorecard not invoked on a partial clone" + GIT_NO_LAZY_FETCH=1 git -C "$T/partial" cat-file -e HEAD:f.txt 2>/dev/null && bad "the scan lazy-fetched a blob" || ok "no blob was lazy-fetched (stays offline)" + [ -z "$(ls -A "$T/xtmp")" ] && ok "export removed after a partial-clone skip" || bad "export left behind after partial-clone skip" +fi exit "$fail" diff --git a/scripts/test_app_pentest.sh b/scripts/test_app_pentest.sh index 2600b5b1e..eb2b020b6 100755 --- a/scripts/test_app_pentest.sh +++ b/scripts/test_app_pentest.sh @@ -1,15 +1,16 @@ #!/usr/bin/env bash # Tests for app_pentest.sh: refusal logic + output parsing, with fake scanners on PATH. -# No real network: targets are localhost or numeric literals (resolved without DNS); fakes only log and print. +# Loopback only: targets are localhost or numeric literals (resolved without DNS); fakes only log and print. Toy APIs +# on 127.0.0.1 serve spec URLs and, when schemathesis is installed, take real wire-level runs (incl. the egress block). set -u ROOT="$(cd "$(dirname "$0")/.." && pwd)" S="$ROOT/.claude/skills/deep-code-review/scripts/app_pentest.sh" -T="$(mktemp -d "${TMPDIR:-/tmp}/ap.XXXXXX")" || exit 1; trap 'rm -rf "$T"' EXIT +T="$(mktemp -d "${TMPDIR:-/tmp}/ap.XXXXXX")" || exit 1; pids=""; trap 'kill $pids 2>/dev/null; rm -rf "$T"' EXIT fails=0; n=0 ok() { n=$((n+1)); if [ "$1" = 0 ]; then :; else fails=$((fails+1)); echo "FAIL: $2"; fi; } # --- fake scanners; each logs its own name ($0) and argv so a refused target proves no tool ran --- -mkdir "$T/fake" "$T/empty" +mkdir "$T/fake" "$T/empty" "$T/home" cat >"$T/fake/nuclei" <<'EOF' #!/bin/sh echo "$(basename "$0") $@" >>"$CALLS" @@ -25,13 +26,28 @@ echo '{"site":[{"@name":"http://localhost:3000","alerts":[{"pluginid":"10038","a exit 2 EOF cp "$T/fake/zap-baseline.py" "$T/fake/zap-full-scan.py" +# schemathesis: logs argv (a --help probe too, which it answers like the real one), records its cwd, proxy environment +# and the spec it was handed, writes the canned report +cat >"$T/fake/schemathesis" <<'EOF' +#!/bin/sh +echo "$(basename "$0") $@" >>"$CALLS" +[ "$1" = --help ] && { echo "Usage: $(basename "$0") [OPTIONS] COMMAND [ARGS]..."; echo " Property-based API testing for OpenAPI and GraphQL."; exit 0; } +env | grep -i _proxy >"$ST_ENV" +pwd -P >"$ST_CWD" +while [ $# -gt 0 ]; do case "$1" in run) s="$2" ;; --report-json-path) j="$2" ;; esac; shift; done +cp "$s" "$ST_SPEC" +[ -f "$ST_REPORT" ] && cp "$ST_REPORT" "$j" +exit 1 +EOF chmod +x "$T/fake/"* -export CALLS="$T/calls"; : >"$CALLS" +mkdir "$T/fake-st" "$T/home2" "$T/home2/.local" "$T/home2/.local/bin" +cp "$T/fake/schemathesis" "$T/fake-st/st"; cp "$T/fake/schemathesis" "$T/home2/.local/bin/schemathesis" +export CALLS="$T/calls" ST_CWD="$T/st.cwd" ST_SPEC="$T/st.spec" ST_REPORT="$T/st-report.json" ST_ENV="$T/st.env"; : >"$CALLS" unset APP_PENTEST_ALLOW -run() { env PATH="$T/fake:/usr/bin:/bin" "$@"; } -refused() { # refused DESC [ENV=VAL] -- args...: exit 2, REFUSED on stderr, empty stdout +run() { env HOME="$T/home" PATH="$T/fake:/usr/bin:/bin" "$@"; } +refused() { # refused DESC cmd...: exit 2, empty stdout, REFUSED on stderr (naming $WANT when set) d="$1"; shift; out="$(run "$@" 2>"$T/err")"; rc=$? - [ $rc -eq 2 ] && [ -z "$out" ] && grep -q '^REFUSED' "$T/err"; ok $? "refuse $d (rc=$rc)" + [ $rc -eq 2 ] && [ -z "$out" ] && grep -q "^REFUSED.*${WANT:-}" "$T/err"; ok $? "refuse $d (rc=$rc)" } # --- refusals --- @@ -47,6 +63,28 @@ APP_PENTEST_ALLOW="nohost.invalid" refused "unresolvable allowlisted host" bash refused "--active on localhost without allowlist" bash "$S" --active http://localhost:3000 APP_PENTEST_ALLOW="staging.example.test" refused "--active off-list" bash "$S" --active http://evil.com APP_PENTEST_ALLOW="93.184.216.34" refused "--active on public IP even if allowlisted" bash "$S" --active http://93.184.216.34 +# --openapi: an unreadable or over-size spec, or a remote $ref/externalValue in its text, refuses (exit 2) before a tool starts +printf '{"openapi":"3.0.3","info":{"title":"t","version":"1"},"paths":{}}' >"$T/spec.json" +printf '{"a":{"$ref":"http://example.com/x.json"}}' >"$T/ref1.json" +printf 'a:\n $ref: https://example.com/x.yaml\n' >"$T/ref2.yaml" +printf '{"a":{"$ref": "//example.com/x.json"}}' >"$T/ref3.json" +printf '{"a":{"externalValue":"http://example.com/x.json"}}' >"$T/ref4.json" +head -c 10000001 /dev/zero >"$T/big.json" +WANT="usage" refused "URL then --openapi" bash "$S" http://localhost --openapi "$T/spec.json" +WANT="--openapi needs" refused "--openapi without a value" bash "$S" --openapi +WANT="is not a file" refused "missing spec file" bash "$S" --openapi "$T/nope.json" http://localhost +WANT="is not a file" refused "spec path is a directory" bash "$S" --openapi "$T" http://localhost +WANT="spec URL must start" refused "ftp spec URL" bash "$S" --openapi ftp://localhost/s.json http://localhost +WANT="spec URL must start" refused "file:// spec URL" bash "$S" --openapi file:///etc/hosts http://localhost +WANT="--openapi: host 'evil.example'" refused "spec URL on a non-allowed host" bash "$S" --openapi http://evil.example/openapi.json http://localhost +WANT="--openapi: host 'localhost.evil.example'" refused "spec URL lookalike host" bash "$S" --openapi http://localhost.evil.example/s.json http://localhost +WANT="--openapi: userinfo" refused "spec URL userinfo" bash "$S" --openapi http://localhost@evil.example/s.json http://localhost +WANT="--openapi: bad port" refused "spec URL bad port" bash "$S" --openapi http://localhost:80a/s.json http://localhost +APP_PENTEST_ALLOW="nohost.invalid" WANT="--openapi: cannot resolve" refused "unresolvable allowlisted spec host" bash "$S" --openapi http://nohost.invalid/s.json http://localhost +APP_PENTEST_ALLOW="localhost" WANT="--openapi: --active needs host '127.0.0.1' listed" refused "--active, spec host not listed" bash "$S" --active --openapi http://127.0.0.1/s.json http://localhost +APP_PENTEST_ALLOW="localhost,93.184.216.34" WANT="--openapi: --active needs loopback/private" refused "--active, spec on a public IP" bash "$S" --active --openapi http://93.184.216.34/s.json http://localhost +for f in ref1.json ref2.yaml ref3.json ref4.json; do WANT='remote \$ref or externalValue' refused "remote reference in $f" bash "$S" --openapi "$T/$f" http://localhost; done +WANT="over 10 MB" refused "oversized spec" bash "$S" --openapi "$T/big.json" http://localhost [ ! -s "$CALLS" ]; ok $? "no scanner ran for any refused target" # --- accepted targets, passive: parsing + flags --- @@ -60,14 +98,15 @@ run bash "$S" http://localhost:3000 >"$T/o.json" python3 -I - "$T/o.json" <<'PY' import json, sys d = json.load(open(sys.argv[1])) -assert d["tools"] == {"zap": "ran", "nuclei": "ran"}, d["tools"] -assert d["active"] is False and d["ips"], d +assert d["tools"] == {"zap": "ran", "nuclei": "ran", "schemathesis": "skipped"}, d["tools"] +assert d["active"] is False and d["ips"] and d["openapi"] is None, d ids = {(l["tool"], l["id"]): l for l in d["leads"]} assert set(ids) == {("zap", "10038"), ("nuclei", "exposed-panel")}, ids assert ids[("zap", "10038")]["severity"] == "medium" and ids[("nuclei", "exposed-panel")]["url"].endswith("/admin") assert all(l["verified"] is False for l in d["leads"]) and "LEADS" in d["note"] PY ok $? "JSON leads parsed, unverified, ips printed, bad nuclei line skipped, zap exit 2 tolerated" +! grep -q '^schemathesis' "$CALLS"; ok $? "no --openapi: schemathesis not run even when installed" grep -q '^nuclei .*-rl 10 ' "$CALLS" && grep -q -- ' -ni ' "$CALLS" && grep -q -- ' -fhr ' "$CALLS" \ && grep -q -- '-etags dos,fuzz,bruteforce,intrusive' "$CALLS"; ok $? "nuclei rate-limited, no interactsh, same-host redirects, intrusive excluded" grep -q '^zap-baseline.py -t http://localhost:3000 -I -J /.*/zap.json' "$CALLS" && ! grep -q 'zap-full-scan' "$CALLS"; ok $? "passive: zap baseline only, absolute report path" @@ -80,10 +119,184 @@ grep -q '^zap-full-scan.py ' "$CALLS" && ! grep -q 'zap-baseline' "$CALLS"; ok $ grep -q 'threadPerHost=2' "$CALLS" && grep -q 'delayInMs=100' "$CALLS" && grep -q -- '-T 10' "$CALLS"; ok $? "active zap throttled" grep -q '^nuclei ' "$CALLS" && ! grep -q 'etags' "$CALLS" && grep -q -- '-rl 10' "$CALLS"; ok $? "active lifts nuclei exclusion, keeps rate limit" +# --- --openapi: schemathesis behind a fake binary (the report decides ran/error) --- +F='[{"type":"ServerError","code":"ST101","title":"Server error","severity":"critical","count":1,"operations":["GET /boom"]},{"type":"AuthError","code":"ST401","title":"Authentication not enforced","severity":"high","count":1,"operations":["GET /me"]},{"type":"JsonSchemaError","code":"ST108","title":"Response violates schema","severity":"high","count":2,"operations":["GET /contract","HEAD /contract"]},{"type":"Future","code":"ST999","title":"Future check","severity":"low","count":1,"operations":["GET /x"]}]' +rep() { printf '{"stop_reason":"%s","complete":%s,"exit_code":1,"operations":{"tested":%s,"errored":%s},"failures":%s,"errors":[]}' "$1" "$2" "$3" "${5:-0}" "${4:-[]}" >"$ST_REPORT"; } +has() { printf '%s\n' "$st" | grep -q -e "$1"; } +sts() { env HOME="${2:-$T/home}" PATH="${1:-$T/fake}:/usr/bin:/bin" bash "$S" --openapi "$T/spec.json" http://localhost:3000 2>/dev/null \ + | python3 -I -c 'import json,sys; d=json.load(sys.stdin); print(d["tools"]["schemathesis"], *[l["id"]+":"+l["severity"] for l in d["leads"] if l["tool"]=="schemathesis"])'; } +rep completed true 3 "$F"; : >"$CALLS" +run bash "$S" --openapi "$T/spec.json" http://localhost:3000 >"$T/o.json" +st="$(grep '^schemathesis ' "$CALLS")" +[ "$(printf '%s\n' "$st" | grep -o -e '--include-method [A-Z]*' | tr '\n' ' ')" = "--include-method GET --include-method HEAD " ] \ + && has '--rate-limit 5/s' && has ' -n 10 ' && has '--phases examples,fuzzing'; ok $? "passive: GET/HEAD only, rate-limited, small budget, no coverage or stateful phase" +has '^schemathesis --config-file /[^ ]*/st.toml run '"$T/spec.json"' ' && has ' --origin http://localhost:3000 ' \ + && [ "$(cat "$ST_CWD")" != "$(pwd -P)" ]; ok $? "pinned config, validated origin, the spec path, run outside the caller's cwd" +python3 -I - "$T/o.json" "$T/spec.json" <<'PY' +import json, sys +d = json.load(open(sys.argv[1])) +assert d["tools"] == {"zap": "ran", "nuclei": "ran", "schemathesis": "ran"}, d["tools"] +assert d["openapi"] == {"spec": sys.argv[2], "ips": [], "tested": 3, "errored": 0}, d["openapi"] +sl = [l for l in d["leads"] if l["tool"] == "schemathesis"] +assert [(l["id"], l["severity"], l["title"]) for l in sl] == [ + ("ST101", "medium", "Server error: GET /boom"), ("ST401", "medium", "Authentication not enforced: GET /me"), + ("ST108", "low", "Response violates schema: GET /contract"), ("ST108", "low", "Response violates schema: HEAD /contract"), + ("ST999", "low", "Future check: GET /x")], sl +assert all(l["verified"] is False and l["url"] == "http://localhost:3000" for l in sl) +assert {l["tool"] for l in d["leads"]} == {"zap", "nuclei", "schemathesis"} +PY +ok $? "schemathesis leads: 5xx and unenforced auth medium, contract and unknown codes low, unverified; openapi reported" +: >"$CALLS" +APP_PENTEST_ALLOW=localhost run bash "$S" --active --openapi "$T/spec.json" http://localhost:3000 >"$T/o.json" +st="$(grep '^schemathesis ' "$CALLS")" +! has '--include-method' && has '--rate-limit 5/s' && has ' -n 10 ' && has '--phases examples,fuzzing'; ok $? "--active: no method filter (all methods), still rate-limited and budgeted" +python3 -I -c 'import json,sys; d=json.load(open(sys.argv[1])); assert d["active"] is True and d["tools"]["schemathesis"]=="ran"' "$T/o.json"; ok $? "--active with --openapi runs schemathesis" +APP_PENTEST_ALLOW=localhost run bash "$S" --openapi "$T/spec.json" --active http://localhost:3000 | python3 -I -c 'import json,sys; assert json.load(sys.stdin)["active"] is True'; ok $? "--openapi and --active in either order" +: >"$CALLS"; (cd "$T" && run bash "$S" --openapi spec.json http://localhost:3000 >/dev/null) +grep -q "^schemathesis .* run $T/spec.json " "$CALLS"; ok $? "a relative spec path is made absolute before schemathesis changes directory" +rep server_unavailable true 1; [ "$(sts)" = "ran server_unavailable:medium" ]; ok $? "server stopped responding: ran plus a medium crash lead" +rep completed true 0; [ "$(sts)" = "error" ]; ok $? "nothing tested (unreachable target): error" +rep interrupted false 2; [ "$(sts)" = "error" ]; ok $? "incomplete report: error" +printf 'not json' >"$ST_REPORT"; [ "$(sts)" = "error" ]; ok $? "unparseable report: error" +rm -f "$ST_REPORT"; [ "$(sts)" = "error" ]; ok $? "no report: error" +rep completed true 3 "[]" 2; run bash "$S" --openapi "$T/spec.json" http://localhost:3000 >"$T/o.json" +python3 -I -c 'import json,sys; d=json.load(open(sys.argv[1])); assert d["tools"]["schemathesis"]=="ran" and (d["openapi"]["tested"], d["openapi"]["errored"])==(3, 2), d' "$T/o.json"; ok $? "partial coverage: ran, with the tested and errored operation counts reported" +rep completed true 3 +# egress block: whatever proxy the caller has set, schemathesis gets a dead proxy for everything but the target host +run env http_proxy=http://proxy.example:3128 HTTPS_PROXY=http://proxy.example:3128 NO_PROXY='*' bash "$S" --openapi "$T/spec.json" http://LOCALHOST:3000 >/dev/null +e=1; for v in http_proxy HTTP_PROXY https_proxy HTTPS_PROXY all_proxy ALL_PROXY; do grep -qx "$v=http://127.0.0.1:1" "$ST_ENV" || e=0; done +[ $e = 1 ] && grep -qx 'no_proxy=localhost' "$ST_ENV" && grep -qx 'NO_PROXY=localhost' "$ST_ENV" && ! grep -q 'proxy.example' "$ST_ENV"; ok $? "schemathesis env: every proxy variable dead, no_proxy only the target host, caller's proxy gone" +: >"$CALLS"; [ "$(sts "$T/fake-st")" = skipped ] && sts >/dev/null && ! grep -q '^st ' "$CALLS" && ! grep -q -e ' --help' "$CALLS"; ok $? "only schemathesis is looked up (not st) and it is never probed with --help" +[ "$(sts "$T/empty" "$T/home2" | cut -d' ' -f1)" = ran ]; ok $? "falls back to ~/.local/bin/schemathesis" +[ "$(sts "$T/empty")" = skipped ]; ok $? "no binary at all: skipped" + +# --- spec URL: fetched once without following redirects, schemathesis gets the local copy; real binary if installed --- +cat >"$T/srv.py" <<'PY' +import http.server, json, sys +OP = {"responses": {"200": {"description": "ok"}}} +SPEC = {"openapi": "3.0.3", "info": {"title": "toy", "version": "1"}, + "paths": {"/ok": {"get": OP}, "/items": {"post": OP}, + "/items/{id}": {"delete": {**OP, "parameters": [{"name": "id", "in": "path", "required": True, "schema": {"type": "integer"}}]}}}} +log = open(sys.argv[1], "a", buffering=1) +class H(http.server.BaseHTTPRequestHandler): + def log_message(self, *a): pass + def parse_request(self): + ok = super().parse_request() + log.write(self.requestline + "\n") # every request line, also for methods this toy does not implement + return ok + def serve(self): + if self.path == "/redir": + self.send_response(302); self.send_header("Location", "/openapi.json"); self.end_headers(); return + if self.path == "/gone": self.send_error(404); return + if self.path == "/big.json": # 64 MiB in 64 KiB chunks; logs how much it got out before the client hung up + self.send_response(200); self.end_headers(); sent = 0 + try: + for _ in range(1024): self.wfile.write(b" " * 65536); sent += 65536 + except OSError: pass + log.write("big sent=%d\n" % sent); return + body = json.dumps(SPEC if self.path == "/openapi.json" else {}).encode() + self.send_response(200); self.send_header("Content-Type", "application/json"); self.send_header("Content-Length", str(len(body))); self.end_headers() + if self.command != "HEAD": self.wfile.write(body) + do_GET = do_HEAD = do_POST = do_PUT = do_PATCH = do_DELETE = do_OPTIONS = do_TRACE = serve +s = http.server.ThreadingHTTPServer(("127.0.0.1", 0), H) +print(s.server_address[1], flush=True) +s.serve_forever() +PY +toy() { # toy LOG VAR: a toy API on 127.0.0.1 appending each request line to LOG; its port goes in $VAR (the EXIT trap kills it by PID) + python3 -I -u "$T/srv.py" "$1" >"$1.port" 2>/dev/null & pids="$pids $!"; disown + for _ in 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20; do [ -s "$1.port" ] && break; sleep 0.25; done + printf -v "$2" %s "$(cat "$1.port")"; [ -n "${!2}" ] +} +toy "$T/wire.log" port; ok $? "toy API started on 127.0.0.1" +: >"$CALLS"; : >"$T/st.spec" +run bash "$S" --openapi "http://127.0.0.1:$port/openapi.json" http://localhost:3000 >"$T/o.json" +python3 -I - "$T/o.json" "$T/st.spec" <<'PY' +import json, sys +d = json.load(open(sys.argv[1])) +assert d["tools"]["schemathesis"] == "ran" and d["openapi"]["ips"] == ["127.0.0.1"], d["openapi"] +assert json.load(open(sys.argv[2]))["openapi"] == "3.0.3" +PY +ok $? "spec URL: fetched, schemathesis handed the copy, the spec host's addresses reported" +grep -q '^schemathesis .* run /[^ ]*/spec ' "$CALLS" && ! grep -q ' run http' "$CALLS"; ok $? "schemathesis never gets the spec URL (its own fetch follows redirects)" +grep -qx 'no_proxy=localhost' "$ST_ENV"; ok $? "no_proxy names the target host, not the spec URL's host" +c0="$(grep -c '^GET /openapi.json' "$T/wire.log")" +WANT="cannot read spec: HTTP Error 302" refused "redirecting spec URL" bash "$S" --openapi "http://127.0.0.1:$port/redir" http://localhost:3000 +[ "$(grep -c '^GET /openapi.json' "$T/wire.log")" = "$c0" ]; ok $? "the redirect target was never requested" +WANT="cannot read spec: HTTP Error 404" refused "missing spec URL" bash "$S" --openapi "http://127.0.0.1:$port/gone" http://localhost:3000 +# the spec fetch ignores env proxies (a dead one, no_proxy cleared) and reads at most 10 MB + 1 byte of a URL body +: >"$T/st.spec" +run env -u NO_PROXY -u no_proxy http_proxy=http://127.0.0.1:1 HTTP_PROXY=http://127.0.0.1:1 bash "$S" --openapi "http://127.0.0.1:$port/openapi.json" http://localhost:3000 >/dev/null 2>&1 +[ -s "$T/st.spec" ]; ok $? "spec URL fetched directly although a dead http_proxy is set" +WANT="over 10 MB" refused "oversized spec URL" bash "$S" --openapi "http://127.0.0.1:$port/big.json" http://localhost:3000 +for _ in $(seq 20); do grep -q '^big sent=' "$T/wire.log" && break; sleep 0.25; done +sent="$(sed -n 's/^big sent=//p' "$T/wire.log")"; [ "${sent:-99999999999}" -lt 48000000 ]; ok $? "spec URL read is capped: the server got out ${sent:-nothing} of 67108864 bytes" +# egress proof inputs: the remote reference, on localhost:PORT (the sink), is hidden from the early text match by a +# JSON unicode escape or a YAML tag, so only the egress block can stop the fetch +mkspecs() { python3 -I - "$T" "$1" <<'PY' +import json, sys +t, port = sys.argv[1:3] +bs = chr(92) +ref = "http://localhost:%s/s.json" % port +def spec(**op): return json.dumps({"openapi": "3.0.3", "info": {"title": "t", "version": "1"}, "paths": {"/go": {"get": {"responses": {"200": {"description": "ok"}}, **op}}}}) +def q(**p): return {"parameters": [{"name": "q", "in": "query", "required": True, **p}]} +tag = """openapi: 3.0.3 +info: {title: t, version: "1"} +paths: + /go: + get: + parameters: + - name: q + in: query + required: true + schema: + $ref: !!str %s + responses: + "200": {description: ok} +""" % ref +files = {"esc.json": spec(**q(schema={"$ref": ref})).replace("$ref", bs + "u0024ref"), + "ext.json": spec(**q(schema={"type": "string"}, examples={"a": {"externalValue": ref}})).replace("externalValue", "external" + bs + "u0056alue"), + "tag.yaml": tag, "plain.json": spec()} +for n, text in files.items(): open("%s/h.%s" % (t, n), "w").write(text) +PY +} +real="$(command -v schemathesis 2>/dev/null)"; [ -n "$real" ] || real="$HOME/.local/bin/schemathesis" +if [ -x "$real" ]; then + w() { env HOME="$T/home" PATH="$(dirname "$real"):/usr/bin:/bin" "$@"; } + n0="$(wc -l <"$T/wire.log")" + w bash "$S" --openapi "http://127.0.0.1:$port/openapi.json" "http://127.0.0.1:$port" >"$T/o.json" 2>/dev/null + tail -n +$((n0 + 1)) "$T/wire.log" >"$T/wire.p" + python3 -I -c 'import json,sys; assert json.load(open(sys.argv[1]))["tools"]["schemathesis"]=="ran"' "$T/o.json" \ + && grep -q '^GET /ok ' "$T/wire.p" && ! grep -Evq '^(GET|HEAD) ' "$T/wire.p"; ok $? "real schemathesis, passive: ran, only GET/HEAD reached the server" + n0="$(wc -l <"$T/wire.log")" + APP_PENTEST_ALLOW=127.0.0.1 w bash "$S" --active --openapi "http://127.0.0.1:$port/openapi.json" "http://127.0.0.1:$port" >"$T/o.json" 2>/dev/null + tail -n +$((n0 + 1)) "$T/wire.log" >"$T/wire.a" + grep -Eq '^(POST|DELETE) ' "$T/wire.a"; ok $? "real schemathesis, --active: the declared POST/DELETE operations are fuzzed" + # Egress block, wire level. Target 127.0.0.1 (A), hidden remote references and the user's proxy on other servers (B, P). + # Each control runs schemathesis directly, with no block: it must reach B or P, else the proof below it is vacuous. + toy "$T/b.log" pb && toy "$T/p.log" pp; ok $? "sink and user-proxy toys started"; mkspecs "$pb" + unp=(-u http_proxy -u HTTP_PROXY -u https_proxy -u HTTPS_PROXY -u all_proxy -u ALL_PROXY -u no_proxy -u NO_PROXY) + direct() { : >"$T/st.toml"; (cd "$T" && env "${unp[@]}" HOME="$T/home" "${@:2}" "$real" --config-file "$T/st.toml" run "$1" --origin "http://127.0.0.1:$port" \ + --phases examples,fuzzing -n 10 -w 1 --no-shrink --max-redirects 0 --request-timeout 10 >/dev/null 2>&1); } + fresh() { : >"$T/wire.log"; : >"$T/b.log"; : >"$T/p.log"; } + cnt() { wc -l <"$1" | tr -d ' '; } + for x in esc.json ext.json tag.yaml; do + fresh; direct "$T/h.$x" + [ "$(cnt "$T/b.log")" -ge 1 ]; ok $? "control, no block: the hidden remote reference in $x is fetched (B=$(cnt "$T/b.log"))" + fresh; w bash "$S" --openapi "$T/h.$x" "http://127.0.0.1:$port" >/dev/null 2>&1 + [ "$(cnt "$T/b.log")" = 0 ] && [ "$(cnt "$T/wire.log")" -ge 1 ]; ok $? "egress block: the hidden remote reference in $x gets 0 requests, the target is still hit (B=$(cnt "$T/b.log") A=$(cnt "$T/wire.log"))" + done + fresh; direct "$T/h.plain.json" http_proxy="http://127.0.0.1:$pp" HTTP_PROXY="http://127.0.0.1:$pp" + [ "$(cnt "$T/p.log")" -ge 1 ] && [ "$(cnt "$T/wire.log")" = 0 ]; ok $? "control, no block: a user http_proxy diverts the traffic (P=$(cnt "$T/p.log") A=$(cnt "$T/wire.log"))" + fresh; w env "${unp[@]}" http_proxy="http://127.0.0.1:$pp" HTTP_PROXY="http://127.0.0.1:$pp" bash "$S" --openapi "$T/h.plain.json" "http://127.0.0.1:$port" >/dev/null 2>&1 + [ "$(cnt "$T/p.log")" = 0 ] && [ "$(cnt "$T/wire.log")" -ge 1 ]; ok $? "egress block: a user http_proxy gets 0 requests, the target is hit directly (P=$(cnt "$T/p.log") A=$(cnt "$T/wire.log"))" +else + echo "app_pentest: real schemathesis run skipped (not installed)" +fi + # --- tools missing -> skipped (only when the host has none in /usr/bin:/bin) --- if ! PATH=/usr/bin:/bin command -v nuclei >/dev/null && ! PATH=/usr/bin:/bin command -v zap-baseline.py >/dev/null; then env PATH="$T/empty:/usr/bin:/bin" bash "$S" http://localhost >"$T/o.json" - python3 -I -c 'import json,sys; d=json.load(open(sys.argv[1])); assert d["tools"]=={"zap":"skipped","nuclei":"skipped"} and d["leads"]==[]' "$T/o.json"; ok $? "missing tools reported skipped" + python3 -I -c 'import json,sys; d=json.load(open(sys.argv[1])); assert d["tools"]=={"zap":"skipped","nuclei":"skipped","schemathesis":"skipped"} and d["leads"]==[]' "$T/o.json"; ok $? "missing tools reported skipped" fi # --- tool crash -> error, still valid JSON --- diff --git a/scripts/test_audit_pr_budget.py b/scripts/test_audit_pr_budget.py new file mode 100644 index 000000000..5d3db312b --- /dev/null +++ b/scripts/test_audit_pr_budget.py @@ -0,0 +1,123 @@ +#!/usr/bin/env python3 +"""comms_lint audit-pr budget: the table budget (1200 words, 160 lines, fail above 1.5x) counts everything outside +`## Changes by area`; each `### ` area there fails above 200 words and the whole body above 3000. A body written to the +template with many areas must pass, and the reference states the same numbers. Synthetic input (Acme); no network.""" +import sys +import unittest +from pathlib import Path + +ROOT = Path(__file__).resolve().parent.parent +sys.path.insert(0, str(ROOT / ".claude/skills/communication-structure/scripts")) +import comms_lint as L # noqa: E402 + +REFERENCE = ROOT / ".claude/skills/communication-structure/references/artifact-templates.md" + + +def words(n): + return " ".join(["plain"] * n) + + +def body(areas=(), bullets=(), deps=""): + """An audit-pr body in the template's shape. `areas`: a word count per `### Area N` heading; `bullets`: a word count per + `- **Area N:**` bullet (the template's own form); `deps`: text for `## Dependency upgrades` (outside the areas section).""" + out = ["## Summary", "Adds sign-in checks to the Acme export routes. Please review the new limit.", + "## What changes for reviewers and users", "- GET /export now needs sign-in.", "## Changes by area"] + out += [f"### Area {i}\n{words(n)}" for i, n in enumerate(areas)] + out += [f"- **Area {i}:** {words(n)}" for i, n in enumerate(bullets)] + out += ["## Dependency upgrades", deps] if deps else [] + out += ["## How to verify", "✅ `npm test` on this head: 42/42 pass.", "## Not fixed", "2 high, 5 medium, tracked privately."] + return "\n".join(out) + "\n" + + +def hits(res, rule): + return [h for r in res["rules"] if r["rule"] == rule for h in r["hits"]] + + +def bullets_of(n_lines, n_words=30): + return "\n".join("- " + words(n_words) for _ in range(n_lines)) + + +class Budget(unittest.TestCase): + def test_many_areas_pass_though_the_body_is_over_the_table_budget(self): + text = body(areas=[190] * 10) + self.assertGreater(len(text.split()), 1800) # over 1.5 x 1200: the earlier rule failed this + res = L.lint("audit-pr", text) + self.assertEqual(hits(res, "budget"), []) + self.assertTrue(res["ok"]) + + def test_bullet_areas_of_the_template_are_outside_the_budget_too(self): + res = L.lint("audit-pr", body(bullets=[250] * 8)) # eight areas, 2000+ words + self.assertEqual(hits(res, "budget"), []) + self.assertTrue(res["ok"]) + + def test_an_area_over_the_cap_fails_at_its_heading(self): + text = body(areas=[150, 201, 150]) + res = L.lint("audit-pr", text) + (h,) = hits(res, "budget") + self.assertEqual(h["line"], text.splitlines().index("### Area 1") + 1) + self.assertIn("area 'Area 1' is 201 words against 200", h["problem"]) + self.assertIn(h["problem"], L.render(res)) + self.assertFalse(res["ok"]) + + def test_an_area_at_the_cap_passes(self): + self.assertEqual(hits(L.lint("audit-pr", body(areas=[200])), "budget"), []) + + def test_whole_body_hard_cap(self): + base = body(areas=[190] * 15) + + def total(n): # a body of exactly n words: a last area of the right length, 200 words or fewer + return body(areas=[190] * 15 + [n - len(base.split()) - 3]) + + for n, fails in ((3000, False), (3001, True)): + text = total(n) + self.assertEqual(len(text.split()), n) + res = L.lint("audit-pr", text) + self.assertEqual([h["problem"] for h in hits(res, "budget")], [f"{n} words against the 3000 hard cap for audit-pr"] * fails) + self.assertEqual(res["ok"], not fails) + + def test_outside_the_areas_section_the_table_budget_still_applies(self): + res = L.lint("audit-pr", body(areas=[100], deps=bullets_of(60))) # about 1900 words outside the section + (h,) = hits(res, "budget") + self.assertIn("outside `## Changes by area` against 1200 / 160 for audit-pr", h["problem"]) + self.assertFalse(res["ok"]) + + def test_between_the_table_value_and_1_5x_outside_it_only_warns(self): + res = L.lint("audit-pr", body(areas=[100], deps=bullets_of(50))) # about 1600 words outside the section + self.assertEqual(hits(res, "budget"), []) + self.assertTrue(hits(res, "length")) + self.assertTrue(res["ok"]) + + def test_without_the_areas_section_the_whole_body_is_counted(self): + res = L.lint("audit-pr", body(areas=[100], deps=bullets_of(60)).replace("## Changes by area", "## Elsewhere")) + (h,) = hits(res, "budget") + self.assertNotIn("outside", h["problem"]) + + def test_fenced_lines_neither_count_nor_split_an_area(self): + text = body(areas=[150]).replace("### Area 0\n", "### Area 0\n" + words(150) + "\n```\n### Not an area\n" + words(300) + "\n```\n") + (h,) = hits(L.lint("audit-pr", text), "budget") + self.assertIn("area 'Area 0' is 300 words", h["problem"]) # 150 + 150 prose words; the fenced 300 and its heading are not counted + + def test_a_deeper_heading_stays_in_its_area(self): + text = body(areas=[120]).replace("### Area 0\n", "### Area 0\n" + words(120) + "\n#### Detail\n") + (h,) = hits(L.lint("audit-pr", text), "budget") + self.assertIn("area 'Area 0' is 242 words", h["problem"]) + + def test_reported_words_and_lines_are_what_the_budget_counts(self): + res = L.lint("audit-pr", body(areas=[190] * 10)) + self.assertLess(res["words"], 100) + self.assertLess(res["lines"], 20) + self.assertIn(f"{res['words']}/1200 words, {res['lines']}/160 lines", L.render(res)) + msg = "Cleanup now runs hourly. Review by Friday 2026-10-30." # other kinds count the whole body + self.assertEqual(L.lint("message", msg)["words"], len(msg.split())) + + def test_reference_states_the_scope_and_the_caps(self): + doc = REFERENCE.read_text(encoding="utf-8") + para = doc[doc.index("`audit-pr` is the body"):doc.index("## Templates")] + for want in ("outside `Changes by area`", f"{L.AREA_WORDS} words", f"{L.TOTAL_WORDS} words"): + self.assertIn(want, para) + self.assertEqual((L.AREA_WORDS, L.TOTAL_WORDS), (200, 3000)) + self.assertEqual(L.load_kinds()["audit-pr"][:2], (1200, 160)) + + +if __name__ == "__main__": + unittest.main() diff --git a/scripts/test_committed_personal_data.py b/scripts/test_committed_personal_data.py new file mode 100644 index 000000000..c93cea038 --- /dev/null +++ b/scripts/test_committed_personal_data.py @@ -0,0 +1,325 @@ +#!/usr/bin/env python3 +"""Tests: committed_personal_data.py counts distinct non-reserved email addresses per tracked file (CSV and binary-ish alike), reports path and count only and never an address, drops reserved, noreply, git@ and asset-scale names unless --include-reserved, honors the threshold edge, --strict, --stdin and the size cap, reports the repository total of addresses spread thin over files that each stay below the threshold, gives a file in a recognised compressed or columnar format under the threshold a not-checked row (never a silent pass; parquet, Arrow and ORC need their footer, a zlib header must decode), and fails closed outside a repository or on a bad flag value (no traceback). Every address is a fictional example.com/example.org name or a made-up .zz domain.""" +import bz2 +import gzip +import importlib.util +import lzma +import os +import random +import subprocess +import sys +import tempfile +import tracemalloc +import unittest +import zlib +from pathlib import Path + +CPD = Path(__file__).resolve().parent.parent / ".claude/skills/deep-code-review/scripts/committed_personal_data.py" +ROW = "\t0\temail-addresses\t" + + +def addrs(n, domain="example.com", start=1): + """n distinct fictional addresses, bytes.""" + return [b"fixture-user-%04d@%s" % (i, domain.encode()) for i in range(start, start + n)] + + +def csv(a): + return b"id,name,email\n" + b"".join(b"%d,Jane Smith,%s\n" % (i, x) for i, x in enumerate(a)) + + +def parquet_like(a): + """Magic, 4-byte little-endian length prefix per value (PLAIN byte arrays), NUL padding, magic.""" + return b"PAR1" + b"".join(len(x).to_bytes(4, "little") + x for x in a) + b"\0" * 64 + b"PAR1" + + +def arrow_like(a): + """String values glued back to back after an offsets buffer, as Arrow lays them out.""" + off = [0] + for x in a: + off.append(off[-1] + len(x)) + return b"ARROW1\0\0" + b"".join(o.to_bytes(4, "little") for o in off) + b"".join(a) + b"\0" * 8 + b"ARROW1" + + +def scan(files, *args, stdin=None, links=None, git=True): + """Build a throwaway repo from {path: bytes}, run the script on it, return (exit code, stdout, stderr).""" + with tempfile.TemporaryDirectory() as d: + for rel, data in files.items(): + Path(d, rel).write_bytes(data) + for name, target in (links or {}).items(): + os.symlink(target, Path(d, name)) + env = {**os.environ, "GIT_CEILING_DIRECTORIES": str(Path(d).parent)} + if git: + subprocess.run(["git", "-C", d, "init", "-q"], check=True, env=env) + subprocess.run(["git", "-C", d, "add", "-A"], check=True, env=env) + r = subprocess.run([sys.executable, str(CPD), *args, d], input=stdin, capture_output=True, text=True, env=env, timeout=60) + return r.returncode, r.stdout, r.stderr + + +class Fixtures(unittest.TestCase): + FILES = {"export.csv": csv(addrs(12)), "data.parquet": parquet_like(addrs(12, "example.org"))} + + def test_csv_and_binary_flagged_with_include_reserved(self): + rc, out, err = scan(self.FILES, "--include-reserved") + self.assertEqual((rc, err), (0, "")) + self.assertEqual(sorted(out.splitlines()), [ + "data.parquet" + ROW + "12 distinct email addresses (threshold 10)", + "export.csv" + ROW + "12 distinct email addresses (threshold 10)"]) + + def test_arrow_like_glued_values_flagged(self): + rc, out, _ = scan({"data.arrow": arrow_like(addrs(12, "acme.zz"))}) + self.assertEqual((rc, out), (0, "data.arrow" + ROW + "12 distinct email addresses (threshold 10)\n")) + + def test_same_fixtures_clean_without_include_reserved(self): + rc, out, err = scan(self.FILES) # no email-addresses row; the parquet-like file starts with PAR1, so it is a not-checked row instead + self.assertEqual((rc, err, out.count("\n")), (0, "", 1)) + self.assertTrue(out.startswith("data.parquet\t0\tnot-checked\tcompressed/columnar container (parquet)"), out) + + def test_no_address_or_domain_in_output(self): + files = {**self.FILES, "people.csv": csv(addrs(12, "acme.zz"))} + for args in ((), ("--include-reserved",), ("--include-reserved", "--strict")): + rc, out, err = scan(files, *args) + self.assertIn(rc, (0, 1)) + self.assertIn("people.csv" + ROW, out) # findings were printed ... + for leak in ("@", "fixture-user", "example", "acme", "Jane"): + self.assertNotIn(leak, out + err) # ... and none carries a value + + def test_threshold_edge(self): + nine, ten = addrs(9), addrs(10) + noisy = csv(nine + [x.upper() for x in nine] + nine) # repeats and case variants count once + files = {"nine.csv": noisy, "ten.csv": csv(ten)} + self.assertEqual(scan(files, "--include-reserved")[1], "ten.csv" + ROW + "10 distinct email addresses (threshold 10)\n") + self.assertEqual(scan(files, "--include-reserved", "--threshold", "9")[1].splitlines(), [ + "nine.csv" + ROW + "9 distinct email addresses (threshold 9)", "ten.csv" + ROW + "10 distinct email addresses (threshold 9)"]) + self.assertEqual(scan(files, "--include-reserved", "--threshold", "11")[1], "") + + def test_default_counts_unreserved_domains_and_drops_reserved(self): + mixed = csv(addrs(6, "acme.zz") + addrs(6, "example.com", 7)) + self.assertEqual(scan({"m.csv": mixed})[1], "") + self.assertEqual(scan({"m.csv": mixed}, "--include-reserved")[1], "m.csv" + ROW + "12 distinct email addresses (threshold 10)\n") + more = csv(addrs(10, "acme.zz") + addrs(6, "example.com", 11)) + self.assertEqual(scan({"m.csv": more})[1], "m.csv" + ROW + "10 distinct email addresses (threshold 10)\n") + + def test_reserved_and_noreply_forms_dropped_lookalikes_counted(self): + dropped = [b"a1@example.com", b"a2@example.net", b"a3@example.org", b"a4@mail.example.com", b"a5@host.test", + b"a6@host.example", b"a7@host.invalid", b"a8@host.localhost", b"noreply@acme.zz", b"no-reply@acme.zz", + b"donotreply@acme.zz", b"do-not-reply@acme.zz", b"a9@users.noreply.acme.zz", b"my.noreply@acme.zz"] + counted = [b"b1@example.com.zz", b"b2@pre.zztest", b"b3@host.zzexample", b"noreplyfan@acme.zz"] # a reserved name only as part of another label + files = {"dropped.txt": b"\n".join(dropped), "counted.txt": b"\n".join(counted)} + self.assertEqual(scan(files, "--threshold", "1")[1], "counted.txt" + ROW + "4 distinct email addresses (threshold 1)\n") + self.assertEqual(scan(files, "--threshold", "1", "--include-reserved")[1].splitlines(), [ + "counted.txt" + ROW + "4 distinct email addresses (threshold 1)", + "dropped.txt" + ROW + "14 distinct email addresses (threshold 1)"]) + + def test_asset_scale_names_are_not_addresses(self): + exts = (b"png", b"jpg", b"jpeg", b"gif", b"webp", b"svg", b"pdf", b"avif", b"ico", b"bmp", b"tif", b"tiff", b"heic", b"heif", b"apng") + scales = (b"2x", b"3x", b"1.5x", b"2x-dark", b"3x-dark-hc") # what an iOS Contents.json, an Android res dir or a web manifest lists + assets = b"\n".join(b"icon-%02d@%s.%s" % (i, scales[i % 5], ext) for i, ext in enumerate(exts)) + self.assertEqual(scan({"Contents.json": assets})[1], "") + self.assertEqual(scan({"Contents.json": assets}, "--include-reserved")[1], "Contents.json" + ROW + "15 distinct email addresses (threshold 10)\n") + near = b"\n".join(b"n%02d@2x.zz" % i for i in range(12)) # a digits-x first label is dropped only before an image extension + self.assertEqual(scan({"near.txt": near})[1], "near.txt" + ROW + "12 distinct email addresses (threshold 10)\n") + other = b"\n".join(b"n%d@%s" % (i, t) for i, t in enumerate((b"2x-dark.zz", b"2x.png.zz", b"x2.png", b"2x.pngx", b"1.5.png"))) + self.assertEqual(scan({"other.txt": other}, "--threshold", "1")[1], "other.txt" + ROW + "5 distinct email addresses (threshold 1)\n") + + def test_git_ssh_account_is_not_an_address(self): + remotes = b"\n".join(b"git@host%02d.zz:org/repo.git\nssh://git@host%02d.zz/org/repo" % (i, i + 6) for i in range(6)) # 12 hosts + self.assertEqual(scan({"deps.txt": remotes})[1], "") + self.assertEqual(scan({"deps.txt": remotes}, "--include-reserved")[1], "deps.txt" + ROW + "12 distinct email addresses (threshold 10)\n") + people = b"\n".join(user + b"@acme.zz" for user in (b"mygit", b"x.git", b"git2", b"git-bot", b"git+x")) # git only inside a longer local part; built, so this file stays under the threshold itself + self.assertEqual(scan({"people.txt": people}, "--threshold", "1")[1], "people.txt" + ROW + "5 distinct email addresses (threshold 1)\n") + + def test_strict_exit_codes(self): + flagged, clean = {"a.csv": csv(addrs(12))}, {"a.csv": csv(addrs(2))} + self.assertEqual(scan(flagged, "--include-reserved")[0], 0) + self.assertEqual(scan(flagged, "--include-reserved", "--strict")[0], 1) + self.assertEqual(scan(clean, "--include-reserved", "--strict")[0], 0) + + def test_size_cap_flags_partial_and_never_reads_as_clean(self): + pad = b"." * 2000 + hit = scan({"a.csv": csv(addrs(12)) + pad}, "--include-reserved", "--max-bytes", "1000")[1] + self.assertEqual(hit, "a.csv" + ROW + "12 distinct email addresses (threshold 10); only the first 1000 bytes were read\n") + late = scan({"b.csv": csv(addrs(2)) + pad + csv(addrs(12, start=50))}, "--include-reserved", "--max-bytes", "1000")[1] + self.assertEqual(late, "b.csv\t0\tnot-checked\tpartially scanned: file over 1000 bytes, only the first 1000 were read\n") + self.assertEqual(scan({"c.csv": b"." * 1000}, "--max-bytes", "1000")[1], "") # exactly the cap: fully scanned + self.assertIn("not-checked", scan({"c.csv": b"." * 1001}, "--max-bytes", "1000")[1]) + + def test_count_cap_marks_plus(self): + out = scan({"big.csv": csv(addrs(100_000))}, "--include-reserved")[1] + self.assertEqual(out, "big.csv" + ROW + "100000+ distinct email addresses (threshold 10)\n") + + def test_stdin_limits_files_and_symlinks_are_skipped(self): + files = {"a.csv": csv(addrs(12)), "b.csv": csv(addrs(12))} + rows = scan(files, "--include-reserved", links={"c.csv": "a.csv"})[1].splitlines() + self.assertEqual([r.split("\t")[0] for r in sorted(rows)], ["a.csv", "b.csv"]) + self.assertEqual(scan(files, "--include-reserved", "--stdin", stdin="a.csv\ngone.csv\n\n")[1].splitlines(), [ + "a.csv" + ROW + "12 distinct email addresses (threshold 10)"]) + + def test_hostile_file_name_cannot_split_the_row(self): + out = scan({"a\tb.csv": csv(addrs(12))}, "--include-reserved")[1] + self.assertEqual(out, "a b.csv" + ROW + "12 distinct email addresses (threshold 10)\n") + + @unittest.skipIf(os.geteuid() == 0, "root can read mode 000 files") + def test_unreadable_file_is_reported_not_skipped(self): + with tempfile.TemporaryDirectory() as d: + Path(d, "locked.csv").write_bytes(csv(addrs(12))) + subprocess.run(["git", "-C", d, "init", "-q"], check=True) + subprocess.run(["git", "-C", d, "add", "-A"], check=True) + os.chmod(Path(d, "locked.csv"), 0) # after the add: git cannot index an unreadable file + r = subprocess.run([sys.executable, str(CPD), d], capture_output=True, text=True) + self.assertEqual((r.returncode, r.stdout), (0, "locked.csv\t0\tnot-checked\tunreadable: Permission denied\n")) + + def test_non_utf8_file_names_do_not_crash_the_scan(self): + with tempfile.TemporaryDirectory() as d: + r = subprocess.run([sys.executable, str(CPD), "--stdin", d], input=b"caf\xe9.csv\n", capture_output=True) + self.assertEqual((r.returncode, r.stdout, r.stderr), (0, b"", b"")) # a path that is not there is skipped, not a decode error + try: + Path(os.fsdecode(os.fsencode(d) + b"/caf\xe9.csv")).write_bytes(csv(addrs(12))) + except OSError: + self.skipTest("this file system refuses non-UTF-8 file names") # macOS; Linux runs the rest + subprocess.run(["git", "-C", d, "init", "-q"], check=True) + subprocess.run(["git", "-C", d, "add", "-A"], check=True) + r = subprocess.run([sys.executable, str(CPD), "--include-reserved", d], capture_output=True) + self.assertEqual((r.returncode, r.stdout), (0, b"caf\xe9.csv" + ROW.encode() + b"12 distinct email addresses (threshold 10)\n")) + + def test_fails_closed_outside_a_repository_and_on_bad_usage(self): + rc, out, err = scan({"a.csv": csv(addrs(12))}, git=False) + self.assertEqual((rc, out), (2, "")) + self.assertIn("git ls-files failed", err) + for bad in (["--threshold", "0"], ["--threshold", "100001"], ["--max-bytes", "0"], ["--threshold", "x"], + ["--repo-threshold", "0"], ["--repo-threshold", "100001"], ["--repo-threshold", "x"], + ["--max-bytes", "1073741825"], ["--max-bytes", "99999999999999999999"]): # a huge size was a traceback with exit 1 + rc, _, err = scan({"a.csv": b""}, *bad) + self.assertEqual(rc, 2, bad) + self.assertIn("usage: committed_personal_data.py", err) + self.assertNotIn("Traceback", err) + for edge in ("1", "100000"): # both ends of the --repo-threshold range are accepted + self.assertEqual(scan({"a.csv": b""}, "--repo-threshold", edge), (0, "", "")) + + +class RepositoryTotal(unittest.TestCase): + @staticmethod + def spread(files, per, start=1): + """`files` files of `per` distinct acme.zz addresses each, none reaching the file threshold.""" + return {"f%02d.txt" % i: csv(addrs(per, "acme.zz", start + i * per)) for i in range(files)} + + @staticmethod + def total(n, files, limit=50): + return ".%s%d distinct email addresses across %d file(s) that each stay below the file threshold 10 (repository threshold %d; --threshold 1 lists the files)\n" % (ROW, n, files, limit) + + def test_addresses_spread_thin_reach_a_repository_total(self): + self.assertEqual(scan(self.spread(20, 3)), (0, self.total(60, 20), "")) # 60 distinct, 3 per file: no file is flagged + + def test_total_edge_repeats_and_flagged_files(self): + files = self.spread(25, 2) # 50 distinct + self.assertEqual(scan(files)[1], self.total(50, 25)) + self.assertEqual(scan(files, "--repo-threshold", "51")[1], "") + same = {"f%02d.txt" % i: csv(addrs(3, "acme.zz")) for i in range(30)} # the same 3 addresses in 30 files count once + self.assertEqual(scan(same, "--repo-threshold", "4")[1], "") + self.assertEqual(scan(same, "--repo-threshold", "3")[1], self.total(3, 30, 3)) + flagged = "big.csv" + ROW + "12 distinct email addresses (threshold 10)" + big = {**self.spread(20, 2), "big.csv": csv(addrs(12, "acme.zz", 5000))} # 40 spread + one flagged file, whose 12 are left out of the total + self.assertEqual(scan(big)[1], flagged + "\n") + self.assertEqual(scan(big, "--repo-threshold", "40")[1].splitlines(), [flagged, self.total(40, 20, 40).rstrip("\n")]) + + def test_total_is_a_strict_finding_and_follows_include_reserved_and_stdin(self): + files = self.spread(20, 3) + self.assertEqual([scan(files)[0], scan(files, "--strict")[0], scan(self.spread(10, 3), "--strict")[0]], [0, 1, 0]) # 60, then 30 < 50 + self.assertEqual(scan({"d.bin": b"PK\x03\x04" + b"\0" * 16}, "--strict")[0], 0) # a not-checked row alone is no finding + reserved = {"f%02d.txt" % i: csv(addrs(3, "example.com", 1 + 3 * i)) for i in range(20)} + self.assertEqual(scan(reserved), (0, "", "")) + self.assertEqual(scan(reserved, "--include-reserved")[1], self.total(60, 20)) + self.assertEqual(scan(files, "--stdin", stdin="\n".join(list(files)[:10]))[1], "") # only the listed files are counted: 30 < 50 + + +class Containers(unittest.TestCase): + # (kind, bytes at the start, bytes at the end): parquet, Arrow and ORC repeat their magic at the end, the others are told by the start alone + FORMATS = ( + ("parquet", b"PAR1", b"PAR1"), ("parquet", b"PARE", b"PARE"), ("arrow", b"ARROW1\0\0", b"ARROW1"), ("orc", b"ORC", b"ORC\x1b"), + ("zip", b"PK\x03\x04", b""), ("gzip", b"\x1f\x8b\x08\x00", b""), ("zstd", b"\x28\xb5\x2f\xfd", b""), ("xz", b"\xfd7zXZ\x00", b""), + ("7z", b"7z\xbc\xaf\x27\x1c", b""), ("lz4", b"\x04\x22\x4d\x18", b""), ("bzip2", b"BZh91AY&SY", b""), ("avro", b"Obj\x01", b""), + ("pdf", b"%PDF-1.7\n", b""), ("zlib", b"\x78\x9c\x03\x00\x00\x00\x00\x01", b""), # an empty zlib stream: header, final empty block, adler32 + ) + + @staticmethod + def blob(start, end, n): + return start + b"\0" * 8 + b"\n".join(addrs(n, "acme.zz")) + b"\0" * 8 + end + + def test_container_under_the_threshold_is_not_checked_never_silent(self): + for kind, start, end in self.FORMATS: # 9 addresses < 10: such a file used to pass silently + rc, out, err = scan({"data.bin": self.blob(start, end, 9)}) + self.assertEqual((rc, err, out.count("\n")), (0, "", 1), (kind, start)) + path, line, rule, msg = out.rstrip("\n").split("\t") + self.assertEqual((path, line, rule), ("data.bin", "0", "not-checked"), (kind, start)) + self.assertIn("compressed/columnar container (%s)" % kind, msg) + for leak in ("@", "fixture-user", "acme"): + self.assertNotIn(leak, out) + + def test_container_at_or_over_the_threshold_is_flagged_once(self): + for kind, start, end in self.FORMATS: + out = scan({"data.bin": self.blob(start, end, 12)})[1] + self.assertEqual(out, "data.bin" + ROW + "12 distinct email addresses (threshold 10)\n", kind) + + def test_magic_counts_only_at_the_start_of_the_file(self): + files = {"late.bin": b"x" + self.blob(b"PAR1", b"PAR1", 9), "plain.csv": csv(addrs(9, "acme.zz"))} + self.assertEqual(scan(files), (0, "", "")) + + def test_a_magic_without_its_footer_is_not_a_container(self): + tail = b"\n".join(addrs(3, "acme.zz")) + b"\n" + names = {"a.txt": b"PAR1 is a name\n", "b.txt": b"PARE is a name\n", "c.txt": b"ARROW1 is a name\n", "d.txt": b"ORC Holdings is a name\n"} + self.assertEqual(scan({k: v + tail for k, v in names.items()}), (0, "", "")) + self.assertEqual(scan({"cut.parquet": b"PAR1" + b"\0" * 8 + tail}), (0, "", "")) # a truncated file has no footer either + + def test_start_and_end_magics_must_not_overlap(self): + self.assertEqual(scan({"a.parquet": b"PAR1", "b.orc": b"ORC\n", "c.arrow": b"ARROW1"}), (0, "", "")) # one magic read as both ends + out = scan({"a.parquet": b"PAR1PAR1", "b.orc": b"ORCORC\n", "c.arrow": b"ARROW1ARROW1"})[1] # the shortest files that hold both + self.assertEqual([r.split("\t")[0] for r in out.splitlines()], ["a.parquet", "b.orc", "c.arrow"], out) + + def test_the_footer_is_read_from_the_end_of_a_file_over_the_cap(self): + out = scan({"d.parquet": self.blob(b"PAR1", b"PAR1", 9)}, "--max-bytes", "100")[1] + self.assertTrue(out.startswith("d.parquet\t0\tnot-checked\tcompressed/columnar container (parquet)"), out) + self.assertTrue(out.endswith("; only the first 100 bytes were read\n"), out) + out = scan({"d.parquet": self.blob(b"PAR1", b"PAR1", 9)}, "--max-bytes", "5")[1] # a cap shorter than both magics still sees parquet + self.assertTrue(out.startswith("d.parquet\t0\tnot-checked\tcompressed/columnar container (parquet)"), out) + + def test_real_compressed_exports_are_never_silent(self): + data = csv(addrs(3000, "acme.zz")) + pdf = b"%PDF-1.4\n1 0 obj\n<< /Filter /FlateDecode >>\nstream\n" + zlib.compress(data) + b"\nendstream\nendobj\n%%EOF\n" + for kind, blob in (("gzip", gzip.compress(data, mtime=0)), ("bzip2", bz2.compress(data)), ("xz", lzma.compress(data)), + ("zlib", zlib.compress(data)), ("zlib", zlib.compress(b"hi")), ("pdf", pdf)): + rc, out, err = scan({"export.bin": blob}) + self.assertEqual((rc, err), (0, ""), kind) + self.assertTrue(out.startswith("export.bin\t0\tnot-checked\tcompressed/columnar container (%s)" % kind), (kind, out)) + self.assertEqual(out.count("\n"), 1, kind) + self.assertNotIn("@", out) + + def test_zlib_header_lookalikes_are_not_zlib(self): + files = {"tok.txt": b"807", "math.txt": b"x^2 + y^2 = 1\n" * 40, "pair.txt": b"x\x9c is not deflate data, only a header-shaped pair of bytes\n"} + self.assertEqual(scan(files), (0, "", "")) # each opens with a pair that passes the 2-byte header check (x^, 80, x + 0x9c) + + +class Noise(unittest.TestCase): + def test_random_bytes_stay_under_the_default_threshold(self): + files = {"r%d.bin" % s: random.Random(s).randbytes(8 << 20) for s in range(3)} + self.assertEqual(scan(files), (0, "", "")) + + def test_long_runs_scan_in_linear_time(self): + rc, out, _ = scan({"run.txt": b"a" * (8 << 20), "ats.txt": (b"a" * 100 + b"@a.bc ") * 100_000}) + self.assertEqual((rc, out), (0, "")) + + + def test_dotted_run_after_an_at_sign_uses_bounded_memory(self): + spec = importlib.util.spec_from_file_location("committed_personal_data", CPD) + mod = importlib.util.module_from_spec(spec) + spec.loader.exec_module(mod) + buf = b"x@" + b"a." * (1 << 20) # an unbounded domain match grew the regex stack to ~140 MB here + tracemalloc.start() + try: + self.assertEqual(mod.addresses(buf, True), set()) + peak = tracemalloc.get_traced_memory()[1] + finally: + tracemalloc.stop() + self.assertLess(peak, 16 << 20) + + +if __name__ == "__main__": + unittest.main() diff --git a/scripts/test_comms_lint_placeholder_template.py b/scripts/test_comms_lint_placeholder_template.py new file mode 100644 index 000000000..500d0b326 --- /dev/null +++ b/scripts/test_comms_lint_placeholder_template.py @@ -0,0 +1,45 @@ +#!/usr/bin/env python3 +"""comms_lint placeholder rule: a mktemp template inside a single-backtick span on one line (`drain.XXXXXX`) is not a +placeholder, while TBD, TODO, , , [date] and a lone XXX in prose still are, even on a line that also carries a +template, and the same run in a double-backtick span or a span split across lines still is. Synthetic input only.""" +import sys +import unittest +from pathlib import Path + +sys.path.insert(0, str(Path(__file__).resolve().parent.parent / ".claude/skills/communication-structure/scripts")) +import comms_lint as L # noqa: E402 + + +def placeholders(text): + """The text each placeholder hit matched, for `text` linted as a message.""" + res = L.lint("message", text) + return [h["problem"].strip("'") for r in res["rules"] if r["rule"] == "placeholder" for h in r["hits"]] + + +class Templates(unittest.TestCase): + def test_mktemp_template_in_a_code_span_passes(self): + for text in ("Scratch lives in `drain.XXXXXX`.", + 'Each run makes `mktemp -d "${TMPDIR:-/tmp}/drain.XXXXXX"` and removes it.', + "Run `mktemp XXXXXX` once.", + "Two dirs: `a.XXXXXX` and `b.XXXXXXXX`."): + self.assertEqual(placeholders(text), [], text) + + def test_real_placeholders_still_fail(self): + for text, want in (("Owner: TBD", "TBD"), ("TODO: name the file", "TODO"), ("Owner ", ""), ("Owner ", ""), + ("Owner ", ""), ("Due [date]", "[date]"), ("Owner: XXX", "XXX"), ("ETA xxx", "xxx")): + self.assertEqual(placeholders(text), [want], text) + + def test_a_template_does_not_hide_a_placeholder_on_its_line(self): + for text, want in (("Owner: TBD; scratch is `drain.XXXXXX`.", "TBD"), ("ETA XXX, scratch is `drain.XXXXXX`.", "XXX"), + ("`drain.XXXXXX` TODO", "TODO"), ("`a.XXXXXX` and a bare XXX", "XXX")): + self.assertEqual(placeholders(text), [want], text) + + def test_the_same_run_outside_a_code_span_still_fails(self): + self.assertEqual(placeholders("Scratch lives in drain.XXXXXX."), ["XXXXXX"]) + self.assertEqual(placeholders("Scratch lives in `drain.XXXXXX and never closed."), ["XXXXXX"]) # no closing backtick: not a code span + self.assertEqual(placeholders("Scratch lives in ``drain.XXXXXX``."), ["XXXXXX"]) # a double-backtick span is not exempt + self.assertEqual(placeholders("Scratch lives in `drain.XXXXXX\nand is removed`."), ["XXXXXX"]) # nor is a span split across lines + + +if __name__ == "__main__": + unittest.main() diff --git a/scripts/test_destructive_guard_w21.py b/scripts/test_destructive_guard_w21.py index 3f8e04479..4da46acee 100644 --- a/scripts/test_destructive_guard_w21.py +++ b/scripts/test_destructive_guard_w21.py @@ -5,6 +5,8 @@ ROOT = pathlib.Path(__file__).resolve().parent.parent GUARD = ROOT / ".claude/skills/agentic-delivery/scripts/destructive_guard.py" +WAIT = 60 # seconds, a ceiling and never a delay: a healthy run returns as soon as the guard answers (the wall-clock case after its 10 s TIMEOUT). +# A 20 s ceiling failed once at load average ~13, because process start-up and exit eat the slack; a real hang still fails, at WAIT. def blocked(cmd): @@ -290,14 +292,14 @@ def test_rows(): def test_stdin_does_not_hang(): - r = subprocess.run([sys.executable, "-I", str(GUARD)], stdin=subprocess.DEVNULL, capture_output=True, timeout=10) + r = subprocess.run([sys.executable, "-I", str(GUARD)], stdin=subprocess.DEVNULL, capture_output=True, timeout=WAIT) assert r.returncode == 0 try: m, s = pty.openpty() # stdin is a terminal except OSError: return print("skip tty half: no pty in this sandbox") try: - r = subprocess.run([sys.executable, "-I", str(GUARD)], stdin=s, capture_output=True, timeout=10) + r = subprocess.run([sys.executable, "-I", str(GUARD)], stdin=s, capture_output=True, timeout=WAIT) assert r.returncode == 0 finally: os.close(m), os.close(s) @@ -307,7 +309,7 @@ def test_wall_clock_cap(): """stdin that never closes must end in a block after TIMEOUT seconds, not spin or wait forever.""" p = subprocess.Popen([sys.executable, "-I", str(GUARD)], stdin=subprocess.PIPE, stderr=subprocess.PIPE) try: - assert p.wait(timeout=20) == 2 and b"did not finish" in p.stderr.read() + assert p.wait(timeout=WAIT) == 2 and b"did not finish" in p.stderr.read() finally: p.kill(), p.stdin.close(), p.stderr.close() @@ -317,7 +319,7 @@ def test_regex_spin_is_killed(): spin = ("import importlib.util as u, re, sys; s = u.spec_from_file_location('g', sys.argv[1]); g = u.module_from_spec(s); " "s.loader.exec_module(g); g.TIMEOUT = 2; g.check = lambda c: re.match(r'(a+)+$', 'a' * 64 + 'b'); sys.exit(g.main())") r = subprocess.run([sys.executable, "-I", "-c", spin, str(GUARD)], input='{"tool_input":{"command":"ls"}}', - capture_output=True, text=True, timeout=20) + capture_output=True, text=True, timeout=WAIT) assert r.returncode == 2 and "did not finish" in r.stderr, (r.returncode, r.stderr) @@ -328,7 +330,7 @@ def test_watchdog_failures_block(): nofork = load + "os.fork = lambda: (_ for _ in ()).throw(OSError(11, 'EAGAIN')); sys.exit(g.main())" crash = load + "g.check = lambda c: os.kill(os.getpid(), 9); sys.exit(g.main())" for code, cmd, want in ((nofork, "rm -rf /", 2), (nofork, "ls", 0), (crash, "ls", 2)): - r = subprocess.run([sys.executable, "-I", "-c", code, str(GUARD)], input=cmd, capture_output=True, text=True, timeout=20) + r = subprocess.run([sys.executable, "-I", "-c", code, str(GUARD)], input=cmd, capture_output=True, text=True, timeout=WAIT) assert r.returncode == want, (cmd, r.returncode, r.stderr) diff --git a/scripts/test_destructive_guard_w21_wait.py b/scripts/test_destructive_guard_w21_wait.py new file mode 100644 index 000000000..61cca7f79 --- /dev/null +++ b/scripts/test_destructive_guard_w21_wait.py @@ -0,0 +1,69 @@ +#!/usr/bin/env python3 +"""Pins the load-tolerant waits in test_destructive_guard_w21.py: its wall-clock case waits up to a 60 s ceiling (a 20 s one +failed once at load average ~13, passing alone) and a real hang still fails, at that ceiling. The guard's own watchdog +(TIMEOUT in destructive_guard_core.py) is the thing waited for, so it is pinned at 20 s or less too. +Offline and instant: a stub stands in for the guard process, so nothing here waits on a clock.""" +import importlib.util +import io +import pathlib +import re +import subprocess +import unittest +from unittest import mock + +SRC = pathlib.Path(__file__).resolve().parent / "test_destructive_guard_w21.py" +spec = importlib.util.spec_from_file_location("guard_w21", SRC) +w21 = importlib.util.module_from_spec(spec) +spec.loader.exec_module(w21) +CORE = w21.GUARD.with_name("destructive_guard_core.py") # the guard's own watchdog lives here + + +class StubGuard: + """Stands in for the guard Popen: records the wait ceiling, then answers like the guard (exit 2) or hangs.""" + + def __init__(self, hang): + self.hang, self.ceiling = hang, None + self.stdin, self.stderr = io.BytesIO(), io.BytesIO(b"Blocked: destructive guard did not finish within 10s") + + def wait(self, timeout=None): + self.ceiling = timeout + if self.hang: + raise subprocess.TimeoutExpired("guard", timeout) + return 2 + + def kill(self): + pass + + +def wall_clock(hang): + """(stub, error): the wall-clock case run against a stub guard that answers promptly or hangs.""" + stub = StubGuard(hang) + try: + with mock.patch.object(subprocess, "Popen", lambda *a, **k: stub): + w21.test_wall_clock_cap() + except Exception as e: # the case's own failure, returned for the assertions + return stub, e + return stub, None + + +class WaitCeiling(unittest.TestCase): + def test_wall_clock_ceiling_tolerates_load(self): + stub, err = wall_clock(hang=False) + self.assertIsNone(err) + self.assertGreaterEqual(stub.ceiling, 60) # the old 20 s ceiling is what a loaded runner beat + + def test_real_hang_still_fails_at_a_finite_ceiling(self): + stub, err = wall_clock(hang=True) + self.assertIsInstance(err, subprocess.TimeoutExpired) + self.assertLessEqual(stub.ceiling, 120) # a ceiling and not a switch-off + + def test_guards_watchdog_is_at_most_20_s(self): + timeout = int(re.search(r"^TIMEOUT = (\d+)\b", CORE.read_text(), re.M).group(1)) + self.assertLessEqual(timeout, 20) # the wall-clock case waits for this watchdog: raising it past 20 s must fail here + + def test_every_wait_uses_the_ceiling(self): + self.assertEqual(set(re.findall(r"\btimeout=(\w+)", SRC.read_text())), {"WAIT"}) # no literal a loaded runner can beat + + +if __name__ == "__main__": + unittest.main() From 0dda58e0bc931bf74d5f2378490b8273ecc5521a Mon Sep 17 00:00:00 2001 From: Remi Antczak <140408771+remigiusz-antczak@users.noreply.github.com> Date: Fri, 30 Oct 2026 09:13:57 +0100 Subject: [PATCH 2/2] Rewrap a comms_lint docstring so no line ends in mktemp CI's cheap job failed its "mktemp TMPDIR regression" step on PR head 1dcc88a3a. The BARE pattern in scripts/test-mktemp-tmpdir.sh matched line 8 of .claude/skills/communication-structure/scripts/comms_lint.py, a docstring line that ended with the words "a mktemp": a line that ends in mktemp reads as a bare call with no template. Move the word "mktemp" to the start of the next docstring line. Same bytes, same words, same line count; no code, SKILL.md, reference, index, checksum or size-budget row is touched. Checked locally with BASE_SHA=6611ca89b and HEAD_SHA=HEAD: the mktemp gate passes, and so does every later step of the cheap job (steps 02 to 18 of ci.yml, run one at a time; the regenerate-unlanded step skipped). Two caveats on this host: python3 pointed at 3.12 to match CI, and two self-tests that call ps ran against a small ps stand-in because the sandbox denies /bin/ps. Tests: 17/17 cheap-job steps Co-Authored-By: Claude Sonnet 5.5 --- .claude/skills/communication-structure/scripts/comms_lint.py | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/.claude/skills/communication-structure/scripts/comms_lint.py b/.claude/skills/communication-structure/scripts/comms_lint.py index 35522e7d0..b3e8eea01 100644 --- a/.claude/skills/communication-structure/scripts/comms_lint.py +++ b/.claude/skills/communication-structure/scripts/comms_lint.py @@ -5,8 +5,8 @@ (the one source of truth; `--list` prints it). Always checked: lead (line 1 is the answer: <=60 words, first sentence <=35, no greeting/background opener), budget (fails only above BUDGET_SLACK x the table value), dates (a weekday must fit its date, computed from the calendar), filler (AI-slop phrases), placeholder (TBD, [owner], and the -stand-ins "owner unassigned", "no deadline", "not stated"; a run of capital Xs inside a single-backtick span on one line, a mktemp -template such as `drain.XXXXXX`, is not one), vague (numberless "significantly", "on track"). Advisory +stand-ins "owner unassigned", "no deadline", "not stated"; a run of capital Xs inside a single-backtick span on one line, a +mktemp template such as `drain.XXXXXX`, is not one), vague (numberless "significantly", "on track"). Advisory WARNs that never put a draft below the bar: hedge, length (over the table value), markers (check/hourglass lines), numbers (a figure in the first 4 content lines), verified, timezone, and ask (runs only with --source, only when the source has an ask, and flags a missing owner or date only if the source gives one: nothing demands what the input