From 8bf18addaca42414c686bd57abaa3b1ce284a030 Mon Sep 17 00:00:00 2001 From: MendixMau Date: Thu, 15 Oct 2026 22:41:11 +0800 Subject: [PATCH 1/4] new(security): check-staged-secrets.sh and a secret-bearing files section (TK-025) Situational skills only; sources and field evidence in CHANGELOG. Verified claims are marked as field observations where mxcli/Mendix docs did not confirm them. Co-Authored-By: Claude Sonnet 5.5 --- CHANGELOG.md | 1 + bin/lib/install-manifest.sh | 2 +- project-bin/check-staged-secrets.sh | 100 +++++++++++++++++++++++ skills/learned-constants-and-secrets.md | 29 +++++++ tests/wave2/test-check-staged-secrets.sh | 67 +++++++++++++++ 5 files changed, 198 insertions(+), 1 deletion(-) create mode 100755 project-bin/check-staged-secrets.sh create mode 100644 tests/wave2/test-check-staged-secrets.sh diff --git a/CHANGELOG.md b/CHANGELOG.md index 6e238acd..dd722a0d 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -19,6 +19,7 @@ Sections dated before 2026-09-19 predate the cycle and stay as they are. Named m ## Unreleased +- new(security): **`project-bin/check-staged-secrets.sh` — compare the project's real secret values (`.env`, `.mxcli/run-local.json` bootConfig) against the staged blobs (or `--tree`, every tracked file at HEAD) and print only the file and the key name, never a value (TK-025).** Plus a section in `learned-constants-and-secrets.md`: the secret-bearing files and commands an agent never prints (`run-local.json`, the `run --local` command line, run state, `describe constant`, `.env`, rendered files that interpolate a secret), read-by-testing habits, and "a leaked value is rotated, not deleted". Field evidence: a five-app lab run with six leak events into transcripts and a demo password in git (mxcli v0.26.0). Opt-in, not wired into the generated pre-commit hook. Test with made-up values: `tests/wave2/test-check-staged-secrets.sh` (15 checks). Not adopted: the report's `exec-secret.sh` is a project-local script that does not ship here, and masking `--constant` values in `app-restart.sh` is TK-003. — five-app lab report, TK-025 - new(mxpatch, exec): **`project-bin/mxpatch.py`, the shared patch core, and a `mxcli fix hashes` check around every `exec.sh --patch` run: the two shipped workflow patchers no longer leave a stale ContentsHash.** mxpatch finds a unit by type + name (+ module) with a byte pre-filter, finds an element by Name, gets/sets a value by key, and writes the unit and its `.mpr` ContentsHash (base64 SHA-256 of the file) in one step; dry run by default, `already applied` on a re-run, and a refusal (exit 3) when the current value is not the expected old one. `wf-set-call-captions.py` and `wf-add-path-terminators.py` now write through it, refuse drift (a caption is replaced only while it is still the stamped microflow name or the `--expect-caption` value; `--expect-paths N` for terminators), and also take the `exec.sh --patch` form (` Module.Workflow`). The index is committed before the file swap, so a kill mid-write leaves a MISMATCH that `fix hashes` reports and a re-run repairs. `mxunit_bson.py` reads BSON 0x09 (UTC datetime). `exec.sh --patch` runs `mxcli fix hashes` (v0.25.0+, probed via `fix --help`) before and after the script and restores the snapshot on any issue the baseline did not have, before mxbuild, which cannot see it. Field evidence: captured mxcli v0.25.0/v0.26.0 runs on a public model (`tests/wave2/fixtures/mxpatch/CAPTURE.txt`): master's patchers leave a MISMATCH, the ported ones verify clean; the hash routine agrees with mxcli on 67 of 67 mismatched and 302 matching units of a second model. Widget-shape copying and pluggable-widget properties wait on a captured page unit. Field run ([#390](https://github.com/MendixMau/mxcli-project-toolkit/issues/390), mxcli v0.26.0, macOS): PASS, and the patch repaired a ContentsHash MISMATCH an old-style `--patch` had left on the unit it rewrote. Fixes from it: the backup of a rewritten unit is kept outside `mprcontents/` (in `$MXPATCH_BACKUP_DIR`, which `exec.sh --patch` points into the snapshot it just took, else a 0700 `/mxpatch-bak-/`; refused when it resolves inside `mprcontents/`, through a symlink or another case spelling; named with pid and counter so two writes in one second keep both), no longer as an untracked `.mxunit.bak` that `git add mprcontents` would commit; the BUILD-LOG row carries the hash bracket (`hashes: baseline 1 mismatch (pre-existing), verify clean (1 pre-existing repaired by this patch's rewrite)`); and the baseline message says that a unit the patch rewrites gets a fresh hash, and to repair the others with `fix hashes --repair`. A baseline MISMATCH that is still reported on a unit the patch rewrote counts as new (restore, exit 1), not as pre-existing. — jopterhorst ([#294](https://github.com/MendixMau/mxcli-project-toolkit/issues/294), [#295](https://github.com/MendixMau/mxcli-project-toolkit/issues/295)) - learn(pipelines/mendix-mpr/samples): **three minimal mxcli repros from the native-module bug hunt, the files the upstream draft cites** (split from [#293](https://github.com/MendixMau/mxcli-project-toolkit/pull/293), 6/6). `fsm-template-9.24.2/native-bughunt/` holds CAPTURE.txt and D1–D3, which are written from scratch in a `Repro` module: `describe entity` printing `index ( desc);`, the native Repeater `content` slot, and no flow writable to a 9.24.2 project. They are the repros for U3, U2 and U14 in `bug-logs/pending-github-issues/2026-10-12-mxcli-0.26-round-trip-report.md`. The other 472 sample files of #293 had no citing reader, because the skills and Rehost splits state their field results inline, so none of them ships. — MendixMau, Decompose/Rehost lab runs 2026-10-12 - learn(inbox, pending-github-issues): **ten field-note drops from the Decompose/Rehost lab runs, and one not-filed draft of 53 mxcli 0.26 defects for mendixlabs/mxcli** (split from [#293](https://github.com/MendixMau/mxcli-project-toolkit/pull/293), 5/6). `contrib/inbox/2026-10-1{2,3}-*.md`: increments 1–8 of the split-a-monolith runs on a public Marketplace template (FSM 9.24.2), the native-module bug hunt, and the GenAI Showcase regeneration (0.26.0 and the 2026-10-13 nightly); each now has the template's front matter and is at most 600 words. `bug-logs/pending-github-issues/2026-10-12-mxcli-0.26-round-trip-report.md`: the U1–U53 table the inbox drops refer to, marked DRAFT — NOT FILED, to be split into one issue per defect and retested before anything is posted. No BUG numbers assigned; triage promotes. — MendixMau, Decompose/Rehost lab runs 2026-10-12/13 diff --git a/bin/lib/install-manifest.sh b/bin/lib/install-manifest.sh index e1c939d3..547c41a8 100644 --- a/bin/lib/install-manifest.sh +++ b/bin/lib/install-manifest.sh @@ -115,7 +115,7 @@ MXTK_AGENTS_STAGE_BUILD="mdl-agent.md gate-agent.md test-agent.md review-agent.m # may run itself (stops only the pid run --local recorded for THIS project), and `mxcli test` on a # scratch copy under the test-port lock. exec.sh names app-restart.sh in its watch advisory, so # it has to be installed beside it. -MXTK_PROJECT_BIN="app-restart.sh test-scratch.sh assemble-prototype.js prototype-route.js check-prototype-links.js wf-add-path-terminators.py wf-set-call-captions.py mxunit_bson.py mxpatch.py _common.sh app-facts.sh _claims.sh snapshot-mpr.sh restore-mpr.sh exec.sh look-ledger.sh save-sp.sh restart-sp.sh check-sp-health.sh verify-module.sh test-stack-up.sh fixture-manifest.sh check-root-clean.sh lint-gate.sh lint-trend.sh metrics-ledger.sh metrics-trend.sh close-task.sh conformance-check.sh coverage-preflight.sh graph-sweep.sh review-module.sh coherence-cadence.sh build-plan-status.sh done-drift-check.sh page-scope.sh render-improvement-register.sh check-design-portability.sh check-design-reaches-app.sh check-page-shell.sh page-fidelity.js model-stamp.sh verify-model.sh install-project-hooks.sh mx-metadata-note.py session-check.sh constants-audit.sh context-pack.sh" +MXTK_PROJECT_BIN="app-restart.sh test-scratch.sh assemble-prototype.js prototype-route.js check-prototype-links.js wf-add-path-terminators.py wf-set-call-captions.py mxunit_bson.py mxpatch.py _common.sh app-facts.sh _claims.sh snapshot-mpr.sh restore-mpr.sh exec.sh look-ledger.sh save-sp.sh restart-sp.sh check-sp-health.sh verify-module.sh test-stack-up.sh fixture-manifest.sh check-root-clean.sh lint-gate.sh lint-trend.sh metrics-ledger.sh metrics-trend.sh close-task.sh conformance-check.sh coverage-preflight.sh graph-sweep.sh review-module.sh coherence-cadence.sh build-plan-status.sh done-drift-check.sh page-scope.sh render-improvement-register.sh check-design-portability.sh check-design-reaches-app.sh check-page-shell.sh page-fidelity.js model-stamp.sh verify-model.sh install-project-hooks.sh mx-metadata-note.py session-check.sh constants-audit.sh context-pack.sh check-staged-secrets.sh" # Files in project-bin/ that are deliberately NOT installed into projects. The reverse check # below flags anything named by NEITHER list, so a new file in project-bin/ has to be either diff --git a/project-bin/check-staged-secrets.sh b/project-bin/check-staged-secrets.sh new file mode 100755 index 00000000..43735b9b --- /dev/null +++ b/project-bin/check-staged-secrets.sh @@ -0,0 +1,100 @@ +#!/usr/bin/env bash +# check-staged-secrets.sh — fail when a secret VALUE from this project's own secret-bearing files +# is about to be committed (staged blobs) or is already in the tracked tree. Prints the file and +# the NAME of the key that matched. It never prints a value. +# +# WHY THIS EXISTS (field run, five apps, mxcli v0.26.0). A demo password reached git through a +# rendered login page, and several lane transcripts through `.mxcli/run-local.json`, whose +# bootConfig holds database and API passwords in clear. `.gitignore` does not help once a file is +# tracked, and `git commit -- ` re-adds an ignored tracked file at commit time. The one +# check that sees what git will actually store is a scan of the blobs, against the real values. +# +# Where the values come from (read here, never echoed): +# .env and .env.* KEY=VALUE where KEY looks like PASSWORD / SECRET / TOKEN / KEY / PAT +# .mxcli/run-local.json every string in bootConfig whose key looks like one of those +# $MXTK_SECRET_VALUES_FILE optional: one literal value per line (name shown as "extra") +# Values shorter than 6 characters are ignored (a scan for "1" would match everything). +# Example files (.env.example, .env.sample, .env.template) are not sources. +# +# Usage: +# bin/check-staged-secrets.sh # staged blobs (what the next commit stores) +# bin/check-staged-secrets.sh --tree # every tracked file at HEAD (the "already pushed?" check) +# Hook it yourself (it is NOT wired into the generated pre-commit): +# "$(git rev-parse --show-toplevel)/bin/check-staged-secrets.sh" || exit 1 +# +# Exit: 0 clean (or no sources found) · 1 a secret value is in the scanned blobs · 2 not a git repo +set -u +PROJECT_ROOT="${PROJECT_ROOT:-$(cd "$(dirname "${BASH_SOURCE[0]}")/.." && pwd)}" +cd "$PROJECT_ROOT" || { echo "check-staged-secrets: cannot cd to $PROJECT_ROOT" >&2; exit 2; } +git rev-parse --show-toplevel >/dev/null 2>&1 || { echo "check-staged-secrets: not a git repository" >&2; exit 2; } +MODE=staged; [ "${1:-}" = "--tree" ] && MODE=tree +PY="$(command -v python3 || command -v python || true)" + +NAMEPAT='(PASS|SECRET|TOKEN|KEY|PAT$|CREDENTIAL)' +SRC="$(mktemp "${TMPDIR:-/tmp}/mxtk-secrets.XXXXXX")"; trap 'rm -f "$SRC"' EXIT +chmod 600 "$SRC" + +# namevalue, one per line. +for f in .env .env.*; do + [ -f "$f" ] || continue + case "$f" in *.example|*.sample|*.template) continue ;; esac + sed -E 's/^[[:space:]]*export[[:space:]]+//' "$f" | while IFS= read -r line; do + case "$line" in \#*|"") continue ;; esac + k="${line%%=*}"; v="${line#*=}"; [ "$k" = "$line" ] && continue + v="${v%\"}"; v="${v#\"}"; v="${v%\'}"; v="${v#\'}" + printf '%s' "$k" | grep -qiE "$NAMEPAT" || continue + printf '%s\t%s\n' "$f:$k" "$v" + done >> "$SRC" +done +if [ -f .mxcli/run-local.json ] && [ -n "$PY" ]; then + "$PY" -I - "$NAMEPAT" >> "$SRC" 2>/dev/null <<'PYEOF' +import json, re, sys +pat = re.compile(sys.argv[1], re.I) +try: + data = json.load(open(".mxcli/run-local.json")) +except Exception: + sys.exit(0) +def walk(o, path=""): + if isinstance(o, dict): + for k, v in o.items(): + yield from walk(v, path + "." + k if path else k) + elif isinstance(o, str) and "bootConfig" in path: + yield path, o +for p, v in walk(data): + if pat.search(p.rsplit(".", 1)[-1]) and "\t" not in v and "\n" not in v: + print(".mxcli/run-local.json:" + p.split("bootConfig.", 1)[-1] + "\t" + v) +PYEOF +fi +if [ -n "${MXTK_SECRET_VALUES_FILE:-}" ] && [ -f "$MXTK_SECRET_VALUES_FILE" ]; then + while IFS= read -r v; do [ -n "$v" ] && printf 'extra\t%s\n' "$v"; done < "$MXTK_SECRET_VALUES_FILE" >> "$SRC" +fi + +n=0 +while IFS="$(printf '\t')" read -r name value; do + [ "${#value}" -ge 6 ] && n=$((n + 1)) +done < "$SRC" +if [ "$n" -eq 0 ]; then echo "check-staged-secrets: no secret values found to scan for (no .env secrets, no run-local.json bootConfig) — nothing checked."; exit 0; fi + +if [ "$MODE" = tree ]; then FILES="$(git ls-files)"; else FILES="$(git diff --cached --name-only --diff-filter=ACMR)"; fi +hits=0 +while IFS= read -r f; do + [ -n "$f" ] || continue + if [ "$MODE" = tree ]; then blob="$(git show "HEAD:$f" 2>/dev/null | LC_ALL=C tr -d '\000')"; else blob="$(git show ":$f" 2>/dev/null | LC_ALL=C tr -d '\000')"; fi + [ -n "$blob" ] || continue + while IFS="$(printf '\t')" read -r name value; do + [ "${#value}" -ge 6 ] || continue + if printf '%s' "$blob" | grep -qF -- "$value"; then + echo " ✗ $f holds the value of $name" + hits=$((hits + 1)) + fi + done < "$SRC" +done <), ignore it, keep the template without the value." + echo " → A value that reached a remote or a transcript is spent: rotate it. Removing the commit does not." + exit 1 +fi +echo "✓ check-staged-secrets: no secret value in the $MODE blobs ($n value(s) checked)." diff --git a/skills/learned-constants-and-secrets.md b/skills/learned-constants-and-secrets.md index 773f485c..831a926b 100644 --- a/skills/learned-constants-and-secrets.md +++ b/skills/learned-constants-and-secrets.md @@ -177,6 +177,35 @@ runtime inside the same misleading error family. --- +## Secret-bearing files and commands — never print them, never stage them + +Six leak events in one five-app lab run (mxcli v0.26.0, Mendix 11.15) reached agent transcripts, and a +rendered login page with the demo password reached git. Every one started with a reasonable +request: "check how the app was started", "see the constant", "list the run state". A transcript is +a place a secret cannot be taken back from, so the rule is about what an agent **prints**, not what +it stores. + +| Never print or paste | Why it holds a secret | Read this instead | +|---|---|---| +| `.mxcli/run-local.json` | `bootConfig` holds the database password and every constant passed at start, in clear. A key filter that looks at top-level keys still prints them | One named field (`app port`, `admin port`, `pid`), or print only a length or an equality result | +| The `mxcli run --local` command line: `ps -o args=`, `start.cmd`, a start script | `--db-password` and `--constant Module.Name=` carry the values | `./bin/app-restart.sh --dry-run` (masks flag values; a `--constant` value is a separate fix, see the toolkit issue list) | +| `.mxcli/run-state.json` and other run records | Same bootConfig copy | Fields by name | +| `describe constant`, `show constants` on a secret constant | They print the default value, which is the secret on a demo | `bin/constants-audit.sh` (verdicts, not values) | +| `.env`, `.env.*` | The values themselves | `grep -c KEY .env`, or test presence with `[ -n "$VAR" ]` | +| Rendered files that interpolate a secret (a login page with the demo password filled in) | Tracked by accident, it puts the value in HEAD | Track only the `*.template.*` file; git-ignore the rendered one | + +Three habits that make it hold: + +1. **Read a secret by testing it, not showing it.** `[ "${#VAR}" -ge 12 ]`, `[ "$A" = "$B" ] && echo same`, an HTTP status from `curl -u "$USER:$PASS"` with output discarded. Never `echo`, never `set -x`. +2. **Put the ban in the lane rules** of any multi-agent build: one line naming these files. An agent that was not told will `cat` them to be helpful. +3. **Scan what git will store.** `bin/check-staged-secrets.sh` (staged blobs) and `--tree` (everything tracked at HEAD) compare your real `.env` and `bootConfig` values against the blobs and print only the file and the key name. It is not wired into the generated pre-commit hook; add it yourself. `git commit -- ` re-adds an ignored tracked file at commit time, so a `.gitignore` entry is not a guard. + +A value that reached a transcript or a remote is spent: **rotate it.** Deleting the commit or the +transcript does not un-leak it. The demo password on a free-node sandbox is the lowest stakes; a +shared database or API credential is not. + +--- + ## Step 4 — Write the decision down where the next session will read it One file per project: `docs/constants-register.md`. It carries two things. diff --git a/tests/wave2/test-check-staged-secrets.sh b/tests/wave2/test-check-staged-secrets.sh new file mode 100644 index 00000000..394966b7 --- /dev/null +++ b/tests/wave2/test-check-staged-secrets.sh @@ -0,0 +1,67 @@ +#!/usr/bin/env bash +# test-check-staged-secrets.sh — project-bin/check-staged-secrets.sh finds a secret VALUE in staged +# blobs and in the tracked tree, names the file and the key, and never prints the value. +# +# Usage: bash tests/wave2/test-check-staged-secrets.sh +# The .env and run-local.json here are written by this test with made-up values; no field capture +# is claimed. Values are arbitrary strings, so the test proves the mechanism, not a real leak. +set -uo pipefail +TOOLKIT="$(cd "$(dirname "${BASH_SOURCE[0]}")/../.." && pwd)" +SUBJECT="$TOOLKIT/project-bin/check-staged-secrets.sh" +W="$(mktemp -d "${TMPDIR:-/tmp}/mxtk-css.XXXXXX")"; trap 'rm -rf "$W"' EXIT +pass=0; fail=0 +ok() { pass=$((pass+1)); echo " ok $1"; } +bad() { fail=$((fail+1)); echo " FAIL $1"; } +has() { case "$2" in *"$3"*) ok "$1" ;; *) bad "$1 (missing '$3')"; echo "$2" | sed 's/^/ /' ;; esac; } +hasnt() { case "$2" in *"$3"*) bad "$1 (leaked '$3')" ;; *) ok "$1" ;; esac; } + +P="$W/proj"; mkdir -p "$P/.mxcli" "$P/bin"; cd "$P" || exit 1 +git init -q . && git config user.email t@t && git config user.name t +cp "$SUBJECT" bin/check-staged-secrets.sh +printf 'DEMO_PASSWORD=Zq7-xylophone-91\nAPP_NAME=shortname\nDB=x\n' > .env +printf '.env\n.mxcli/\n' > .gitignore +cat > .mxcli/run-local.json <<'J' +{"pid": 1, "bootConfig": {"DatabasePassword": "pgpass-Tango-55", "DatabaseName": "appdb", "Cards.HubApiPassword": "hubpass-Delta-77"}} +J +git add .gitignore && git commit -q -m init + +echo "-- nothing staged: clean" +OUT="$(bash bin/check-staged-secrets.sh 2>&1)"; RC=$? +[ "$RC" = 0 ] && ok "clean exits 0" || bad "clean exits 0 (got $RC)" +has "reports how many values it checked" "$OUT" "3 value(s) checked" + +echo "-- a staged file holding a .env value" +printf '\n' > login.html; git add login.html +OUT="$(bash bin/check-staged-secrets.sh 2>&1)"; RC=$? +[ "$RC" = 1 ] && ok "exits 1" || bad "exits 1 (got $RC)" +has "names the file" "$OUT" "login.html" +has "names the key" "$OUT" ".env:DEMO_PASSWORD" +hasnt "never prints the value" "$OUT" "Zq7-xylophone-91" + +echo "-- a staged file holding a run-local.json bootConfig value" +git reset -q login.html; rm login.html +printf 'x=pgpass-Tango-55\n' > notes.txt; git add notes.txt +OUT="$(bash bin/check-staged-secrets.sh 2>&1)"; RC=$? +[ "$RC" = 1 ] && ok "bootConfig value found" || bad "bootConfig value found (got $RC)" +has "names the bootConfig key" "$OUT" "DatabasePassword" +hasnt "never prints the value" "$OUT" "pgpass-Tango-55" + +echo "-- a short or non-secret value is not a source" +git reset -q notes.txt; printf 'shortname appdb\n' > notes.txt; git add notes.txt +OUT="$(bash bin/check-staged-secrets.sh 2>&1)"; RC=$? +[ "$RC" = 0 ] && ok "non-secret keys are not scanned for" || bad "non-secret keys are not scanned for (got $RC: $OUT)" + +echo "-- --tree sees what is already committed" +printf 'hubpass-Delta-77\n' > committed.txt; git add committed.txt; git commit -q -m leak +git reset -q; OUT="$(bash bin/check-staged-secrets.sh --tree 2>&1)"; RC=$? +[ "$RC" = 1 ] && ok "--tree exits 1" || bad "--tree exits 1 (got $RC)" +has "--tree names the file" "$OUT" "committed.txt" +hasnt "--tree never prints the value" "$OUT" "hubpass-Delta-77" + +echo "-- no sources: says so, exits 0" +rm -f .env .mxcli/run-local.json +OUT="$(bash bin/check-staged-secrets.sh 2>&1)"; RC=$? +[ "$RC" = 0 ] && ok "no sources exits 0" || bad "no sources exits 0 (got $RC)" +has "no sources is stated, not silent" "$OUT" "nothing checked" + +echo; echo "$pass passed, $fail failed"; [ "$fail" = 0 ] From e119867b64ac6f0dd41791c1da19964075d6b845 Mon Sep 17 00:00:00 2001 From: MendixMau Date: Fri, 16 Oct 2026 11:11:37 +0800 Subject: [PATCH 2/4] fix(check-staged-secrets): ignore default passwords, git grep speed, .docker/.env, CRLF (TK-025 review) Skip values under 8 chars, well-known defaults (mendix etc.) and values equal to .env.example. One git grep -F -f per value (value via 600 file, not argv). Read .docker/.env and similar. Normalise CRLF, spaces and inline comments. Tests for each. Skill text states what the check does not protect. Co-Authored-By: Claude Opus 5.5 --- CHANGELOG.md | 2 +- project-bin/check-staged-secrets.sh | 125 ++++++++++++++++------- skills/learned-constants-and-secrets.md | 2 +- tests/wave2/test-check-staged-secrets.sh | 40 +++++++- 4 files changed, 130 insertions(+), 39 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index e0aee114..7e7486a8 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -19,7 +19,7 @@ Sections dated before 2026-09-19 predate the cycle and stay as they are. Named m ## Unreleased -- new(security): **`project-bin/check-staged-secrets.sh` — compare the project's real secret values (`.env`, `.mxcli/run-local.json` bootConfig) against the staged blobs (or `--tree`, every tracked file at HEAD) and print only the file and the key name, never a value (TK-025).** Plus a section in `learned-constants-and-secrets.md`: the secret-bearing files and commands an agent never prints (`run-local.json`, the `run --local` command line, run state, `describe constant`, `.env`, rendered files that interpolate a secret), read-by-testing habits, and "a leaked value is rotated, not deleted". Field evidence: a five-app lab run with six leak events into transcripts and a demo password in git (mxcli v0.26.0). Opt-in, not wired into the generated pre-commit hook. Test with made-up values: `tests/wave2/test-check-staged-secrets.sh` (15 checks). Not adopted: the report's `exec-secret.sh` is a project-local script that does not ship here, and masking `--constant` values in `app-restart.sh` is TK-003. — five-app lab report, TK-025 +- new(security): **`project-bin/check-staged-secrets.sh` — compare the project's real secret values (`.env`, `.mxcli/run-local.json` bootConfig) against the staged blobs (or `--tree`, every tracked file at HEAD) and print only the file and the key name, never a value (TK-025).** Plus a section in `learned-constants-and-secrets.md`: the secret-bearing files and commands an agent never prints (`run-local.json`, the `run --local` command line, run state, `describe constant`, `.env`, rendered files that interpolate a secret), read-by-testing habits, and "a leaked value is rotated, not deleted". Field evidence: a five-app lab run with six leak events into transcripts and a demo password in git (mxcli v0.26.0). Opt-in, not wired into the generated pre-commit hook. Test with made-up values: `tests/wave2/test-check-staged-secrets.sh` (15 checks). Not adopted: the report's `exec-secret.sh` is a project-local script that does not ship here, and masking `--constant` values in `app-restart.sh` is TK-003. Values under 8 characters, well-known defaults such as `mendix` (the mxcli database password, present in thousands of tracked files) and values equal to `.env.example` are not sources, and `.docker/.env` is read. It runs one `git grep -F -f` per value, not a process per file: 230 staged files in a 6,000-file synthetic repo took 1.0 s (the whole tree 2.6 s). — five-app lab report, TK-025 - fix(lint-gate): **a ratchet rise now lists the findings that are NEW, not three arbitrary members of the rule.** The baseline keeps a multiset of findings (rule, module, document, message) next to the counts; on a rise the gate prints the surplus per rule, up to three per rule (the rest in `.claude/loop/lint-last.json`), labelled as new or changed findings next to the net rise. An old counts-only baseline falls back to the first three and says they are not necessarily new. Not done here: pre-exec snapshot attribution, a per-slice growth cap, a slice-close lint check (the rest of [#239](https://github.com/MendixMau/mxcli-project-toolkit/issues/239)). Test: `tests/wave2/test-lint-gate-new-findings.sh` — MendixMau (#239) - fix(conformance-check): read ledger rows whose pointer cell is bare (no backticks) and treat status `planned` as not-built; a ledger whose acceptance text is also bare prose is still not measured (acceptance must hold a backticked SHOW/DESCRIBE), but it now exits 2 with a FAULT that says rows were found and the text is prose, instead of "shape changed or glob matched nothing" (TK-006, partial: that ledger shape is diagnosed, not fixed) — field report TK-006 - fix(app-restart): **a secret with spaces in quotes is masked whole, and `Pwd` and `Credential` join the secret words.** Follow-up to #403: `--constant "Mod.ApiKey=two words"` and `--db-password 'two words'` printed everything after the first word (the quoted-flag case leaked before #403 too). `mask()` now masks each argument before quoting it for display, so the whole quoted value goes; the executed command is untouched, and a real-launch test proves the started app receives the raw argv; an env assignment at the start of a `--cmd` string keeps the whole command visible; the per-argument seds only run for arguments holding a secret word. The substring match still over-masks names like `Monkey` (Key) and `Turkey`; that is display only, kept, and pinned by a test. `tests/wave2/test-app-restart.sh` gains 22 cases, 8 of them failing on master. — review of #403 diff --git a/project-bin/check-staged-secrets.sh b/project-bin/check-staged-secrets.sh index 43735b9b..228b3416 100755 --- a/project-bin/check-staged-secrets.sh +++ b/project-bin/check-staged-secrets.sh @@ -10,11 +10,28 @@ # check that sees what git will actually store is a scan of the blobs, against the real values. # # Where the values come from (read here, never echoed): -# .env and .env.* KEY=VALUE where KEY looks like PASSWORD / SECRET / TOKEN / KEY / PAT +# .env .env.* at the project root and in .docker/, docker/, .devcontainer/ (the +# `mxcli docker` layout keeps M2EE_ADMIN_PASS and DB_PASSWORD in +# .docker/.env). KEY=VALUE where KEY looks like PASSWORD / SECRET / +# TOKEN / KEY / PAT. CRLF, `KEY = value`, quotes and an unquoted +# trailing ` # comment` are normalised. # .mxcli/run-local.json every string in bootConfig whose key looks like one of those -# $MXTK_SECRET_VALUES_FILE optional: one literal value per line (name shown as "extra") -# Values shorter than 6 characters are ignored (a scan for "1" would match everything). -# Example files (.env.example, .env.sample, .env.template) are not sources. +# $MXTK_SECRET_VALUES_FILE optional: one literal value per line (name shown as "extra"); +# the filters below do NOT apply, you named it a secret +# What is NOT treated as a secret, and why (this is a leak detector, not a password-strength +# check; a weak demo password is not protected by it): +# - values under 8 characters: too short to scan for without matching ordinary text; +# - well-known development defaults (mendix, password, admin, postgres, changeme, ...): every +# mxcli project carries `mendix` as the database password, so it sits in thousands of tracked +# files (model blobs, java sources, docs) and would fail every commit. A check that is red on +# every commit gets switched off, which is worse than not guarding a published default; +# - a value equal to the same key's value in a sibling .env.example/.sample/.template file: +# that value is already published on purpose. +# Example files are never sources. +# +# Speed: one `git grep -l -F -f <600 temp file>` per value, over the staged paths (--cached) or +# HEAD, instead of a process per file per value. The value goes through a mode 600 file, never +# argv, so `ps` cannot show it. # # Usage: # bin/check-staged-secrets.sh # staged blobs (what the next commit stores) @@ -31,20 +48,41 @@ MODE=staged; [ "${1:-}" = "--tree" ] && MODE=tree PY="$(command -v python3 || command -v python || true)" NAMEPAT='(PASS|SECRET|TOKEN|KEY|PAT$|CREDENTIAL)' -SRC="$(mktemp "${TMPDIR:-/tmp}/mxtk-secrets.XXXXXX")"; trap 'rm -f "$SRC"' EXIT -chmod 600 "$SRC" +MINLEN=8 +# Well-known development defaults, lowercase. Compared case-insensitively. +DEFAULTS=" mendix password passw0rd admin administrator postgres postgresql changeme change-me change_me secret root test testing demo example default " +SRC="$(mktemp "${TMPDIR:-/tmp}/mxtk-secrets.XXXXXX")" +VF="$(mktemp "${TMPDIR:-/tmp}/mxtk-secval.XXXXXX")" +trap 'rm -f "$SRC" "$VF"' EXIT +chmod 600 "$SRC" "$VF" -# namevalue, one per line. -for f in .env .env.*; do +# namevalue, one per line, before filtering. +for f in .env .env.* .docker/.env .docker/.env.* docker/.env docker/.env.* .devcontainer/.env .devcontainer/.env.*; do [ -f "$f" ] || continue case "$f" in *.example|*.sample|*.template) continue ;; esac - sed -E 's/^[[:space:]]*export[[:space:]]+//' "$f" | while IFS= read -r line; do - case "$line" in \#*|"") continue ;; esac - k="${line%%=*}"; v="${line#*=}"; [ "$k" = "$line" ] && continue - v="${v%\"}"; v="${v#\"}"; v="${v%\'}"; v="${v#\'}" - printf '%s' "$k" | grep -qiE "$NAMEPAT" || continue - printf '%s\t%s\n' "$f:$k" "$v" - done >> "$SRC" + d="$(dirname "$f")"; ex="" + for e in "$d/.env.example" "$d/.env.sample" "$d/.env.template"; do [ -f "$e" ] && ex="$ex $e"; done + # shellcheck disable=SC2086 + awk -v NAMEPAT="$NAMEPAT" -v SRCF="$f" ' + function trim(x) { sub(/^[ \t\r]+/, "", x); sub(/[ \t\r]+$/, "", x); return x } + function parse(line, kv, k, v, i) { + sub(/\r$/, "", line); line = trim(line) + if (line == "" || substr(line, 1, 1) == "#") return 0 + sub(/^export[ \t]+/, "", line) + i = index(line, "="); if (i == 0) return 0 + k = trim(substr(line, 1, i - 1)); v = trim(substr(line, i + 1)) + if (v ~ /^"/) { sub(/^"/, "", v); sub(/".*$/, "", v) } + else if (v ~ /^\x27/) { sub(/^\x27/, "", v); sub(/\x27.*$/, "", v) } + else { sub(/[ \t]+#.*$/, "", v); v = trim(v) } + kv[1] = k; kv[2] = v; return 1 + } + FNR == 1 { isex = (FILENAME != SRCF) } + { split("", kv); if (!parse($0, kv)) next + if (isex) { exv[kv[1]] = kv[2]; next } + if (toupper(kv[1]) !~ NAMEPAT) next + if ((kv[1] in exv) && exv[kv[1]] == kv[2]) next + printf "%s:%s\t%s\n", SRCF, kv[1], kv[2] } + ' $ex "$f" >> "$SRC" done if [ -f .mxcli/run-local.json ] && [ -n "$PY" ]; then "$PY" -I - "$NAMEPAT" >> "$SRC" 2>/dev/null <<'PYEOF' @@ -65,36 +103,51 @@ for p, v in walk(data): print(".mxcli/run-local.json:" + p.split("bootConfig.", 1)[-1] + "\t" + v) PYEOF fi -if [ -n "${MXTK_SECRET_VALUES_FILE:-}" ] && [ -f "$MXTK_SECRET_VALUES_FILE" ]; then - while IFS= read -r v; do [ -n "$v" ] && printf 'extra\t%s\n' "$v"; done < "$MXTK_SECRET_VALUES_FILE" >> "$SRC" -fi -n=0 +# Filter: length and known defaults (lowercased compare). Then the explicit extra values. +FILT="$(mktemp "${TMPDIR:-/tmp}/mxtk-secfilt.XXXXXX")"; chmod 600 "$FILT"; trap 'rm -f "$SRC" "$VF" "$FILT"' EXIT +skipped=0 while IFS="$(printf '\t')" read -r name value; do - [ "${#value}" -ge 6 ] && n=$((n + 1)) + [ -n "$name" ] || continue + lv="$(printf '%s' "$value" | tr 'A-Z' 'a-z')" + case "$DEFAULTS" in *" $lv "*) skipped=$((skipped + 1)); continue ;; esac + [ "${#value}" -ge "$MINLEN" ] || { skipped=$((skipped + 1)); continue; } + printf '%s\t%s\n' "$name" "$value" >> "$FILT" done < "$SRC" -if [ "$n" -eq 0 ]; then echo "check-staged-secrets: no secret values found to scan for (no .env secrets, no run-local.json bootConfig) — nothing checked."; exit 0; fi +if [ -n "${MXTK_SECRET_VALUES_FILE:-}" ] && [ -f "$MXTK_SECRET_VALUES_FILE" ]; then + while IFS= read -r v; do v="${v%$(printf '\r')}"; [ -n "$v" ] && printf 'extra\t%s\n' "$v"; done < "$MXTK_SECRET_VALUES_FILE" >> "$FILT" +fi +sort -u "$FILT" -o "$FILT" +n="$(wc -l < "$FILT" | tr -d ' ')" +if [ "$n" -eq 0 ]; then + echo "check-staged-secrets: no secret values to scan for (none found, or $skipped skipped as known defaults / under $MINLEN chars) — nothing checked." + exit 0 +fi -if [ "$MODE" = tree ]; then FILES="$(git ls-files)"; else FILES="$(git diff --cached --name-only --diff-filter=ACMR)"; fi +if [ "$MODE" = staged ]; then + STAGED="$(git diff --cached --name-only -z --diff-filter=ACMR | tr '\0' '\n' | head -c 1)" + if [ -z "$STAGED" ]; then echo "✓ check-staged-secrets: nothing staged ($n value(s) would be checked)."; exit 0; fi +fi hits=0 -while IFS= read -r f; do - [ -n "$f" ] || continue - if [ "$MODE" = tree ]; then blob="$(git show "HEAD:$f" 2>/dev/null | LC_ALL=C tr -d '\000')"; else blob="$(git show ":$f" 2>/dev/null | LC_ALL=C tr -d '\000')"; fi - [ -n "$blob" ] || continue - while IFS="$(printf '\t')" read -r name value; do - [ "${#value}" -ge 6 ] || continue - if printf '%s' "$blob" | grep -qF -- "$value"; then - echo " ✗ $f holds the value of $name" - hits=$((hits + 1)) - fi - done < "$SRC" -done < "$VF" + if [ "$MODE" = tree ]; then + files="$(git grep -l -F -f "$VF" HEAD 2>/dev/null | sed 's/^HEAD://')" + else + files="$(git diff --cached --name-only -z --diff-filter=ACMR | xargs -0 git --literal-pathspecs grep --cached -l -F -f "$VF" -- 2>/dev/null)" + fi + [ -n "$files" ] || continue + while IFS= read -r f; do + echo " ✗ $f holds the value of $name" + hits=$((hits + 1)) + done <), ignore it, keep the template without the value." echo " → A value that reached a remote or a transcript is spent: rotate it. Removing the commit does not." exit 1 fi -echo "✓ check-staged-secrets: no secret value in the $MODE blobs ($n value(s) checked)." +echo "✓ check-staged-secrets: no secret value in the $MODE blobs ($n value(s) checked, $skipped skipped as defaults / under $MINLEN chars)." diff --git a/skills/learned-constants-and-secrets.md b/skills/learned-constants-and-secrets.md index 29ecbd2e..b7628726 100644 --- a/skills/learned-constants-and-secrets.md +++ b/skills/learned-constants-and-secrets.md @@ -198,7 +198,7 @@ Three habits that make it hold: 1. **Read a secret by testing it, not showing it.** `[ "${#VAR}" -ge 12 ]`, `[ "$A" = "$B" ] && echo same`, an HTTP status from `curl -u "$USER:$PASS"` with output discarded. Never `echo`, never `set -x`. 2. **Put the ban in the lane rules** of any multi-agent build: one line naming these files. An agent that was not told will `cat` them to be helpful. -3. **Scan what git will store.** `bin/check-staged-secrets.sh` (staged blobs) and `--tree` (everything tracked at HEAD) compare your real `.env` and `bootConfig` values against the blobs and print only the file and the key name. It is not wired into the generated pre-commit hook; add it yourself. `git commit -- ` re-adds an ignored tracked file at commit time, so a `.gitignore` entry is not a guard. +3. **Scan what git will store.** `bin/check-staged-secrets.sh` (staged blobs) and `--tree` (everything tracked at HEAD) compare your real `.env`, `.docker/.env` and `bootConfig` values against the blobs and print only the file and the key name. It skips values under 8 characters, well-known defaults such as `mendix`, and a value equal to the same key in `.env.example`, so a weak demo password like `mendix` is NOT protected by it (the default sits in thousands of tracked files, and a check that fails every commit gets turned off). It is not wired into the generated pre-commit hook; add it yourself. `git commit -- ` re-adds an ignored tracked file at commit time, so a `.gitignore` entry is not a guard. A value that reached a transcript or a remote is spent: **rotate it.** Deleting the commit or the transcript does not un-leak it. The demo password on a free-node sandbox is the lowest stakes; a diff --git a/tests/wave2/test-check-staged-secrets.sh b/tests/wave2/test-check-staged-secrets.sh index 394966b7..d352d2e1 100644 --- a/tests/wave2/test-check-staged-secrets.sh +++ b/tests/wave2/test-check-staged-secrets.sh @@ -25,11 +25,13 @@ cat > .mxcli/run-local.json <<'J' J git add .gitignore && git commit -q -m init -echo "-- nothing staged: clean" +echo "-- a clean staged file: clean" +printf 'hello\n' > hello.txt; git add hello.txt OUT="$(bash bin/check-staged-secrets.sh 2>&1)"; RC=$? [ "$RC" = 0 ] && ok "clean exits 0" || bad "clean exits 0 (got $RC)" has "reports how many values it checked" "$OUT" "3 value(s) checked" +git reset -q hello.txt; rm hello.txt echo "-- a staged file holding a .env value" printf '\n' > login.html; git add login.html OUT="$(bash bin/check-staged-secrets.sh 2>&1)"; RC=$? @@ -64,4 +66,40 @@ OUT="$(bash bin/check-staged-secrets.sh 2>&1)"; RC=$? [ "$RC" = 0 ] && ok "no sources exits 0" || bad "no sources exits 0 (got $RC)" has "no sources is stated, not silent" "$OUT" "nothing checked" +echo "-- the default database password is not a secret (review: mass false positives)" +mkdir -p .mxcli; printf '{"bootConfig": {"DatabasePassword": "mendix"}}' > .mxcli/run-local.json +printf 'DB_PASSWORD=mendix\nOTHER_PASSWORD=Password\n' > .env +printf 'mendix Mendix password\n' > notes.txt; git add notes.txt +OUT="$(bash bin/check-staged-secrets.sh 2>&1)"; RC=$? +[ "$RC" = 0 ] && ok "staged 'mendix' with default bootConfig/.env password exits 0" || bad "default password exits 0 (got $RC: $OUT)" +has "says it skipped defaults" "$OUT" "nothing checked" +git reset -q notes.txt + +echo "-- .docker/.env is a source; CRLF, spaces and inline comment are normalised" +rm -f .env .mxcli/run-local.json; mkdir -p .docker +printf '%s = %s # local only\r\n%s=%s\r\n%s="%s"\r\n' M2EE_ADMIN_PASS "Kx9-admin-Quartz-31" DB_PASSWORD mendix TOKEN "Rv4-token-Magnolia-08" > .docker/.env +printf 'a Kx9-admin-Quartz-31 b\n' > one.txt; printf 'Rv4-token-Magnolia-08\n' > two.txt; git add one.txt two.txt +OUT="$(bash bin/check-staged-secrets.sh 2>&1)"; RC=$? +[ "$RC" = 1 ] && ok ".docker/.env value found" || bad ".docker/.env value found (got $RC)" +has "names the .docker/.env key" "$OUT" ".docker/.env:M2EE_ADMIN_PASS" +has "CRLF quoted value matches" "$OUT" "two.txt holds the value of .docker/.env:TOKEN" +hasnt "never prints the value" "$OUT" "Kx9-admin-Quartz-31" +hasnt "never prints a CRLF value" "$OUT" "Rv4-token-Magnolia-08" +git reset -q one.txt two.txt + +echo "-- a value equal to .env.example for the same key is published, not a secret" +printf 'API_TOKEN=Published-Example-99\n' > .docker/.env.example +printf 'API_TOKEN=Published-Example-99\n' > .docker/.env +printf 'Published-Example-99\n' > pub.txt; git add pub.txt +OUT="$(bash bin/check-staged-secrets.sh 2>&1)"; RC=$? +[ "$RC" = 0 ] && ok "example-equal value ignored" || bad "example-equal value ignored (got $RC: $OUT)" +git reset -q pub.txt + +echo "-- MXTK_SECRET_VALUES_FILE is not filtered" +printf 'mendix\n' > "$W/extra.txt"; printf 'mendix\n' > tracked.txt; git add tracked.txt; git commit -q -m t; git reset -q +OUT="$(MXTK_SECRET_VALUES_FILE="$W/extra.txt" bash bin/check-staged-secrets.sh --tree 2>&1)"; RC=$? +[ "$RC" = 1 ] && ok "explicit extra value 'mendix' is found (rc 1)" || bad "extra value (rc $RC)" +OUT="$(bash bin/check-staged-secrets.sh --tree 2>&1)"; RC=$? +[ "$RC" = 0 ] && ok "without the extra file the default is ignored in --tree" || bad "tree default (rc $RC: $OUT)" + echo; echo "$pass passed, $fail failed"; [ "$fail" = 0 ] From 61ba350da55bf35aae05be782b9207f1f65bdac7 Mon Sep 17 00:00:00 2001 From: MendixMau Date: Fri, 16 Oct 2026 11:32:27 +0800 Subject: [PATCH 3/4] fix(secrets): skip MicroflowConstants values already in the committed model run --local writes every constant's model default into bootConfig, so a placeholder default of a key-like constant counted as a secret and could not be unstaged. Values present at HEAD in *.mpr or mprcontents/ are skipped; a constant passed with a value not in the model is still found. Co-Authored-By: Claude Opus 5.5 --- project-bin/check-staged-secrets.sh | 12 ++++++++++++ skills/learned-constants-and-secrets.md | 2 +- tests/wave2/test-check-staged-secrets.sh | 17 +++++++++++++++++ 3 files changed, 30 insertions(+), 1 deletion(-) diff --git a/project-bin/check-staged-secrets.sh b/project-bin/check-staged-secrets.sh index 228b3416..25ec0aa7 100755 --- a/project-bin/check-staged-secrets.sh +++ b/project-bin/check-staged-secrets.sh @@ -16,6 +16,13 @@ # TOKEN / KEY / PAT. CRLF, `KEY = value`, quotes and an unquoted # trailing ` # comment` are normalised. # .mxcli/run-local.json every string in bootConfig whose key looks like one of those +# Exception: a bootConfig.MicroflowConstants value that is already in +# the committed model (HEAD, *.mpr or mprcontents/). `run --local` writes +# EVERY constant's model default there, so a placeholder default such as +# `your-api-key-here` is the model's own text, not a leak, and unstaging +# cannot fix it. A real credential in a model default is already +# committed: that is bin/constants-audit.sh's job, not this script's. A +# constant passed with a value that is NOT in the model is still a source. # $MXTK_SECRET_VALUES_FILE optional: one literal value per line (name shown as "extra"); # the filters below do NOT apply, you named it a secret # What is NOT treated as a secret, and why (this is a leak detector, not a password-strength @@ -112,6 +119,11 @@ while IFS="$(printf '\t')" read -r name value; do lv="$(printf '%s' "$value" | tr 'A-Z' 'a-z')" case "$DEFAULTS" in *" $lv "*) skipped=$((skipped + 1)); continue ;; esac [ "${#value}" -ge "$MINLEN" ] || { skipped=$((skipped + 1)); continue; } + case "$name" in + .mxcli/run-local.json:MicroflowConstants.*) + printf '%s\n' "$value" > "$VF" + if git grep -q -F -f "$VF" HEAD -- '*.mpr' 'mprcontents/' 2>/dev/null; then skipped=$((skipped + 1)); continue; fi ;; + esac printf '%s\t%s\n' "$name" "$value" >> "$FILT" done < "$SRC" if [ -n "${MXTK_SECRET_VALUES_FILE:-}" ] && [ -f "$MXTK_SECRET_VALUES_FILE" ]; then diff --git a/skills/learned-constants-and-secrets.md b/skills/learned-constants-and-secrets.md index b7628726..f9ab562e 100644 --- a/skills/learned-constants-and-secrets.md +++ b/skills/learned-constants-and-secrets.md @@ -198,7 +198,7 @@ Three habits that make it hold: 1. **Read a secret by testing it, not showing it.** `[ "${#VAR}" -ge 12 ]`, `[ "$A" = "$B" ] && echo same`, an HTTP status from `curl -u "$USER:$PASS"` with output discarded. Never `echo`, never `set -x`. 2. **Put the ban in the lane rules** of any multi-agent build: one line naming these files. An agent that was not told will `cat` them to be helpful. -3. **Scan what git will store.** `bin/check-staged-secrets.sh` (staged blobs) and `--tree` (everything tracked at HEAD) compare your real `.env`, `.docker/.env` and `bootConfig` values against the blobs and print only the file and the key name. It skips values under 8 characters, well-known defaults such as `mendix`, and a value equal to the same key in `.env.example`, so a weak demo password like `mendix` is NOT protected by it (the default sits in thousands of tracked files, and a check that fails every commit gets turned off). It is not wired into the generated pre-commit hook; add it yourself. `git commit -- ` re-adds an ignored tracked file at commit time, so a `.gitignore` entry is not a guard. +3. **Scan what git will store.** `bin/check-staged-secrets.sh` (staged blobs) and `--tree` (everything tracked at HEAD) compare your real `.env`, `.docker/.env` and `bootConfig` values against the blobs and print only the file and the key name. It skips values under 8 characters, well-known defaults such as `mendix`, a value equal to the same key in `.env.example`, and a `bootConfig.MicroflowConstants` value that is already in the committed model (`run --local` writes every constant's model default there; a credential in a model default is `bin/constants-audit.sh`'s job), so a weak demo password like `mendix` is NOT protected by it (the default sits in thousands of tracked files, and a check that fails every commit gets turned off). It is not wired into the generated pre-commit hook; add it yourself. `git commit -- ` re-adds an ignored tracked file at commit time, so a `.gitignore` entry is not a guard. A value that reached a transcript or a remote is spent: **rotate it.** Deleting the commit or the transcript does not un-leak it. The demo password on a free-node sandbox is the lowest stakes; a diff --git a/tests/wave2/test-check-staged-secrets.sh b/tests/wave2/test-check-staged-secrets.sh index d352d2e1..b7f0d6ba 100644 --- a/tests/wave2/test-check-staged-secrets.sh +++ b/tests/wave2/test-check-staged-secrets.sh @@ -95,6 +95,23 @@ OUT="$(bash bin/check-staged-secrets.sh 2>&1)"; RC=$? [ "$RC" = 0 ] && ok "example-equal value ignored" || bad "example-equal value ignored (got $RC: $OUT)" git reset -q pub.txt +echo "-- run --local writes every model default into MicroflowConstants: a default already in the model is not a source" +rm -rf .docker; rm -f .env; mkdir -p mprcontents/ab +printf 'default: your-api-key-here\n' > mprcontents/ab/unit1.mxunit; git add mprcontents; git commit -q -m model +printf '{"bootConfig": {"MicroflowConstants": {"MyModule.ApiKey": "your-api-key-here", "MyModule.AuthToken": "passed-Cedar-4821"}}}' > .mxcli/run-local.json +printf 'your-api-key-here\n' > unit2.mxunit; git add unit2.mxunit +OUT="$(bash bin/check-staged-secrets.sh 2>&1)"; RC=$? +[ "$RC" = 0 ] && ok "placeholder model default staged: exit 0" || bad "placeholder default staged (got $RC: $OUT)" +git reset -q unit2.mxunit +OUT="$(bash bin/check-staged-secrets.sh --tree 2>&1)"; RC=$? +[ "$RC" = 0 ] && ok "placeholder model default in --tree: exit 0" || bad "placeholder default --tree (got $RC: $OUT)" +printf 'x=passed-Cedar-4821\n' > leak.txt; git add leak.txt +OUT="$(bash bin/check-staged-secrets.sh 2>&1)"; RC=$? +[ "$RC" = 1 ] && ok "constant value NOT in the model is still found" || bad "constant not in model (got $RC)" +has "names the constant" "$OUT" "MicroflowConstants.MyModule.AuthToken" +hasnt "never prints the value" "$OUT" "passed-Cedar-4821" +git reset -q leak.txt; rm -f leak.txt + echo "-- MXTK_SECRET_VALUES_FILE is not filtered" printf 'mendix\n' > "$W/extra.txt"; printf 'mendix\n' > tracked.txt; git add tracked.txt; git commit -q -m t; git reset -q OUT="$(MXTK_SECRET_VALUES_FILE="$W/extra.txt" bash bin/check-staged-secrets.sh --tree 2>&1)"; RC=$? From c28de7a296ee1c6848c10017b39a320c9d6e29c6 Mon Sep 17 00:00:00 2001 From: MendixMau Date: Fri, 16 Oct 2026 16:12:58 +0800 Subject: [PATCH 4/4] fix(check-staged-secrets): resolve Python through resolve_py, not a bare name Same non-fatal probe exec.sh uses; a local fallback when _common.sh is absent. Co-Authored-By: Claude Opus 5.5 --- project-bin/check-staged-secrets.sh | 19 ++++++++++++++++++- 1 file changed, 18 insertions(+), 1 deletion(-) diff --git a/project-bin/check-staged-secrets.sh b/project-bin/check-staged-secrets.sh index 25ec0aa7..f00b21bc 100755 --- a/project-bin/check-staged-secrets.sh +++ b/project-bin/check-staged-secrets.sh @@ -48,11 +48,28 @@ # # Exit: 0 clean (or no sources found) · 1 a secret value is in the scanned blobs · 2 not a git repo set -u +SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)" PROJECT_ROOT="${PROJECT_ROOT:-$(cd "$(dirname "${BASH_SOURCE[0]}")/.." && pwd)}" cd "$PROJECT_ROOT" || { echo "check-staged-secrets: cannot cd to $PROJECT_ROOT" >&2; exit 2; } git rev-parse --show-toplevel >/dev/null 2>&1 || { echo "check-staged-secrets: not a git repository" >&2; exit 2; } MODE=staged; [ "${1:-}" = "--tree" ] && MODE=tree -PY="$(command -v python3 || command -v python || true)" +# Non-fatal interpreter probe: without Python the named-key scan is skipped, the rest still runs. +# resolve_py is _common.sh's (portable.sh is toolkit-side and not copied into projects). +[ -f "$SCRIPT_DIR/_common.sh" ] && . "$SCRIPT_DIR/_common.sh" +if ! type resolve_py >/dev/null 2>&1; then + resolve_py() { + _c="" + for _c in "${PYTHON:-}" python3 python py; do # portability-ok: this IS the interpreter probe + [ -n "$_c" ] || continue + case "$(command -v "$_c" 2>/dev/null)" in *[Ww]indows[Aa]pps*) continue ;; esac + if "$_c" -c 'import sys; sys.exit(0 if sys.version_info[0] == 3 else 1)' >/dev/null 2>&1; then + echo "$_c"; return 0 + fi + done + return 1 + } +fi +PY="$(resolve_py || true)" NAMEPAT='(PASS|SECRET|TOKEN|KEY|PAT$|CREDENTIAL)' MINLEN=8