diff --git a/CHANGELOG.md b/CHANGELOG.md index 95bed483..2d7e1ea6 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -19,6 +19,7 @@ Sections dated before 2026-09-19 predate the cycle and stay as they are. Named m ## Unreleased +- new(security): **`project-bin/check-staged-secrets.sh` — compare the project's real secret values (`.env`, `.mxcli/run-local.json` bootConfig) against the staged blobs (or `--tree`, every tracked file at HEAD) and print only the file and the key name, never a value (TK-025).** Plus a section in `learned-constants-and-secrets.md`: the secret-bearing files and commands an agent never prints (`run-local.json`, the `run --local` command line, run state, `describe constant`, `.env`, rendered files that interpolate a secret), read-by-testing habits, and "a leaked value is rotated, not deleted". Field evidence: a five-app lab run with six leak events into transcripts and a demo password in git (mxcli v0.26.0). Opt-in, not wired into the generated pre-commit hook. Test with made-up values: `tests/wave2/test-check-staged-secrets.sh` (15 checks). Not adopted: the report's `exec-secret.sh` is a project-local script that does not ship here, and masking `--constant` values in `app-restart.sh` is TK-003. Values under 8 characters, well-known defaults such as `mendix` (the mxcli database password, present in thousands of tracked files) and values equal to `.env.example` are not sources, and `.docker/.env` is read. It runs one `git grep -F -f` per value, not a process per file: 230 staged files in a 6,000-file synthetic repo took 1.0 s (the whole tree 2.6 s). — five-app lab report, TK-025 - fix(wire-agents): **a sync re-wire no longer wipes `.claude/settings.json`.** `mxcli init` overwrites that file; wire-agents now snapshots it and merges after init: the project's entries win and keep their order (permission profile, user entries, the `mxcli exec` deny rule, hooks), and entries that only mxcli's new file has are appended. Merged rather than added to PRESERVE because PRESERVE restores old bytes and would also drop anything a newer mxcli adds; an unparsable snapshot is restored byte for byte. Test: `tests/wave2/test-wire-agents-settings.sh` (4 of 6 checks fail on master). — review of #414 (#419) - fix(sync): **the exec-approval item-3 rewrite now works on BSD awk (stock macOS) and `sync-project.sh` no longer prints "Rewrote" for a rewrite that failed.** `_ea_rewrite_item3` passed the multi-line replacement through `awk -v`, which BSD awk rejects ("newline in string", exit 2); the file stayed unchanged and sync still reported the rewrite on every run. The replacement is now streamed from a temp file (as the Q9 repair does), the helper returns 2 when awk fails, and the caller prints a `WARN:` instead of "Rewrote" and does not count a change. `test-exec-approval.sh` gains 3 assertions (failing-awk shim: no "Rewrote", a WARN, file unchanged). No other multi-line `awk -v` call site exists (the rest are single-line values) ([#426](https://github.com/MendixMau/mxcli-project-toolkit/issues/426)) — MendixMau - process(testing): **unit tests are now off by default and `affected-tests.sh` enforces it; each project picks its test strategy with the developer at CAC-5.** Fast-tracked by the maintainer on 2026-10-16 because users report tests firing on every change (the baseline-text A/B is waived; E3 follows as a check). Moments: every change = gate plus what it touched (unit only for affected modules with unit on); module done = that module's journeys (+ its unit tests if on); build complete and before every later deploy = everything once; "run everything" any time. The CAC-5 Offer records it in PROJECT.md `## Test strategy` (Status `ASSUMED` in auto mode). `affected-tests.sh` reads `Unit tests default:` (`no` or absent means unit off, one `unit off` line, never a `.test.mdl` run row); turn unit on per project with `Unit tests default: yes`, per module with `Unit modules: Shop, Admin` in PROJECT.md or `Unit tests: yes` in the module brief; the strategy row's Unit cell is a scope only and never turns unit on. A grant on a microflow or page the same script changes is part of that subject and no longer selects the owning module's tests. `module-review.md` PROVE now runs `--moment module --module `. `testing-shape.md`, `checkpoint-build.md`, `iterative-build-loop.md`, `conversion-runbook.md` and the exec.sh logic lane text updated. `tests/wave2/test-test-strategy-docs.sh` and the extended `test-affected-tests.sh` pin it. Baseline words stay at or below master (75,219 vs 75,225). Split out of #409; found by the field test of the release candidate — the maintainer's decision, 2026-10-16 diff --git a/bin/lib/install-manifest.sh b/bin/lib/install-manifest.sh index d7557919..729457e2 100644 --- a/bin/lib/install-manifest.sh +++ b/bin/lib/install-manifest.sh @@ -115,7 +115,7 @@ MXTK_AGENTS_STAGE_BUILD="mdl-agent.md gate-agent.md test-agent.md review-agent.m # may run itself (stops only the pid run --local recorded for THIS project), and `mxcli test` on a # scratch copy under the test-port lock. exec.sh names app-restart.sh in its watch advisory, so # it has to be installed beside it. -MXTK_PROJECT_BIN="app-restart.sh test-scratch.sh assemble-prototype.js prototype-route.js check-prototype-links.js wf-add-path-terminators.py wf-set-call-captions.py mxunit_bson.py mxpatch.py _common.sh app-facts.sh _claims.sh snapshot-mpr.sh restore-mpr.sh exec.sh affected-tests.sh look-ledger.sh save-sp.sh restart-sp.sh check-sp-health.sh verify-module.sh test-stack-up.sh fixture-manifest.sh check-root-clean.sh lint-gate.sh lint-trend.sh metrics-ledger.sh metrics-trend.sh close-task.sh conformance-check.sh coverage-preflight.sh graph-sweep.sh review-module.sh coherence-cadence.sh build-plan-status.sh done-drift-check.sh page-scope.sh render-improvement-register.sh check-design-portability.sh check-design-reaches-app.sh check-page-shell.sh page-fidelity.js model-stamp.sh verify-model.sh install-project-hooks.sh mx-metadata-note.py session-check.sh constants-audit.sh context-pack.sh" +MXTK_PROJECT_BIN="app-restart.sh test-scratch.sh assemble-prototype.js prototype-route.js check-prototype-links.js wf-add-path-terminators.py wf-set-call-captions.py mxunit_bson.py mxpatch.py _common.sh app-facts.sh _claims.sh snapshot-mpr.sh restore-mpr.sh exec.sh affected-tests.sh look-ledger.sh save-sp.sh restart-sp.sh check-sp-health.sh verify-module.sh test-stack-up.sh fixture-manifest.sh check-root-clean.sh lint-gate.sh lint-trend.sh metrics-ledger.sh metrics-trend.sh close-task.sh conformance-check.sh coverage-preflight.sh graph-sweep.sh review-module.sh coherence-cadence.sh build-plan-status.sh done-drift-check.sh page-scope.sh render-improvement-register.sh check-design-portability.sh check-design-reaches-app.sh check-page-shell.sh page-fidelity.js model-stamp.sh verify-model.sh install-project-hooks.sh mx-metadata-note.py session-check.sh constants-audit.sh context-pack.sh check-staged-secrets.sh" # Files in project-bin/ that are deliberately NOT installed into projects. The reverse check # below flags anything named by NEITHER list, so a new file in project-bin/ has to be either diff --git a/project-bin/check-staged-secrets.sh b/project-bin/check-staged-secrets.sh new file mode 100755 index 00000000..f00b21bc --- /dev/null +++ b/project-bin/check-staged-secrets.sh @@ -0,0 +1,182 @@ +#!/usr/bin/env bash +# check-staged-secrets.sh — fail when a secret VALUE from this project's own secret-bearing files +# is about to be committed (staged blobs) or is already in the tracked tree. Prints the file and +# the NAME of the key that matched. It never prints a value. +# +# WHY THIS EXISTS (field run, five apps, mxcli v0.26.0). A demo password reached git through a +# rendered login page, and several lane transcripts through `.mxcli/run-local.json`, whose +# bootConfig holds database and API passwords in clear. `.gitignore` does not help once a file is +# tracked, and `git commit -- ` re-adds an ignored tracked file at commit time. The one +# check that sees what git will actually store is a scan of the blobs, against the real values. +# +# Where the values come from (read here, never echoed): +# .env .env.* at the project root and in .docker/, docker/, .devcontainer/ (the +# `mxcli docker` layout keeps M2EE_ADMIN_PASS and DB_PASSWORD in +# .docker/.env). KEY=VALUE where KEY looks like PASSWORD / SECRET / +# TOKEN / KEY / PAT. CRLF, `KEY = value`, quotes and an unquoted +# trailing ` # comment` are normalised. +# .mxcli/run-local.json every string in bootConfig whose key looks like one of those +# Exception: a bootConfig.MicroflowConstants value that is already in +# the committed model (HEAD, *.mpr or mprcontents/). `run --local` writes +# EVERY constant's model default there, so a placeholder default such as +# `your-api-key-here` is the model's own text, not a leak, and unstaging +# cannot fix it. A real credential in a model default is already +# committed: that is bin/constants-audit.sh's job, not this script's. A +# constant passed with a value that is NOT in the model is still a source. +# $MXTK_SECRET_VALUES_FILE optional: one literal value per line (name shown as "extra"); +# the filters below do NOT apply, you named it a secret +# What is NOT treated as a secret, and why (this is a leak detector, not a password-strength +# check; a weak demo password is not protected by it): +# - values under 8 characters: too short to scan for without matching ordinary text; +# - well-known development defaults (mendix, password, admin, postgres, changeme, ...): every +# mxcli project carries `mendix` as the database password, so it sits in thousands of tracked +# files (model blobs, java sources, docs) and would fail every commit. A check that is red on +# every commit gets switched off, which is worse than not guarding a published default; +# - a value equal to the same key's value in a sibling .env.example/.sample/.template file: +# that value is already published on purpose. +# Example files are never sources. +# +# Speed: one `git grep -l -F -f <600 temp file>` per value, over the staged paths (--cached) or +# HEAD, instead of a process per file per value. The value goes through a mode 600 file, never +# argv, so `ps` cannot show it. +# +# Usage: +# bin/check-staged-secrets.sh # staged blobs (what the next commit stores) +# bin/check-staged-secrets.sh --tree # every tracked file at HEAD (the "already pushed?" check) +# Hook it yourself (it is NOT wired into the generated pre-commit): +# "$(git rev-parse --show-toplevel)/bin/check-staged-secrets.sh" || exit 1 +# +# Exit: 0 clean (or no sources found) · 1 a secret value is in the scanned blobs · 2 not a git repo +set -u +SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)" +PROJECT_ROOT="${PROJECT_ROOT:-$(cd "$(dirname "${BASH_SOURCE[0]}")/.." && pwd)}" +cd "$PROJECT_ROOT" || { echo "check-staged-secrets: cannot cd to $PROJECT_ROOT" >&2; exit 2; } +git rev-parse --show-toplevel >/dev/null 2>&1 || { echo "check-staged-secrets: not a git repository" >&2; exit 2; } +MODE=staged; [ "${1:-}" = "--tree" ] && MODE=tree +# Non-fatal interpreter probe: without Python the named-key scan is skipped, the rest still runs. +# resolve_py is _common.sh's (portable.sh is toolkit-side and not copied into projects). +[ -f "$SCRIPT_DIR/_common.sh" ] && . "$SCRIPT_DIR/_common.sh" +if ! type resolve_py >/dev/null 2>&1; then + resolve_py() { + _c="" + for _c in "${PYTHON:-}" python3 python py; do # portability-ok: this IS the interpreter probe + [ -n "$_c" ] || continue + case "$(command -v "$_c" 2>/dev/null)" in *[Ww]indows[Aa]pps*) continue ;; esac + if "$_c" -c 'import sys; sys.exit(0 if sys.version_info[0] == 3 else 1)' >/dev/null 2>&1; then + echo "$_c"; return 0 + fi + done + return 1 + } +fi +PY="$(resolve_py || true)" + +NAMEPAT='(PASS|SECRET|TOKEN|KEY|PAT$|CREDENTIAL)' +MINLEN=8 +# Well-known development defaults, lowercase. Compared case-insensitively. +DEFAULTS=" mendix password passw0rd admin administrator postgres postgresql changeme change-me change_me secret root test testing demo example default " +SRC="$(mktemp "${TMPDIR:-/tmp}/mxtk-secrets.XXXXXX")" +VF="$(mktemp "${TMPDIR:-/tmp}/mxtk-secval.XXXXXX")" +trap 'rm -f "$SRC" "$VF"' EXIT +chmod 600 "$SRC" "$VF" + +# namevalue, one per line, before filtering. +for f in .env .env.* .docker/.env .docker/.env.* docker/.env docker/.env.* .devcontainer/.env .devcontainer/.env.*; do + [ -f "$f" ] || continue + case "$f" in *.example|*.sample|*.template) continue ;; esac + d="$(dirname "$f")"; ex="" + for e in "$d/.env.example" "$d/.env.sample" "$d/.env.template"; do [ -f "$e" ] && ex="$ex $e"; done + # shellcheck disable=SC2086 + awk -v NAMEPAT="$NAMEPAT" -v SRCF="$f" ' + function trim(x) { sub(/^[ \t\r]+/, "", x); sub(/[ \t\r]+$/, "", x); return x } + function parse(line, kv, k, v, i) { + sub(/\r$/, "", line); line = trim(line) + if (line == "" || substr(line, 1, 1) == "#") return 0 + sub(/^export[ \t]+/, "", line) + i = index(line, "="); if (i == 0) return 0 + k = trim(substr(line, 1, i - 1)); v = trim(substr(line, i + 1)) + if (v ~ /^"/) { sub(/^"/, "", v); sub(/".*$/, "", v) } + else if (v ~ /^\x27/) { sub(/^\x27/, "", v); sub(/\x27.*$/, "", v) } + else { sub(/[ \t]+#.*$/, "", v); v = trim(v) } + kv[1] = k; kv[2] = v; return 1 + } + FNR == 1 { isex = (FILENAME != SRCF) } + { split("", kv); if (!parse($0, kv)) next + if (isex) { exv[kv[1]] = kv[2]; next } + if (toupper(kv[1]) !~ NAMEPAT) next + if ((kv[1] in exv) && exv[kv[1]] == kv[2]) next + printf "%s:%s\t%s\n", SRCF, kv[1], kv[2] } + ' $ex "$f" >> "$SRC" +done +if [ -f .mxcli/run-local.json ] && [ -n "$PY" ]; then + "$PY" -I - "$NAMEPAT" >> "$SRC" 2>/dev/null <<'PYEOF' +import json, re, sys +pat = re.compile(sys.argv[1], re.I) +try: + data = json.load(open(".mxcli/run-local.json")) +except Exception: + sys.exit(0) +def walk(o, path=""): + if isinstance(o, dict): + for k, v in o.items(): + yield from walk(v, path + "." + k if path else k) + elif isinstance(o, str) and "bootConfig" in path: + yield path, o +for p, v in walk(data): + if pat.search(p.rsplit(".", 1)[-1]) and "\t" not in v and "\n" not in v: + print(".mxcli/run-local.json:" + p.split("bootConfig.", 1)[-1] + "\t" + v) +PYEOF +fi + +# Filter: length and known defaults (lowercased compare). Then the explicit extra values. +FILT="$(mktemp "${TMPDIR:-/tmp}/mxtk-secfilt.XXXXXX")"; chmod 600 "$FILT"; trap 'rm -f "$SRC" "$VF" "$FILT"' EXIT +skipped=0 +while IFS="$(printf '\t')" read -r name value; do + [ -n "$name" ] || continue + lv="$(printf '%s' "$value" | tr 'A-Z' 'a-z')" + case "$DEFAULTS" in *" $lv "*) skipped=$((skipped + 1)); continue ;; esac + [ "${#value}" -ge "$MINLEN" ] || { skipped=$((skipped + 1)); continue; } + case "$name" in + .mxcli/run-local.json:MicroflowConstants.*) + printf '%s\n' "$value" > "$VF" + if git grep -q -F -f "$VF" HEAD -- '*.mpr' 'mprcontents/' 2>/dev/null; then skipped=$((skipped + 1)); continue; fi ;; + esac + printf '%s\t%s\n' "$name" "$value" >> "$FILT" +done < "$SRC" +if [ -n "${MXTK_SECRET_VALUES_FILE:-}" ] && [ -f "$MXTK_SECRET_VALUES_FILE" ]; then + while IFS= read -r v; do v="${v%$(printf '\r')}"; [ -n "$v" ] && printf 'extra\t%s\n' "$v"; done < "$MXTK_SECRET_VALUES_FILE" >> "$FILT" +fi +sort -u "$FILT" -o "$FILT" +n="$(wc -l < "$FILT" | tr -d ' ')" +if [ "$n" -eq 0 ]; then + echo "check-staged-secrets: no secret values to scan for (none found, or $skipped skipped as known defaults / under $MINLEN chars) — nothing checked." + exit 0 +fi + +if [ "$MODE" = staged ]; then + STAGED="$(git diff --cached --name-only -z --diff-filter=ACMR | tr '\0' '\n' | head -c 1)" + if [ -z "$STAGED" ]; then echo "✓ check-staged-secrets: nothing staged ($n value(s) would be checked)."; exit 0; fi +fi +hits=0 +while IFS="$(printf '\t')" read -r name value; do + printf '%s\n' "$value" > "$VF" + if [ "$MODE" = tree ]; then + files="$(git grep -l -F -f "$VF" HEAD 2>/dev/null | sed 's/^HEAD://')" + else + files="$(git diff --cached --name-only -z --diff-filter=ACMR | xargs -0 git --literal-pathspecs grep --cached -l -F -f "$VF" -- 2>/dev/null)" + fi + [ -n "$files" ] || continue + while IFS= read -r f; do + echo " ✗ $f holds the value of $name" + hits=$((hits + 1)) + done <), ignore it, keep the template without the value." + echo " → A value that reached a remote or a transcript is spent: rotate it. Removing the commit does not." + exit 1 +fi +echo "✓ check-staged-secrets: no secret value in the $MODE blobs ($n value(s) checked, $skipped skipped as defaults / under $MINLEN chars)." diff --git a/skills/learned-constants-and-secrets.md b/skills/learned-constants-and-secrets.md index ce330cc8..f9ab562e 100644 --- a/skills/learned-constants-and-secrets.md +++ b/skills/learned-constants-and-secrets.md @@ -177,6 +177,35 @@ runtime inside the same misleading error family. --- +## Secret-bearing files and commands — never print them, never stage them + +Six leak events in one five-app lab run (mxcli v0.26.0, Mendix 11.15) reached agent transcripts, and a +rendered login page with the demo password reached git. Every one started with a reasonable +request: "check how the app was started", "see the constant", "list the run state". A transcript is +a place a secret cannot be taken back from, so the rule is about what an agent **prints**, not what +it stores. + +| Never print or paste | Why it holds a secret | Read this instead | +|---|---|---| +| `.mxcli/run-local.json` | `bootConfig` holds the database password and every constant passed at start, in clear. A key filter that looks at top-level keys still prints them | One named field (`app port`, `admin port`, `pid`), or print only a length or an equality result | +| The `mxcli run --local` command line: `ps -o args=`, `start.cmd`, a start script | `--db-password` and `--constant Module.Name=` carry the values | `./bin/app-restart.sh --dry-run` (masks flag values; a `--constant` value is a separate fix, see the toolkit issue list) | +| `.mxcli/run-state.json` and other run records | Same bootConfig copy | Fields by name | +| `describe constant`, `show constants` on a secret constant | They print the default value, which is the secret on a demo | `bin/constants-audit.sh` (verdicts, not values) | +| `.env`, `.env.*` | The values themselves | `grep -c KEY .env`, or test presence with `[ -n "$VAR" ]` | +| Rendered files that interpolate a secret (a login page with the demo password filled in) | Tracked by accident, it puts the value in HEAD | Track only the `*.template.*` file; git-ignore the rendered one | + +Three habits that make it hold: + +1. **Read a secret by testing it, not showing it.** `[ "${#VAR}" -ge 12 ]`, `[ "$A" = "$B" ] && echo same`, an HTTP status from `curl -u "$USER:$PASS"` with output discarded. Never `echo`, never `set -x`. +2. **Put the ban in the lane rules** of any multi-agent build: one line naming these files. An agent that was not told will `cat` them to be helpful. +3. **Scan what git will store.** `bin/check-staged-secrets.sh` (staged blobs) and `--tree` (everything tracked at HEAD) compare your real `.env`, `.docker/.env` and `bootConfig` values against the blobs and print only the file and the key name. It skips values under 8 characters, well-known defaults such as `mendix`, a value equal to the same key in `.env.example`, and a `bootConfig.MicroflowConstants` value that is already in the committed model (`run --local` writes every constant's model default there; a credential in a model default is `bin/constants-audit.sh`'s job), so a weak demo password like `mendix` is NOT protected by it (the default sits in thousands of tracked files, and a check that fails every commit gets turned off). It is not wired into the generated pre-commit hook; add it yourself. `git commit -- ` re-adds an ignored tracked file at commit time, so a `.gitignore` entry is not a guard. + +A value that reached a transcript or a remote is spent: **rotate it.** Deleting the commit or the +transcript does not un-leak it. The demo password on a free-node sandbox is the lowest stakes; a +shared database or API credential is not. + +--- + **`describe constant` and `show constants` print the stored default verbatim (mxcli v0.26.0 and v0.27.0, no masking).** Never run them on a project whose constants may hold a credential: the value lands in the transcript. Keep secret constants empty in the model and pass them with `--constant Module.Name=value`. ## Step 4 — Write the decision down where the next session will read it diff --git a/tests/wave2/test-check-staged-secrets.sh b/tests/wave2/test-check-staged-secrets.sh new file mode 100644 index 00000000..b7f0d6ba --- /dev/null +++ b/tests/wave2/test-check-staged-secrets.sh @@ -0,0 +1,122 @@ +#!/usr/bin/env bash +# test-check-staged-secrets.sh — project-bin/check-staged-secrets.sh finds a secret VALUE in staged +# blobs and in the tracked tree, names the file and the key, and never prints the value. +# +# Usage: bash tests/wave2/test-check-staged-secrets.sh +# The .env and run-local.json here are written by this test with made-up values; no field capture +# is claimed. Values are arbitrary strings, so the test proves the mechanism, not a real leak. +set -uo pipefail +TOOLKIT="$(cd "$(dirname "${BASH_SOURCE[0]}")/../.." && pwd)" +SUBJECT="$TOOLKIT/project-bin/check-staged-secrets.sh" +W="$(mktemp -d "${TMPDIR:-/tmp}/mxtk-css.XXXXXX")"; trap 'rm -rf "$W"' EXIT +pass=0; fail=0 +ok() { pass=$((pass+1)); echo " ok $1"; } +bad() { fail=$((fail+1)); echo " FAIL $1"; } +has() { case "$2" in *"$3"*) ok "$1" ;; *) bad "$1 (missing '$3')"; echo "$2" | sed 's/^/ /' ;; esac; } +hasnt() { case "$2" in *"$3"*) bad "$1 (leaked '$3')" ;; *) ok "$1" ;; esac; } + +P="$W/proj"; mkdir -p "$P/.mxcli" "$P/bin"; cd "$P" || exit 1 +git init -q . && git config user.email t@t && git config user.name t +cp "$SUBJECT" bin/check-staged-secrets.sh +printf 'DEMO_PASSWORD=Zq7-xylophone-91\nAPP_NAME=shortname\nDB=x\n' > .env +printf '.env\n.mxcli/\n' > .gitignore +cat > .mxcli/run-local.json <<'J' +{"pid": 1, "bootConfig": {"DatabasePassword": "pgpass-Tango-55", "DatabaseName": "appdb", "Cards.HubApiPassword": "hubpass-Delta-77"}} +J +git add .gitignore && git commit -q -m init + +echo "-- a clean staged file: clean" +printf 'hello\n' > hello.txt; git add hello.txt +OUT="$(bash bin/check-staged-secrets.sh 2>&1)"; RC=$? +[ "$RC" = 0 ] && ok "clean exits 0" || bad "clean exits 0 (got $RC)" +has "reports how many values it checked" "$OUT" "3 value(s) checked" + +git reset -q hello.txt; rm hello.txt +echo "-- a staged file holding a .env value" +printf '\n' > login.html; git add login.html +OUT="$(bash bin/check-staged-secrets.sh 2>&1)"; RC=$? +[ "$RC" = 1 ] && ok "exits 1" || bad "exits 1 (got $RC)" +has "names the file" "$OUT" "login.html" +has "names the key" "$OUT" ".env:DEMO_PASSWORD" +hasnt "never prints the value" "$OUT" "Zq7-xylophone-91" + +echo "-- a staged file holding a run-local.json bootConfig value" +git reset -q login.html; rm login.html +printf 'x=pgpass-Tango-55\n' > notes.txt; git add notes.txt +OUT="$(bash bin/check-staged-secrets.sh 2>&1)"; RC=$? +[ "$RC" = 1 ] && ok "bootConfig value found" || bad "bootConfig value found (got $RC)" +has "names the bootConfig key" "$OUT" "DatabasePassword" +hasnt "never prints the value" "$OUT" "pgpass-Tango-55" + +echo "-- a short or non-secret value is not a source" +git reset -q notes.txt; printf 'shortname appdb\n' > notes.txt; git add notes.txt +OUT="$(bash bin/check-staged-secrets.sh 2>&1)"; RC=$? +[ "$RC" = 0 ] && ok "non-secret keys are not scanned for" || bad "non-secret keys are not scanned for (got $RC: $OUT)" + +echo "-- --tree sees what is already committed" +printf 'hubpass-Delta-77\n' > committed.txt; git add committed.txt; git commit -q -m leak +git reset -q; OUT="$(bash bin/check-staged-secrets.sh --tree 2>&1)"; RC=$? +[ "$RC" = 1 ] && ok "--tree exits 1" || bad "--tree exits 1 (got $RC)" +has "--tree names the file" "$OUT" "committed.txt" +hasnt "--tree never prints the value" "$OUT" "hubpass-Delta-77" + +echo "-- no sources: says so, exits 0" +rm -f .env .mxcli/run-local.json +OUT="$(bash bin/check-staged-secrets.sh 2>&1)"; RC=$? +[ "$RC" = 0 ] && ok "no sources exits 0" || bad "no sources exits 0 (got $RC)" +has "no sources is stated, not silent" "$OUT" "nothing checked" + +echo "-- the default database password is not a secret (review: mass false positives)" +mkdir -p .mxcli; printf '{"bootConfig": {"DatabasePassword": "mendix"}}' > .mxcli/run-local.json +printf 'DB_PASSWORD=mendix\nOTHER_PASSWORD=Password\n' > .env +printf 'mendix Mendix password\n' > notes.txt; git add notes.txt +OUT="$(bash bin/check-staged-secrets.sh 2>&1)"; RC=$? +[ "$RC" = 0 ] && ok "staged 'mendix' with default bootConfig/.env password exits 0" || bad "default password exits 0 (got $RC: $OUT)" +has "says it skipped defaults" "$OUT" "nothing checked" +git reset -q notes.txt + +echo "-- .docker/.env is a source; CRLF, spaces and inline comment are normalised" +rm -f .env .mxcli/run-local.json; mkdir -p .docker +printf '%s = %s # local only\r\n%s=%s\r\n%s="%s"\r\n' M2EE_ADMIN_PASS "Kx9-admin-Quartz-31" DB_PASSWORD mendix TOKEN "Rv4-token-Magnolia-08" > .docker/.env +printf 'a Kx9-admin-Quartz-31 b\n' > one.txt; printf 'Rv4-token-Magnolia-08\n' > two.txt; git add one.txt two.txt +OUT="$(bash bin/check-staged-secrets.sh 2>&1)"; RC=$? +[ "$RC" = 1 ] && ok ".docker/.env value found" || bad ".docker/.env value found (got $RC)" +has "names the .docker/.env key" "$OUT" ".docker/.env:M2EE_ADMIN_PASS" +has "CRLF quoted value matches" "$OUT" "two.txt holds the value of .docker/.env:TOKEN" +hasnt "never prints the value" "$OUT" "Kx9-admin-Quartz-31" +hasnt "never prints a CRLF value" "$OUT" "Rv4-token-Magnolia-08" +git reset -q one.txt two.txt + +echo "-- a value equal to .env.example for the same key is published, not a secret" +printf 'API_TOKEN=Published-Example-99\n' > .docker/.env.example +printf 'API_TOKEN=Published-Example-99\n' > .docker/.env +printf 'Published-Example-99\n' > pub.txt; git add pub.txt +OUT="$(bash bin/check-staged-secrets.sh 2>&1)"; RC=$? +[ "$RC" = 0 ] && ok "example-equal value ignored" || bad "example-equal value ignored (got $RC: $OUT)" +git reset -q pub.txt + +echo "-- run --local writes every model default into MicroflowConstants: a default already in the model is not a source" +rm -rf .docker; rm -f .env; mkdir -p mprcontents/ab +printf 'default: your-api-key-here\n' > mprcontents/ab/unit1.mxunit; git add mprcontents; git commit -q -m model +printf '{"bootConfig": {"MicroflowConstants": {"MyModule.ApiKey": "your-api-key-here", "MyModule.AuthToken": "passed-Cedar-4821"}}}' > .mxcli/run-local.json +printf 'your-api-key-here\n' > unit2.mxunit; git add unit2.mxunit +OUT="$(bash bin/check-staged-secrets.sh 2>&1)"; RC=$? +[ "$RC" = 0 ] && ok "placeholder model default staged: exit 0" || bad "placeholder default staged (got $RC: $OUT)" +git reset -q unit2.mxunit +OUT="$(bash bin/check-staged-secrets.sh --tree 2>&1)"; RC=$? +[ "$RC" = 0 ] && ok "placeholder model default in --tree: exit 0" || bad "placeholder default --tree (got $RC: $OUT)" +printf 'x=passed-Cedar-4821\n' > leak.txt; git add leak.txt +OUT="$(bash bin/check-staged-secrets.sh 2>&1)"; RC=$? +[ "$RC" = 1 ] && ok "constant value NOT in the model is still found" || bad "constant not in model (got $RC)" +has "names the constant" "$OUT" "MicroflowConstants.MyModule.AuthToken" +hasnt "never prints the value" "$OUT" "passed-Cedar-4821" +git reset -q leak.txt; rm -f leak.txt + +echo "-- MXTK_SECRET_VALUES_FILE is not filtered" +printf 'mendix\n' > "$W/extra.txt"; printf 'mendix\n' > tracked.txt; git add tracked.txt; git commit -q -m t; git reset -q +OUT="$(MXTK_SECRET_VALUES_FILE="$W/extra.txt" bash bin/check-staged-secrets.sh --tree 2>&1)"; RC=$? +[ "$RC" = 1 ] && ok "explicit extra value 'mendix' is found (rc 1)" || bad "extra value (rc $RC)" +OUT="$(bash bin/check-staged-secrets.sh --tree 2>&1)"; RC=$? +[ "$RC" = 0 ] && ok "without the extra file the default is ignored in --tree" || bad "tree default (rc $RC: $OUT)" + +echo; echo "$pass passed, $fail failed"; [ "$fail" = 0 ]