Skip to content

Guard against a leftover mxcli test injection (#244), app-restart pid check and test-scratch takeover (#304 review), lint-a-copy recipe (#240) - #322

Closed
MendixMau wants to merge 5 commits into
masterfrom
fix/mxtest-endpoint-guard
Closed

MendixMau wants to merge 5 commits into
masterfrom
fix/mxtest-endpoint-guard

Conversation

@MendixMau

@MendixMau MendixMau commented Oct 13, 2026 •

Copy link
Copy Markdown
Owner

#244: AfterStartupMicroflow left on MxTest.RegisterEndpoint (BUG-157)

Problem. mxcli test and run --test-endpoint point after-startup at a generated MxTest.RegisterEndpoint (plain mxcli test: MxTest.TestRunner), which chains to the original. Only a clean exit restores the original. A run stopped by a signal leaves the setting on disk. The next injection then chains RegisterEndpoint to itself: StackOverflowError, and the runtime never starts. If the MxTest module was removed in the meantime, the build fails with CE1613 instead. This happened twice in one day on one machine, one app went down, and it happened again on a second project (AlvinnYeo, inbox note in #309, used as evidence only; #309 is not modified). Reading mxcli's v0.26.0 source (cmd/mxcli/testrunner) shows no self-wrap guard and no signal restore.

Fix.

  • project-bin/test-scratch.sh and project-bin/app-restart.sh read the setting before starting.
    • The toolkit had no existing way to read settings. This reads the BSON string element AfterStartupMicroflow straight from the settings unit's bytes with od + awk. It needs no mxcli and no running app, and it works on both mprcontents and v1 .mpr.
    • When the value is MxTest.RegisterEndpoint or MxTest.TestRunner, both scripts refuse. The message prints the value HEAD has, plus three repairs: git checkout -- <unit>, alter settings runtime (AfterStartupMicroflow: '<HEAD value>') through exec.sh (the statement mxcli's own cleanup runs, field-proven on v0.25.0), or Studio Pro.
  • test-scratch.sh: exit 3 before copying anything.
  • app-restart.sh:
    • Exit 3 before stopping anything, except when the recorded app is a live --test-endpoint run. That app holds the injection legitimately.
    • After the stop it checks again. If the stop left the injection behind, it exits 5 (new: stopped, not started).
  • Bug-log entry BUG-157. The upstream issue is drafted, not posted.

Evidence / golden input. A real settings unit (4e70063d-….mxunit, the same unit id as in the issue), captured from mxcli's own test model (mendixlabs/mxcli testdata pedapp, Apache-2.0, 12ce281). It is committed as tests/wave2/fixtures/mxtest-guard/settings-unit.mxunit.

  • Each case re-encodes it with a small BSON re-encoder (set-after-startup.js), rewriting every enclosing length. The fixture checks that re-encoding with the original value reproduces the capture byte for byte.
  • The reader returns '' on all three real mprcontents captures and on a real v1 App.mpr.
  • A capture of a real model with the injection was not possible: there is no mxcli binary or Mendix runtime in this environment.

Risks.

  • A clean model sees no change.
  • Exit 5 is new. No script consumes app-restart's exit codes; ui-loop.md and agent-roles.md (prose) do not mention codes.
  • On a v1 .mpr, the scan reads the file until the first match. That costs seconds on a large model, and a settings blob split across SQLite overflow pages would be missed (no false refusal, just no guard).
  • The helper is duplicated in both scripts to stay inside this PR's files. Moving it into _common.sh would be a later cleanup.

#304 review follow-ups

  1. app-restart killed an unrelated process via a reused pid. With --cmd or MXTK_APP_START_CMD, the run --local identity check read the override string, not the process. A record left behind by a SIGKILL, plus a reused pid, meant TERM/KILL went to that process and its children. Now the pid itself must be an mxcli run: its /proc/<pid>/cmdline (else ps -o args=) must name mxcli and run, or on Windows its image name must be mxcli.exe. Otherwise it is treated as not running and never stopped.
  2. test-scratch take_lock takeover race. test-scratch shares .mpr-snapshots/.exec.lock with exec.sh, so its takeover now uses exactly exec.sh's protocol from fix(exec): follow-ups from independent review of #304/#306/#290 #324 (fix/exec-followups @ 79de404, exec_lock_takeover; the bodies differ only in signature and parameter names). That means the same O_EXCL <lock>.takeover file, which counts as stale when its pid is dead or it is older than 10 s (an empty one after a minute). Under it the lock is re-read, and only the exact stale line is removed; a fresh lock is never moved aside. A wrongly moved takeover file is put back with ln. The machine-wide port lock uses the same protocol.
    • New cross-script fixture, tests/wave2/test-shared-lock-takeover.sh: 4 rounds of 5 exec.sh + 5 test-scratch contenders on one dead-holder lock. It takes exec.sh's lock block from the real exec.sh once that file carries exec_lock_takeover; until fix(exec): follow-ups from independent review of #304/#306/#290 #324 merges it uses a verbatim copy (fixtures/mxtest-guard/exec-lock-block.sh, diffed identical to 79de404, which carries the aligned -mmin +1 and own-file-only removal).

#240: lint on a copy silently drops the project rules (BUG-158)

  • skills/lint-that-actually-runs.md (on demand) gets a new section, "Linting a copy of the model: copy the rules in first". It includes:
    • the field numbers (51/8 against 721/29);
    • the recipe: copy .claude/lint-rules and lint-config.yaml beside the copy;
    • a completion check: compare distinct ruleId counts before any diff.
  • From reading the v0.26.0 source:
    • lint-config.yaml is read only from beside the .mpr, so even an in-repo snapshot loses it.
    • With no .git above, the rules walk continues to /.
  • Bug-log entry BUG-158. The upstream issue is drafted, not posted.
  • lint-gate.sh is not touched; it is owned by another PR.

Checks run

Field-run status

Still owed. Both guards and the lint recipe are proven against captured settings units and fixtures, not on a live project with a real interrupted --test-endpoint run. The probe to run is in BUG-157: kill -TERM a run --test-endpoint, then ./bin/app-restart.sh must exit 3 or 5 with the repair. Two-tree (app/) layout: the code resolves the model dir through the existing MODEL_DIR logic, but the fixtures cover the single-tree layout only.

Refs #244 #240

🤖 Generated with Claude Code

https://claude.ai/code/session_01VJgWP5vEoAsNsJYqCDMGNw


Generated by Claude Code


Generated by Claude Code

…id; serialise test-scratch takeovers; lint-a-copy recipe

#244 (BUG-157): test-scratch.sh and app-restart.sh read AfterStartupMicroflow from the
settings unit's bytes (no mxcli) and refuse to start while it is MxTest.RegisterEndpoint
or MxTest.TestRunner, printing the git / alter-settings / Studio Pro repair with HEAD's
value. app-restart lets a live --test-endpoint app through and re-checks after the stop
(new exit 5). Golden: a settings unit captured from mxcli's own test model, re-encoded
byte-faithfully per case. Second field case credited to AlvinnYeo (PR #309 inbox note).

Review of #304: app-restart verifies the recorded pid is itself an mxcli run before
stopping (a reused pid with --cmd was killed); test-scratch's stale-lock takeover is
serialised by an O_EXCL .takeover marker and re-read, restored with ln, not mv -f.

#240 (BUG-158): skills/lint-that-actually-runs.md gains the copy-the-rules-first recipe
and the rule-count comparison before any diff.

Refs #244 #240

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VJgWP5vEoAsNsJYqCDMGNw
…tocol (#324)

test-scratch.sh shares .mpr-snapshots/.exec.lock with exec.sh, so its stale-lock
takeover now mirrors exec_lock_takeover from PR #324 (fix/exec-followups @ ed4efbb):
the same O_EXCL <lock>.takeover file, stale when its pid is dead or after 10 s (empty:
after a minute), re-read and remove only the exact stale line, put a wrongly moved
takeover file back with ln. The port lock uses the same protocol.

New cross-script fixture tests/wave2/test-shared-lock-takeover.sh: 4 rounds of 5 exec.sh
+ 5 test-scratch contenders on one dead-holder lock never produce two holders (master's
test-scratch: overlaps in 3 of 3 runs). exec.sh's block comes from the real exec.sh once
it carries exec_lock_takeover, else from a verbatim copy of #324's block.

Refs #244

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VJgWP5vEoAsNsJYqCDMGNw
…s, MxTest. prefilter, override

Independent review of #322 at 333d435:
1. The reader ran under the callers' pipefail: awk exits at the first match, od dies of
   SIGPIPE, and the 141 discarded the answer — on a large v1 .mpr the guard did nothing.
   It is now a subshell function with pipefail off. Fixture: 2 MB v1 model after the setting.
2. `grep -rla … | head -1` took the first unit mentioning the name; a doc string elsewhere
   switched the guard off. Every candidate unit is read; only the BSON element counts.
   Fixture: doc-string units beside the injected settings unit.
3. Units without the bytes "MxTest." are not scanned (an 8 MB v1 .mpr took 8 s).
4. MXTK_SKIP_MXTEST_GUARD=1 skips the check, printed loudly (both fixtures).
5. test-app-restart.sh alive() is zombie-aware (3/10 flakes; now 5/5 clean).
6. in_our_tree's comment is back above it; set-after-startup.js cites the real fixture.
7. test-scratch's remedy for a live --test-endpoint app: test a copy whose settings unit
   comes from HEAD (#244's workaround) before stopping the live app.

Refs #244

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VJgWP5vEoAsNsJYqCDMGNw
…ith #324's review)

`find -mmin +0` treats a fresh empty takeover file as stale at once (GNU matches 0-2 s
old files, BSD rounds up), so a takeover in progress could be broken into: two holders
under load. Now -mmin +1 (empty for over a minute). The takeover file is removed only
when its content is what was judged stale (a moved-aside live one is put back with ln,
and kept if that fails), and a finished takeover removes the file only while it still
holds its own line. Fixture: a fresh empty takeover file blocks the takeover and is left
alone; one over a minute old (touch -t 2000) is stale. Fails on the previous body.

Keep in step with exec.sh's exec_lock_takeover (PR #324).

Refs #244

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VJgWP5vEoAsNsJYqCDMGNw
MendixMau pushed a commit that referenced this pull request Oct 13, 2026
…ith test-scratch

Review of #324: `find -mmin +0` matched a takeover file created a second
ago, so a live contender's fresh empty file was moved aside and a third
contender got in (reviewer: 2-3 holders in 6/15 and 16/30 rounds with a
100 ms delay). Now -mmin +1; a moved-aside copy is deleted only if it is
what was judged stale, else restored with ln (left on disk if ln fails);
the holder removes the takeover file only while it holds its own line.
Body identical to test-scratch.sh take_over() in #322. Reviewer's delayed
harness against this body: 0/15 rounds with >1 holder.

Also: microflow-body verbs, CREATE FOLDER and MOVE ... TO FOLDER are
neutral for the watch verdict; agent-roles.md names the unknown verdict.

Refs #287 #289

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VJgWP5vEoAsNsJYqCDMGNw
The aligned exec_lock_takeover (-mmin +1, own-file-only removal) now matches
test-scratch take_over() apart from signature and parameter names (diffed).

Refs #244

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VJgWP5vEoAsNsJYqCDMGNw
MendixMau pushed a commit that referenced this pull request Oct 13, 2026
…MxTest after-startup; mxcli-run pid check (closes #244)

Conflict: tests/wave2/test-app-restart.sh header comment — kept both
blocks (#319's first). project-bin/app-restart.sh auto-merged: #319's
one-line in_our_tree via mxtk_pid_in_tree kept, #322's mxtest guard,
pid_is_mxcli_run, step 2a and exit 5 present.

Review: independent joint review (agent a1e0b45) of head 1cb70ad — MERGE.
Follow-ups: move duplicated mxtest_* helpers and pid_is_mxcli_run into
_common.sh; document exit 4/5 in dispatching.md; one ls|grep -q under
pipefail in test-test-scratch.sh. Fixtures: app-restart 64/0,
test-scratch 46/0.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VJgWP5vEoAsNsJYqCDMGNw
MendixMau added a commit that referenced this pull request Oct 14, 2026
)

Every constituent PR was reviewed by a fresh agent that did not write it (CLAUDE.md "Merging a PR"); only MERGE verdicts joined. CI green on each constituent head and on this head (40814fc); full suites run on the branch.

Review lines (per merged PR):
- #315 context trim — MERGE; 360/385 removed lines found verbatim in their new homes, rest read by hand; baseline 79,997 -> 76,259.
- #316 independent-review + word-budget rules — MERGE-AFTER-FIX (3 rounds), fixes applied as specified.
- #318 page-fidelity displayed text — MERGE after fix round + confirmation.
- #317 interaction-model skill (#291 part 1) — MERGE after fix round + confirmation; no client names leaked.
- #321 PROOF-OF-LOOK per section; final-look not due before derived Stage 6 — MERGE after fix round + confirmation.
- #307 graph-sweep guard; per-BRD conformance ledgers — MERGE after two fix rounds + confirmation.
- #325 extraction-quality section; token-burn fixes — MERGE after fix round + confirmation.
- #310 e2e OQL by alias, negative-span liveness, APP-DOWN, #302 credential regression fix — MERGE after fix round + 3 confirmations.
- #323 inbox promotions — MERGE after fix round + confirmation.
- #308 lint gate BLIND on reverted stock rules — MERGE (FIRST-SEEN reverted) after two fix rounds + confirmation.
- #328 pipefail + grep -q flake — MERGE; 0/30 flakes vs 15/100 on master. Follow-up: run-all resolves bin/lib/ subjects (reviewed, MERGE).
- #319 test-stack-up command-line ownership proof — MERGE after fix round + 2 confirmations.
- #327 change-intake core + on-demand detail; CRLF-safe change log — MERGE after fix round + confirmation.
- #320 follow-ups to #290/#292; Dispatching split out — MERGE after fix round + confirmation.
- #326 status delivery rounds (existing-app mode) — MERGE after fix round + confirmation.
- #312 lint batch 4: SEC014, toolkit CONV009 — MERGE after review + 2 fix rounds + confirmation; field run on mxcli 0.26.
- #324 exec.sh change classes, shared stale-lock takeover, SEC014 note echo — MERGE (joint review); follow-up owed for a takeover race after a mid-takeover crash.
- #322 MxTest after-startup guard, mxcli-run pid check — MERGE (joint review).
- #314 doctor property-set exit code; sync announces its bound — MERGE (joint review).
- #330 review follow-ups (exit 4/5 docs, pipefail, NUL strip) — MERGE-AFTER-FIX, fixes applied; takeover re-check dropped.

Field runs are still owed for most instruments in this batch (stated in each PR).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VJgWP5vEoAsNsJYqCDMGNw
@MendixMau

Copy link
Copy Markdown
Owner Author

Landed on master as part of the integration squash #329 (commit 31e90ae), after independent review — its Review: line is in that commit message. Closing as superseded.


Generated by Claude Code

@MendixMau MendixMau closed this Oct 14, 2026
MendixMau pushed a commit that referenced this pull request Oct 14, 2026
…dence pointer)

- v1.0.md: 1.0 is the first published train tag; note on PRs closed via #329;
  sync prints "none tagged yet" until tags exist; app-restart.sh is #304,
  #322 added the injection refusal; "UX001/002 plus Batches 1-4"; known gap
  for the stale-lock takeover race after a killed holder.
- RELEASE-PROCESS.md §8: tags start at 1.0; placeholders instead of a concrete
  tag; session ritual warns while pinned (expected); cadence defers to the
  CHANGELOG header instead of restating it.
- README.md: Toolkit release line before/after tags; tags start at 1.0;
  interview-mode --set persists in .claude/.interview-mode, env var is
  per-session; "alias: unattended" next to auto.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VJgWP5vEoAsNsJYqCDMGNw
@MendixMau
MendixMau deleted the fix/mxtest-endpoint-guard branch October 15, 2026 10:46
MendixMau added a commit that referenced this pull request Oct 16, 2026
…rom #309 by @AlvinnYeo) (#450)

Lands #309 by @AlvinnYeo. Independent review: APPROVE.

# PR 450 (lands #309 by @AlvinnYeo) review r1 (delta self-review)
Verdict: APPROVE
- w1 must-fix 1 done: every #244 status now says closed 2026-10-14 by #322, upstream root cause remains (gh issue view 244 confirms). No STOP-table wording left in the note.
- w1 must-fix 2 done: body drops the stale byte-identical claim.
- Inbox lane only (contrib/inbox + one CHANGELOG line, credit " — AlvinnYeo"); nothing loaded or routed; zero behaviour risk.
- Optional word limits not enforced (inbox notes, maintainer call).
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants