Skip to content

fix(exec): pause the project watcher during the gate and retry once on an infra exit - #451

Merged
MendixMau merged 6 commits into
release/v1.0from
fix/gate-watcher-race
Oct 17, 2026
Merged

MendixMau merged 6 commits into
release/v1.0from
fix/gate-watcher-race

Conversation

@MendixMau

@MendixMau MendixMau commented Oct 16, 2026 •

Copy link
Copy Markdown
Owner

Problem

The exec gate's mxbuild and a live mxcli run --local --watch on the same project both write deployment/. In a field migration project (12 modules), 17 of 27 failed gates were "gate could not run" (mxbuild exit 1 or 3, no CE code, no errors file), against 0 of 22 gates run in watcher-free copies of the same project. Each cost 2 to 4 minutes of rerun plus blocked guards.

Refs #441, #444, #171, #285, #424

Change

  • project-bin/_common.sh: mxtk_gate_mxbuild wraps mxtk_mxbuild_error_count. It stops this project's watcher (SIGSTOP, found by pgrep on run --local --watch, matched by project path or cwd) for the length of the gate and continues it (SIGCONT) after. mxbuild has no option to move the deploy target's output out of deployment/, so a separate output dir is not available; pausing was the simplest robust choice. Only the watcher process is stopped, not its runtime.
  • Retry: mxbuild exit 1 or 3 with no CE code in the output and an empty errors file is retried once, printed as "retry after infra exit" and added to the BUILD-LOG detail. An errors file with content (for example a version mismatch) is deterministic and is not retried.
  • project-bin/exec.sh: the three gate call sites (baseline, gate, whose-error rebuild) use the wrapper; the EXIT trap resumes the watcher; the fix(exec): roll back the model write when mxbuild crashes #424 rollback is untouched and applies to the final failure.
  • CHANGELOG entry.

Test

New tests/wave2/test-exec-gate-retry.sh (stub mxcli, mxbuild, java; no real mxbuild): A) infra exit then clean: retried, kept, row and output carry the note; B) infra exit twice: retried once, rolled back, model byte-identical; C) a real CE error list is not retried; D) a live stub watcher is in state T during both mxbuild runs and running afterwards. 14 of 14 pass. Existing test-exec-crash-rollback.sh (13) and test-exec-lock-watch.sh (84) pass. check-pr-discipline.sh and check-portability.sh (changed files) are clean. test-common-windows.sh has 2 ELF failures that are identical on unmodified master.

Risk

  • A watcher rebuild already in flight when the pause starts can still collide; the retry covers it.
  • If exec.sh is SIGKILLed during the gate, the watcher stays stopped until kill -CONT <pid>.
  • A deterministic infra failure with no errors file now costs one extra mxbuild before it is reported.
  • Touches the same BUILD-LOG detail line as the full-log PR (fix/gate-full-log); a one-line rebase conflict is possible.

🤖 Generated with Claude Code

… on an infra exit

The gate's mxbuild and a live `mxcli run --local --watch` both write deployment/.
In a field migration project (12 modules) 17 of 27 failed gates were "gate could
not run" (mxbuild exit 1, no CE code), against 0 of 22 in watcher-free copies.

The watcher of this project is stopped for the gate and resumed after (and in the
EXIT trap). An exit 1 or 3 with no CE code and no errors file is retried once and
logged as "retry after infra exit". A final failure still takes the #424 rollback.

Refs #441, #442, #171, #285, #424

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@MendixMau
MendixMau changed the base branch from master to release/v1.0 October 17, 2026 06:49
MendixMau and others added 5 commits October 17, 2026 14:53
…check

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…a killed exec.sh

The watcher match is now an equality test on the resolved project dir, so a sibling
copy or a nested worktree watcher is no longer paused. The paused pids are recorded in
.mpr-snapshots/.watcher-paused; the next exec.sh resumes them once it holds the lock.
CHANGELOG ref corrected to #444.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…re stopping

ps drops the quotes around -p, so the path was cut at the first space and a
duplicated folder named "<proj> copy" matched this project. Keep the longest
run of words that exists on disk instead. Write the pause record before the
SIGSTOP loop so a kill in between cannot leave an unrecorded stopped watcher.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@MendixMau
MendixMau merged commit dc840ee into release/v1.0 Oct 17, 2026
2 checks passed
MendixMau added a commit that referenced this pull request Oct 17, 2026
fix(exec): keep the gate's full mxbuild output in a per-run log. Independent review: APPROVE.

Verdict: APPROVE

r3 on head after merging the fixed #451 (f4cc5ae, fd90628) into fix/gate-full-log; the merge was clean. r2 APPROVE stands. gate-log 24/24 and gate-retry 24/24 on the merged head, check-pr-discipline clean. Merges after #451.
MendixMau added a commit that referenced this pull request Oct 17, 2026
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant