Skip to content

learn(load-test): record the real page, replay it, say what the number does not prove - #431

Merged
MendixMau merged 4 commits into
masterfrom
feat/load-test-unit-test-skills
Oct 16, 2026
Merged

MendixMau merged 4 commits into
masterfrom
feat/load-test-unit-test-skills

Conversation

@MendixMau

@MendixMau MendixMau commented Oct 16, 2026 •

Copy link
Copy Markdown
Owner

Direct lane

What changed and why (one paragraph):
New on-demand skill skills/load-test.md for "will this page hold N users". Mendix pages call the undocumented /xas/ client protocol (Mendix 11: every microflow is runtimeOperation with an opaque id), so hand-written or HAR-replayed protocol scripts need dynamic ids and the CSRF token correlated again after every release. The skill's recipe skips correlation: Playwright logs a real persona in and records the page's /xas/ calls, a probe fires each once and reads Jaeger to map it to a microflow (and refuse writes), and the chosen call is replayed with that session's own cookie and mx.session.getCSRFToken(). It carries eleven traps (incl. the 6-session cap of an unlicensed local runtime) each hit in the field runs and a four-line report that states what a run does not prove (requests vs users, empty data, harness vs app limit). Discoverable by asking for testing skills: routing row plus one row in testing-shape.md's owner table. Not wired into the build plan, not a gate. The roadmap's "no load testing" line is reworded: the method ships, a generic instrument waits for a second project.

Impact

Row Answer
Process none: on request only; no gate, no stage artifact, nothing blocks
Dependencies bin/lib/skill-routing.tsv + re-rendered surfaces (README, ROUTING, test/review agents, gate-check stage map); testing-shape.md owner table, one row (#417 touches other sections of that file); no new tools; no instrument shipped
Context cost baseline 75,204 → 75,223 words (budget 80,000), render-routing.sh --check exit 0 on both. Stage 5 test 21,639 → 21,658, review 31,367 → 31,386; stage 6 test 21,414 → 21,433. The skill itself is ondemand
Behaviour NOT RUN: no A/B; new ondemand text, no existing skill rewritten
Runtime n/a

Field evidence: a warehouse demo app, Mendix 11 with mxcli run --local + OTel to Jaeger, 16 recorded runs 2026-09-09 to 2026-10-08. Bursts of 1000 on one session: p95 0.2 to 4.6 s, 0 transport and 0 app errors; 2000: p95 1.2 to 11.5 s. Every trap in the skill is a measured incident from those runs (the 771-byte no-op target, the stale tape with six dashboard microflows, the fail-open tracing check with serverSide: null, the one-account two-session nav loss).

  • No client data anywhere in the diff (leak guard with denylist: pass, no new capitalised words)
  • Size cap: 10 files, 156 insertions / 4 deletions
  • Test tier: T0, render-routing.sh --check exit 0, leak guard and doc-numbering pre-commit pass. Skill text not field-run as a skill
  • For a new skill: routing row added
  • CHANGELOG.md line appended

🤖 Generated with Claude Code

…r does not prove

New on-demand skill from a field implementation (warehouse demo app, Mendix 11,
16 runs, bursts of 1000 and 2000 concurrent calls). Playwright records the
page's /xas/ calls, a Jaeger probe maps each runtimeOperation to its microflow,
and the chosen call is replayed with the session's own cookie and CSRF token,
so nothing is correlated by hand. Ten field traps and a four-line report.
Routing row, one owner-table row in testing-shape.md, roadmap line reworded.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…Data.csrftoken, teardown via helpers.logout

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
MendixMau added a commit that referenced this pull request Oct 16, 2026
…432)

Independent review: CHANGES in round 1 (432.md, Opus), all three items fixed; round 2 self-review APPROVE (432-r2).

# Review #432 round 2 (self-review of fix delta to 69ff044)
Round 1 (432.md, Opus) asked 3 changes, all addressed: default-no pointer now names PROJECT.md `## Test strategy` / `Unit modules:` from CAC-5 and states #417's run moments; reasons reduced to #417's three (calculation or mapping, Java or REST, fixed bug), the "branchy rule" reason removed; `--require-assertions` added for mxcli 0.21+. Fixer ran render-routing --check, check-skill-routing, test-render-routing 5/0, test-routing-tier 26/0, portability clean. Merge before #431.

APPROVE
MendixMau and others added 2 commits October 16, 2026 21:48
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@MendixMau
MendixMau merged commit 5d09fdb into master Oct 16, 2026
2 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant