Skip to content
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 3 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -17,6 +17,9 @@ Sections dated before 2026-09-19 predate the cycle and stay as they are.

## Unreleased

- fix(token-burn): **usage per `message.id` is each counter's max over its records (in practice the last), not the first record's — subagent output was undercounted; a late empty record cannot erase real numbers.** The header claimed every streamed record of a message carries the same usage; in subagent transcripts the early records carry a partial `output_tokens`. On the captured fixture the subagent model's output went 60 → 1,559 (headline 59,858 → 61,357); on a 2026-10 multi-agent session 37,791 → 84,915 subagent output tokens; a field measurement on harness 2.1.296 saw up to 7×. Main-thread records were identical first to last, so main numbers do not move. **Every existing subagent-heavy token count rises after sync**; fixture expectations re-derived independently (`tests/wave2/fixtures/token-burn/CAPTURE.md`) — found while promoting [#171](https://github.com/MendixMau/mxcli-project-toolkit/issues/171) (measured-claims §7), MendixMau
- new(extraction-report): **the Stage 1 surface now reads the validator's `extraction-quality*.json` — one row per dimension the file declares, a self-graded pass shown as a fault** ([#151](https://github.com/MendixMau/mxcli-project-toolkit/issues/151)). `extractor-quality-loop.md` makes that file the proof of the 95% gate, yet nothing read it: a field build scored its process-model extractor at 100% on 7 dimensions and the report showed "2 pass, 18 skipped" with no quality section. Found at `analysis/<src>/`, `analysis/<src>/extracted/` and the same two inside the KB; dimension keys come from the file. Faults only where the skill's rules show in the file: `selfGraded` + `passed`, `passed` under its own `threshold`, a 0/0 dimension absent from `confirmedZeros`, unparseable JSON. A code-extracted or mixed KB with no score file shows a `manual` section ("not produced yet — no shipped pipeline writes a validator"), not a fault: nothing mandatory produces the file yet; documents KBs get `skipped`; the gate (`gate-check.sh`, presence of the page) and the exit code are unchanged. A score file inside `<kb>/extracted/` is no longer misread as an unparseable item file. No captured validator output exists in this repo (no pipeline ships a validator), so the fixture reads the skill's documented example at run time; `tests/wave2/test-extraction-quality-report.sh` — MendixMau
- new(token-burn): **`--rates`, `--windows`, `--effort`: cost at the project's own per-Mtok rates, agent working time, human prompt count, per-window attribution — aggregates only** ([#296](https://github.com/MendixMau/mxcli-project-toolkit/issues/296)). A field team had to answer "money and time per round and per feature" with ad-hoc transcript scripts. No price is built in (rates file, model globs, first match wins; an unmatched model is UNPRICED and the total marked incomplete, never 0); agent time = main-thread gaps <= `--gap` minutes (default 5) not ending at a human prompt; human prompts = `origin.kind == "human"` user records (task notifications, peer messages, tool results, meta and compact summaries excluded), once per uuid, text never printed. Without the new flags the new code adds nothing to screen, `--json` or `--brief` output (byte-identical to the previous script before the last-record fix below; fixture T9 asserts no new section or key). Field run on a real multi-day transcript: human prompts matched an independent count (89 = 89). The human-prompt fixture records are hand-built from a profile of real record shapes — committing a scrubbed real capture was refused; `tests/wave2/test-token-burn.sh` T9-T14 — jopterhorst
- learn(testing-shape, test-result-audit): **three test-strength rules, each with a denominator: refusals probed on the server, every undeclared 4xx a failure, one clean-database final run** ([#172](https://github.com/MendixMau/mxcli-project-toolkit/issues/172)). A field comparison of two builds against one frozen spec found the toolkit build checked permission refusals only in the client (a hidden menu item), logged unexpected 400/422 responses as information, and reached 20/20 by stitching three runs. `testing-shape.md` §7 gains three report lines (`N of N restricted pairs probed server-side`, `undeclared 4xx/5xx: 0 of M requests — each fails its step`, `one clean-DB run <id>: N of N scenarios incl. timers, not stitched`); `test-result-audit.md` gains check 2e, which carries the reasons and the field evidence and asks for those numbers. Both baseline skills stay inside the 80,000-word baseline budget (79,997): the reasons live in the on-demand audit skill, and two redundant sentences in `testing-shape.md` §4's verdict paragraphs were folded (the empty-set corollary is stated in the Trace row, `harness-architecture.md` and `report-schema.md`). The backing inbox note is trimmed to its unpromoted "Keep" items. — MendixMau
- learn(learned-detection-gaps, module-review): **two UI defects every model gate passes: a lazy-loaded combo box filter hides its preset, and a theme that pins row heights by position lets the next row cover an inserted widget** ([#299](https://github.com/MendixMau/mxcli-project-toolkit/issues/299)). Both passed check, mxbuild, lint and the scripted checks on a 127-module production app and were found only in the running app. Two register rows with the first oracle and the pre-build check (pluggable properties that change *when* data loads; grep the theme for fixed `height` on positional row classes before an insert), and two LOOK lines in `module-review.md` §4c with denominators (`N of N filters show their preset on first load`, `N of N widgets inserted into an existing container fully visible`). — jopterhorst
- fix(app-restart, test-stack-up): **a port held by another process before `run --local` is named and reported as "instrument blocked" (exit 4), never killed** ([#262](https://github.com/MendixMau/mxcli-project-toolkit/issues/262)). Studio Pro had started its own runtime on the app/admin ports a restart needed; `mxcli run --local` then failed with "Authentication failed" (the admin API answering was Studio Pro's), the next journey ran against Studio Pro's runtime, and four more runs were NOT RUN. `app-restart.sh` now checks the app port and the admin port (record's `adminPort`, else app + 10) before stopping anything — a holder outside the recorded process tree exits 4 with its name, pid and parent — and again after the stop, where anything still holding a port is by definition not ours. `test-stack-up.sh` runs the same check wherever it would tell you to start `run --local` (`--no-docker`, no Docker daemon, `--check` with a leftover record), on the record's ports, `RUN_LOCAL_APP_PORT`/`RUN_LOCAL_ADMIN_PORT`, or mxcli's defaults 8080/8090. Holder lookup: lsof, then ss, then netstat + tasklist on Windows; unnamed holders still block when nothing of ours is running. Fixtures: `test-app-restart.sh` +2 cases (14 assertions, all fail on the old script), `test-stack-up-local.sh` T7 (4 of 6 fail on the old script). — AlvinnYeo
Expand Down
Loading
Loading