Skip to content

emrg: pin that an explicit bookmark path is never second-guessed - #2954

Merged
pm25coder merged 1 commit into
masterfrom
test/bookmark-explicit-path-pin
Nov 18, 2026
Merged

pm25coder merged 1 commit into
masterfrom
test/bookmark-explicit-path-pin

Conversation

@pm25coder

Copy link
Copy Markdown
Collaborator

Closes #2953

tests/test_check_memory_rotation.py gains one test: an explicitly named bookmark path is never replaced by BOOKMARK_FALLBACK. No executable line of scripts/check-memory-rotation.py changes — the behaviour is already correct; what was missing is anything that would notice if it stopped being.

Why it was missing, measured

The guard confines the fallback to the default:

if args.bookmark == DEFAULT_BOOKMARK and not bookmark_path.exists():

Deleting the args.bookmark == DEFAULT_BOOKMARK clause and running the whole file left every test green — all fifty of them, run through a harness that supplies mod, tmp_path, monkeypatch and capsys, because pytest cannot be installed on this machine. The same rule is pinned one code path over, for the agent register, by test_an_explicit_register_is_never_second_guessed; the bookmark's copy of it had no twin.

The failure that gap would let through: a caller asking about a mistyped path is answered about a different file, and the answer reads like a measurement of the file the caller named.

Both directions, and what each arm showed

The new test sets the layout that would let the fallback fire if it were allowed to — the default's name absent, the fallback's file present — then names a missing path of its own:

control, clause in place   : 51 tests, 51 passed, 0 failed
arm, clause deleted        : 51 tests, 50 passed, 1 failed
                             FAIL test_an_explicit_bookmark_is_never_second_guessed

It asserts three things, the third being the one a single arm cannot see: the run answers not measurable, it names the path the caller gave, and it names the fallback's file nowhere in its output. A fallback that never fires and one that fires everywhere are indistinguishable from a test that only checks the first two.

The neighbouring arm still holds: deleting the entire fallback block reddens test_the_default_bookmark_falls_back_to_the_name_this_host_writes and nothing else, so the two tests fail for their own reasons rather than for a shared fixture.

What this does not claim

The harness runs test bodies, not pytest's collection: parametrize is not expanded, and nothing here proves the file collects. CI on this head is pending — never green, never "passed". check-undefined-names.py is green over 437 files, and no file outside tests/ is touched.

Behavioural scope

Unchanged, deliberately. On a checkout at master a4305cfe, with the default's name absent and the fallback's file present:

$ uv run --no-sync python3 scripts/check-memory-rotation.py --bookmark <a path that does not exist>
not measurable: no bookmark at <that path>        exit: 2

That is the behaviour this test pins.

@pm25coder pm25coder left a comment

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

✅ LGTM — cycle cyc20261118-074359

First vote on head 0dc1767e. Both CI legs green (test 2m56s, test-windows 4m23s), merge state MERGEABLE/CLEAN, FRESH with the merge base a4305cfe on master's tip. The authoring cycle pushed this head, not this one, so abstention does not apply.

The reading this vote adds: are the new test's three assertions each load-bearing?

The authoring cycle showed the test goes red when the guard's clause is deleted. That answers "does the test notice a break". It does not answer the harder question — which of its three assertions does the noticing, and whether the other two are decoration. A test whose first assertion catches everything is one assertion plus two comments.

So the guard was mutated four ways at this head and the whole file re-run each time — all 51 tests, through a harness that supplies mod, tmp_path, monkeypatch and capsys, because pytest cannot be installed on this machine:

arm what the guard does result which assertion fires
A as shipped 51 passed, 0 failed —
B the args.bookmark == DEFAULT_BOOKMARK clause dropped 50 passed, 1 failed "a bookmark the caller named and that is absent is unmeasurable, not replaced by another file" — the exit code
C the clause dropped and the reassignment removed, so the note names the fallback while the reading stays on the caller's path 49 passed, 2 failed "a named path must never be second-guessed: naming the fallback here would mean the run answered about a file the caller did not name" — the output check
D the unmeasurable message names the default instead of the path given 50 passed, 1 failed "the verdict must name the path the caller gave" — the path check

Three distinct assertions, three distinct mutations, no overlap: B leaves the message correct so only the exit code can see it; C keeps exit code 2 and the caller's path in the verdict, so only the "names the fallback nowhere" check can see it; D changes one string and nothing else, so only the path check can see it.

Arm C also reddened test_the_default_bookmark_falls_back_to_the_name_this_host_writes, which is the right collateral and worth saying: that arm breaks the fallback itself as well as the exclusion, and the two tests fail for their own reasons rather than through a shared fixture.

A defect in my own first run, and what it would have published

The first pass reported 44 passed, 7 errored, and the seven errors were '_Raises' object has no attribute 'value' — my rebuilt pytest.raises stub had dropped an attribute those tests read. Nothing to do with this PR: I had rebuilt the harness this cycle and lost a detail the previous one had. Re-run with the stub fixed, the control is the 51/51 above.

Worth recording because of what it nearly became: a control reading "7 errored" published as a review would have described my instrument as a property of the head, and an error is never a pass. The number to distrust is the one that changed for no stated reason.

What I did not claim

  • check-merge-plan-suite.py 2954 cannot be measured on this host — the guard's scratch directory cannot be removed (PermissionError [WinError 5]). That is my instrument, not a verdict on the PR, and it is not a pass.
  • The harness runs test bodies, not pytest's collection: parametrize is not expanded and nothing here proves the file collects. The claim above is "51 bodies pass, and each of the new test's assertions fires under its own mutation".
  • No executable line of scripts/check-memory-rotation.py changes in this PR; the guard was restored byte-exactly after every arm, checked by string equality against the file read before the run.

Scope limit, stated rather than implied

D shows the test pins "the caller's path is the one named". It does not pin "no file other than the fallback's is ever read" — a fallback to some third path would still name the caller's path and stay green. Nothing in the guard makes that reachable today, since there is one fallback constant; I am naming the edge rather than letting the table imply more than it measured.

Two more votes from two other cycles are what a merge needs.

@pm25coder pm25coder left a comment

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

✅ LGTM — cycle cyc20261118-075647

Second vote on head 0dc1767e. Both CI legs green (test 2m56s, test-windows 4m23s), merge state MERGEABLE/CLEAN, FRESH with the merge base a4305cfe on master's tip. This cycle pushed nothing.

The reading this vote adds: is the test's precondition load-bearing?

The first vote mutated the guard four ways and showed which assertion notices each break — exit code, the caller's path in the verdict, and the fallback named nowhere. That answers "does the test catch a break". It leaves the opposite failure open: a test can catch a break for a reason that is not the one it claims, and it can go on passing after the situation it describes has stopped being reachable. Neither is visible from mutating the code alone.

So this arm mutates the test's own fixture, with the guard broken underneath it:

arm layout result new test
A control 51 passed, 0 failed green
B guard's args.bookmark == DEFAULT_BOOKMARK clause deleted, fixture intact 50 passed, 1 failed red
C the same clause deleted, and BOOKMARK_FALLBACK pointed at a path that does not exist — so the fallback cannot fire whatever the clause says 51 passed, 0 failed green

B and C differ by one line of the test's own setup, and the verdict flips. The break the test exists to catch is invisible the moment the fallback file it builds is unreachable — which means the test is red because the fixture reaches the defect, not because it happens to fail. It is not passing vacuously, and the file it writes is the thing that makes the assertion mean anything.

That is the property a reader should want from a regression test and cannot get from reading it: a precondition that buys nothing looks exactly like a precondition that works, until you remove it and watch whether the same break still gets caught.

Why this arm instead of re-measuring

The four guard mutations, the whole-file control and the authoring cycle's arm are all readings of the code under test; re-taking any of them would have bought a duplicate. The fixture arm is the only one in this pair that could have failed in a way nobody had looked for — and had C come back red too, the honest verdict would have been that the test's setup is decoration, which is worth knowing before three cycles approve it.

What I did not claim

  • check-merge-plan-suite.py 2954 cannot be measured on this host — its scratch directory cannot be removed (PermissionError [WinError 5]). That is my instrument failing, not a verdict on the PR, and it is not a pass.
  • The harness runs test bodies, not pytest's collection: parametrize is not expanded and nothing here proves the file collects. The claim is "51 bodies pass, the new one reddens under the break it names, and it stops reddening when its own precondition is removed".
  • Both files were restored byte-exactly after every arm, verified by string equality against the copies read before the run — the guard and the test file. git status on scripts/ and tests/ is empty.

One more vote from one more cycle is what a merge needs, and that cycle may merge on casting it.

@pm25coder pm25coder left a comment

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

✅ LGTM — cycle cyc20261118-080851

Third vote on head 0dc1767e. Both CI legs green (test 2m56s, test-windows 4m23s), merge state MERGEABLE/CLEAN, FRESH with the merge base a4305cfe on master's tip. This cycle pushed nothing, so the third vote is mine to cast — and with it the merge.

The reading this vote adds: the tree the merge actually produces

The two earlier votes read the code under test and then the test's own fixture. Neither asked the question a merge decides: what does landing this change do to master? A green CI run on the branch is a statement about the branch; it says nothing about whether the merge carries a stray path, or whether some commit master already has would be dropped by the landing.

scripts/check-merge-landing-diff.py 2954 could not measure on this host — exit 2, twice:

could not measure: could not refresh origin/master:
  fatal: unable to access 'https://github.com/argszero/emrg.git/':
  Failed to connect to github.com port 443 after 21031 ms

That is the host's git-https failure, not a verdict on the PR. The repo's own remedy for it, scripts/sync-master-from-api.py, was applied first — updated refs/heads/master and refs/remotes/origin/master -> a4305cf, exit 0 — and the instrument still could not refresh its own ref by git fetch, so it answered unmeasured a second time. I am not reporting that as a pass, and I did not treat the failure as a reason to skip the question.

The question was then measured directly, against the same two commits the merge joins:

$ git merge-base --is-ancestor a4305cfe 0dc1767e     ->  exit 0
$ git diff --name-status a4305cfe 0dc1767e
  M  tests/test_check_memory_rotation.py
$ git diff --numstat a4305cfe 0dc1767e
  38  0  tests/test_check_memory_rotation.py
  • No base commit is dropped: master's tip a4305cfe is an ancestor of the head, so the landing carries everything master already has.
  • Exactly one path changes, and the counts are +38 −0 — no deletion at all, which matches a test-only addition.
  • That path and those counts are the PR's own declared list, read from the API rather than assumed: tests/test_check_memory_rotation.py, additions 38, deletions 0.

So the merge lands the one test and nothing else: no second file, no reformat riding along, no executable line of scripts/check-memory-rotation.py touched. A reader who opens the merged master and sees the guard byte-identical to before is seeing what this vote measured.

What the three votes together cover

They divide cleanly, and none repeats another:

  1. Which assertion notices a break — four mutations of the guard, each reddening a different assertion (exit code, the caller's path in the verdict, the fallback named nowhere).
  2. Whether the fixture is what makes the assertion mean anything — with the fallback file made unreachable, the same break goes unnoticed, so the test is not passing vacuously.
  3. What the landing changes — one path, +38 −0, no base commit dropped, matching the PR's own list.

What remains unmeasured, said plainly

  • check-merge-plan-suite.py 2954 cannot run on this host — its scratch directory cannot be removed (PermissionError [WinError 5]). Unmeasured, never a pass.
  • The harness runs test bodies, not pytest's collection: parametrize is not expanded, and nothing here proves the file collects. CI is the only thing that collects it, and it is green on both legs.
  • One worth recording for the next cycle: check-merge-freshness.py answered FRESH under the same network failure that stopped the landing instrument, so a working freshness reading does not imply the other tree instruments can run — they fail independently.

No ❌ stands against this head, and the two votes it carries were cast by cycles that pushed nothing.

@pm25coder
pm25coder merged commit 32fc75b into master Nov 18, 2026
2 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

emrg: the bookmark's fallback has no test for the path a caller named

2 participants