Skip to content

tests: the suite is green on macOS, and the allocation law prices its marshals itself - #1722

Merged
AbirAbbas merged 11 commits into
devfrom
zeropoint95/test-suite-green-on-macos
Oct 2, 2026
Merged

AbirAbbas merged 11 commits into
devfrom
zeropoint95/test-suite-green-on-macos

Conversation

@ZeroPoint95

@ZeroPoint95 ZeroPoint95 commented Oct 1, 2026 •

Copy link
Copy Markdown
Contributor

Supersedes #1703 and carries its commit.

Three back-to-back make pr-ready BASE=origin/dev~5 runs on a clean dev (df7022d) on a Mac failed with the same ten tests every time, while PR gate and Full check on ubuntu were green for the same commit. Nothing flaked — the failure set was identical across all three runs. Every one was a Linux assumption baked into a test.

Four internal/session tests assumed /proc. They hold a task under a 1 TiB memory floor, and the governor reads /proc/loadavg and /proc/meminfo (task_pressure.go). A Mac has neither, the reading is unknown, and an unknown governor admits everything, so the hold never came: TestNewAdmissionReadsSettingsChangedBeforeItsCreation, TestHeldBeltRunStopsWithoutPreparingRepository, TestHeldRecoveryRetainsTheAcceptedCrew and TestHeldRunReopensOnTheSameAdmissionWithoutPreparingFiles. hostReading was the one place both files are read, so the governors are now built over a package-level readHost seam, and littleMemoryHost(t) states a quiet machine with 1 GiB available for the rest of a test and restores the seam on cleanup. The four tests now run and pass on every host, deterministically, rather than skipping on a Mac and holding only on Linux. requireHostMemoryReading (the /proc/meminfo skip two sibling tests already carried inline) stays, as a helper, for the one test whose claim is about this host's own reading.

Five tests compared a raw t.TempDir() with the engine's canonical path. macOS spells the temp dir through the /var → /private/var link; the engine records the resolved root. beltRepoWorkspace and TestDoOnTheRunEngineSignsFirstCommitOnUnbornBranch now resolve the folder with filepath.EvalSymlinks, as team_launch_test.go and update_test.go already do, which fixes TestDoOnTheRunEngineNeverCommitsThePersonsOwnWork, …SignsWorkerCommitsUnlessContributingForbids, …SignsFirstCommitOnUnbornBranch and …CompletesABriefAndNamesTheRootResult. #1703's fix for TestStandingIsolationRecordsItsCopyBeforeWorkerInitialization is the same shape and is included as its own commit. Once the hold was reachable on a Mac, the reopen test met the same mismatch on its recorded ground and compares against canonicalPath now.

The warm breakpoints encode law named a Go 1.26 number. TestTheWarmTranscriptEncodeCostsTheSameAtEightyTurnsAsAtEight demanded 8 allocations; on Go 1.27 the same code measures 11, because the two marked marshals went from 7 to 10 inside encoding/json with the memo unchanged (the slice plus the system marshal plus the tail marshal is exactly 11). The test now prices marshalMarked for the two placed positions in-process and demands the result slice and nothing more on top of them — the actual promise, toolchain-independent, and stricter than any constant. PERF.md is updated in the same change.

Proof. Each fixed test run alone with -count=3 on this Mac (go1.27.0 darwin/arm64) passes, including the four held-run tests, which no longer skip. Full make pr-ready BASE=origin/dev~6 results are in the comments.

🤖 Generated with Claude Code

ZeroPoint95 and others added 4 commits October 1, 2026 11:20
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…y root

TestStandingIsolationRecordsItsCopyBeforeWorkerInitialization compared the
recorded Root, which repositoryRoot canonicalizes, against the raw
t.TempDir(). On macOS those spell /private/var/folders and /var/folders for
the same directory, so a correct record failed. The test now compares
against canonicalPath(repo), as its siblings already do.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
… marshals itself

Three back-to-back `make pr-ready BASE=origin/dev~5` runs on a clean dev
(df7022d) failed with the same ten tests every time on a Mac, while CI on
ubuntu was green for the same commit. None of them flaked; each was a Linux
assumption in a test.

Five internal/session tests hold a task under a 1 TiB memory floor, and the
governor reads /proc/meminfo, which a Mac does not have — an unknown reading
admits everything, so the hold never came. They now go through
requireHostMemoryReading, the skip two sibling tests already carried inline.

Four cmd/codeaf engine tests compared the envelope's files against a raw
t.TempDir(), which macOS spells through the /var -> /private/var link while the
engine records the canonical root. beltRepoWorkspace and the unborn-branch test
resolve the folder, as team_launch_test and update_test already do.

The warm breakpoints encode law said 8 allocations; that figure belonged to
encoding/json on Go 1.26 and is 11 on Go 1.27 with the memo unchanged. The test
now prices the two marked marshals in-process and demands the result slice and
nothing more on top of them, which is the actual promise. PERF.md says so.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
@ZeroPoint95

Copy link
Copy Markdown
Contributor Author

make pr-ready BASE=origin/dev~6 on 57344ce, this Mac (go1.27.0 darwin/arm64): exit 0, zero failing tests, 517 s wall. The touched set was cmd/codeaf, internal/config, internal/delegate, internal/exec, internal/manual, internal/provider, internal/provider/modelapi, internal/remote, internal/run, internal/seniordev/app, internal/session, internal/tui2/prose and internal/tui3; internal/session and internal/tui3 ran as 8 shards each (112 s and 19 s). The same target with BASE=origin/dev~5 on a clean dev was red three times in a row before this branch, with the same ten tests each time.

A follow-up commit is coming that takes the review suggestion — a readHost seam so the five held-run tests run on macOS too instead of skipping — and I will post that run's result here as well.

Skipping the four held-run tests on a host without /proc/meminfo meant nobody
on a Mac ever ran them, and a regression in holding, reopening on the same
admission, keeping the accepted crew or reading settings before creation would
show up only on CI. hostReading was the one place both /proc files are read,
so the governors are now built over a package-level readHost, and
littleMemoryHost states a quiet machine with one gibibyte available for the
rest of a test — the 1 TiB floor holds everywhere, deterministically, and the
seam is restored when the test ends. requireHostMemoryReading stays for the
one test whose claim is about this host's own reading.

With the hold finally reachable on a Mac, the reopen test met the same
/var -> /private/var mismatch the others did on its recorded ground, and
compares against canonicalPath now.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
@ZeroPoint95

Copy link
Copy Markdown
Contributor Author

make pr-ready BASE=origin/dev~6 on 23f4f36 (with the readHost seam), this Mac: exit 0, zero failing tests, 546 s wall; internal/session 8 shards 119 s, internal/tui3 8 shards 19 s, cmd/codeaf 127 s, internal/provider 69 s. The four held-run tests now run on this Mac rather than skipping, and each passed -count=3 alone. GitHub's light gate and touched packages on the same commit are green on ubuntu.

@ZeroPoint95
ZeroPoint95 marked this pull request as ready for review October 1, 2026 15:45
ZeroPoint95 and others added 5 commits October 2, 2026 09:25
The breakpoints law measured marshalMarked itself and allowed the result
slice on top, so an allocation added inside marshalMarked raised its own
allowance and passed: a defensive strings.Clone of the tool content failed
the old constant (9 against 8) and passed the new form. The law now prices
only the encoding/json calls the two marked positions are designed to make,
built from the package's own wire types with their inputs prebuilt, names
this package's own allocations (the result slice, the parts slice of the
marked system message, nothing for a tool result), and checks each marked
position before the whole warm path. It stays toolchain-independent: 8 on
Go 1.26 and 11 on Go 1.27, both measured. The equality between 8 and 81
turns is unchanged.

PERF.md and the change entry now say exactly that, call the allocation
failure a toolchain assumption rather than a Linux one, and give the held
runs' memory floor in MiB.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
The littleMemoryHost comment called the 1 << 40 floor 1 TiB, but
TaskMinFreeMB is in MiB, so the floor is 1 EiB. Only the comment changes.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
TestHeldBeltRunStopsWithoutPreparingRepository polled agent.beltRun without
agent.beltMu while releaseBeltRun clears it under that lock, so -race
reported a data race in about one run in three (on dev as well; CI does not
run -race). The poll now takes the lock, as the test's other read already
does; -race -count=10 is clean.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
The law prices encoding/json by marshalling the package's own wire types,
so a MarshalJSON added to one of them would put this package's allocations
inside the measured price and raise its own allowance: a defensive copy in
such a method took the warm encode from 8 to 11 with identical wire bytes,
and the law passed. The price now refuses json.Marshaler and
encoding.TextMarshaler on markedTextPart, markedMessage and
markedToolMessage, as values and as pointers, before measuring; a marshaler
added on purpose has to be priced as named overhead here and in PERF.md
together.

The per-position failure now says which way the count moved, and names
what to update together when the change is deliberate.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
@AbirAbbas

Copy link
Copy Markdown
Collaborator

Taking over: @ZeroPoint95, thanks, the macOS diagnosis holds. I checked it on Linux. With no /proc and a symlinked TMPDIR standing in for macOS, all nine fixed tests fail on dev and pass on this branch (×3). In production every governor still reads exactly hostReading. In the real binary, codeaf do is held with waiting · machine busy · available memory under task.min_free_mb … under a huge floor and starts the work under the defaults. Go 1.27 does measure 11 where 1.26 measures 8.

One claim did not hold: the new allocation law was looser than the constant, not stricter. It priced marshalMarked itself, so an allocation added inside it raised its own allowance (a strings.Clone of the tool content failed the old 8 with 9 and passed the new form). Pushed:

  • 011f06482: the warm encode law now measures only the encoding/json calls the two marked positions make. It names this package's own allocations (the result slice, the system message's parts slice, nothing for a tool result) and checks each position before the whole path. It is still toolchain-independent (8 on Go 1.26, 11 on Go 1.27) and now fails the regressions above. PERF.md and the change entry say exactly that, and the entry calls the allocation failure a toolchain assumption rather than a Linux one.
  • 2c35c8669: 1 << 40 MiB is 1 EiB, not 1 TiB, so the littleMemoryHost comment now says EiB.
  • 52397a076: TestHeldBeltRunStopsWithoutPreparingRepository polled agent.beltRun without beltMu, which -race reported in about one run in three. This predates the PR (it shows on dev too) and CI does not run -race. The poll now takes the lock.
  • 8448f431c: the law refuses a MarshalJSON/MarshalText on the three wire types. Otherwise such a method would land inside the measured encoding/json price and hide its own allocations.

Follow-up, not done here: TestAGroundedTaskRecordsWhichRungMadeItsWorld (internal/session/groundladder_test.go:85) compares a canonical description with the raw temp path. It passes on macOS only because /private/var/… contains /var/… as a substring, and it fails under any other symlinked TMPDIR.

@AbirAbbas AbirAbbas left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Verified and taken over; merging.

@AbirAbbas
AbirAbbas merged commit 3a00ee6 into dev Oct 2, 2026
8 checks passed
@AbirAbbas
AbirAbbas deleted the zeropoint95/test-suite-green-on-macos branch October 2, 2026 15:08
@santoshkumarradha santoshkumarradha added hygiene Tests, laws, dead code, duplication — no person-facing change area:tests The suite itself — flakes, harnesses, laws, CI reds labels Oct 3, 2026
@santoshkumarradha santoshkumarradha added this to the Tests & tooling milestone Oct 3, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area:tests The suite itself — flakes, harnesses, laws, CI reds hygiene Tests, laws, dead code, duplication — no person-facing change

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants