feat(moatinit): rewrite the container entrypoint in Go and remove the shell by dpup · Pull Request #441 · majorcontext/moat

dpup · 2026-07-17T21:15:55Z

What

Replaces the 611-line internal/deps/scripts/moat-init.sh container entrypoint with a compiled Go binary (internal/moatinit, embedded and shipped as /usr/local/bin/moat-init), then removes the shell entirely so the Go binary is the sole container entrypoint.

Behavioral parity with the shell is the contract: same phase ordering, same fail-closed vs best-effort classification per operation, and verbatim user-facing error wording.

Why

The entrypoint was the most under-tested, brittle-to-modify component in the repo — verified almost entirely by strings.Contains(MoatInitScript, …) assertions that grep the script text, not its behavior. Yet it encodes fail-closed /etc/hosts handling, four chmod 600 secret contracts, a GNU-tar-1.34 exclude quirk, POSIX pipeline exit-code capture, and gosu privilege-drop semantics — each a landmine for a naive edit. Porting the logic to Go makes every phase a directly unit-testable function.

How

Logic in Go, mechanics delegated. Env parsing, branch/mode selection, phase ordering, error classification, arg/exclude computation, and privilege-drop selection move into Go behind a Sys seam (identity/filesystem/subprocess/DNS are injectable, so phases run under go test against a t.TempDir() root with no container and no root). The security-sensitive/mechanical steps stay as the audited tools already in the image: gosu performs the privilege drop, socat the SSH bridge, tar the workspace byte-copy.
15 ordered phases (/etc/hosts → SSH bridge → agent staging → init files → clipboard → git → docker → volume populate → workspace .mcp.json → pre-run hook → exec dispatch). The hand-off is syscall.Exec (never fork+wait), preserving PID-1 semantics so detached children (socat/Xvfb/dockerd) reparent exactly as under the shell's exec.
Embedded + fail-closed. The binary is //go:embed'd (static, CGO_ENABLED=0, linux amd64/arm64). Committed blobs are fail-closed shell-script stubs so a fresh clone compiles; go generate ./internal/initbin cross-compiles the real binaries; a goreleaser before-hook gate refuses stub/stale blobs and execs --plan as a positive functional check.
--plan — a side-effect-free dry-run that prints the ordered actions the entrypoint would take for the current environment (moat exec <run> -- /usr/local/bin/moat-init --plan).

Testing

Unit + integration for every phase — injected root, recorded chowns, stubbed resolver; the direct replacement for the old grep-the-text tests.
Differential against the live tools it emulates: IFS=<tab> read vs sh, base64 accept/reject vs base64 -d. This caught a real divergence — coreutils base64 -d rejects carriage returns while Go's decoder silently strips them — now fixed.
e2e acceptance harness (internal/e2e/entrypoint_acceptance_test.go): runs the entrypoint in a real container and asserts the privilege drop, staged file modes/ownership, MOAT_INIT_FILES env scrub, DISPLAY export, and Xvfb surviving the exec handoff.
Golden files for --plan output and the fatal-error stderr contract.

Build note (footgun-proofed)

make build / build-cli / test-e2e regenerate the real embedded binaries, run their work, then restore the committed stubs — even on failure — so the ~2.5 MB artifacts never linger as tracked-file modifications. TestCommittedBlobsAreStubs is a commit guard that fails if real binaries are ever committed (closing the gap where a real binary + its regenerated checksum would pass the checksum test).

Notes for reviewers

Cache keys for run images are re-salted (moat-init-v3), so cached images rebuild once. Workflows pinning a concrete moat/run:<hash> tag must re-tag/rebuild.
origin/main (feat(copilot): carry over host settings into container #438) is merged in; the only overlap was a CHANGELOG entry.
Manual sign-off still recommended on the macOS/Apple runtime legs that CI can't exercise: Docker-Desktop-macOS @host.docker.internal DNS resolution and Apple-container child reaping.

🤖 Generated with Claude Code

Design and implementation plan for replacing the 611-line embedded moat-init.sh container entrypoint with a compiled Go binary (cmd/moat-init). The rewrite lifts the business logic into Go where it is unit-testable while keeping gosu/socat/tar as targeted subprocesses (privilege drop, SSH bridge, volume byte-copy) — not a base-image slimming effort. Includes a 144-item requirements catalog extracted line-by-line from the current script (the acceptance-test basis), 36 adversarial-review additions, the build/embed/release design (per-arch //go:embed, offline build preserved, Homebrew unchanged), the unit/integration/e2e + parity harness test strategy, an example-based probe matrix, a 9-commit plan, and a risk register. Refined over two ce-doc-review passes plus a design-principle retarget. Open Decisions #1 (keep gosu) and #2 (keep tar) resolve to targeted subprocesses; the native-replacement PR-split is moot; --plan scoping remains under Deferred / Open Questions.

…d init binaries Commit 1 of the moat-init shell->Go rewrite (docs/plans/2026-07-01-moat-init-go-rewrite-plan.md). No behavior change: the dispatcher defaults to the shell entrypoint. - cmd/moat-init + internal/moatinit skeleton with the Sys seam (identity/ fs/subprocess/DNS operations injectable for tests); pipeline fails closed until the exec-dispatch phase lands - internal/initbin embeds prebuilt static linux/{amd64,arm64} entrypoint binaries; committed blobs are fail-closed shell-script stubs (reviewable text) regenerated by 'go generate ./internal/initbin' via make build and the goreleaser before hook; checksums.txt + unit test catch stale blobs; embed/ dir avoids the bare dist/ gitignore pattern - writeEntrypoint dual-ships: dispatcher at /usr/local/bin/moat-init plus moat-init-sh and arch-matched moat-init-go; MOAT_INIT_IMPL/MOAT_INIT_LEGACY are a closed enum, read once, unset before handoff - run.Create() rejects the reserved dispatcher vars in moat.yaml env and -e (always-on, unlike the proxy-var filter which is gated on needsProxy) - image cache key folds in dispatcher + binary bytes under a bumped moat-init-v2 salt so pre-dispatcher cached images re-key

Commit 2 of the moat-init rewrite: the branch/mode/parse logic moves into table-tested Go — extra-hosts tokenizing (first-colon split, word-split, skip rules, @-target discrimination), the shared targetHome/ownership idiom, MOAT_INIT_FILES record parsing (exact POSIX IFS=<tab> read -r semantics, verified against a live /bin/sh — leading tabs strip, interior runs collapse, trailing runs trim), buffer-first base64 decode, docker dind/host-mode predicates and mutex, git config command assembly (insteadOf exact-"1" gate), and the workspace-volume gates (exact-"1" enable, staging default, verbatim exclude content, set -f chown paths).

… workspace mcp.json, git config Commit 3 of the moat-init rewrite: the fail-closed /etc/hosts injection (exact three-line error contracts, IPv4-preferred resolve with the 25x0.2s budget, sanctioned IPv6/loopback-fallback warning), the four agent staging blocks (allowlist copies, cp -p mode preservation, the chmod-600 secret contracts, best-effort lchown hand-off), MOAT_INIT_FILES writes (0600 files, 0755 immediate parent, ancestor-chown walk stopping at INIT_HOME, decode-to-buffer fail-closed on invalid base64, unset after the loop), workspace .mcp.json copy, and best-effort git --system config gated on a git binary. Integration tests run the real phase bodies against a t.TempDir()-rooted Sys with recorded chowns and a stubbed resolver; a real-git test verifies the assembled argv against GIT_CONFIG_SYSTEM.

… named-volume chown Commit 4 of the moat-init rewrite. Go now owns the populate business logic: the exact-"1" gate, the defensive root guard, staging resolution (/mnt/host-workspace default), verbatim exclude-file assembly (newline delimited — GNU tar 1.34's --null --exclude-from applies only the first record), the tar -cf - . | tar -xf - argument vectors, BOTH pipe exit codes (a source-side read error can no longer hide behind the rightmost status), temp-file cleanup on success and failure paths, and the fatal recursive re-own of /workspace via lchown (symlink targets outside the tree are never re-owned). The byte copy stays a targeted tar subprocess. A missing staging root maps to the same fatal rc-check path — never a silently empty /workspace. Also ports the inline named-volume chown block at its exact script position (before the populate/mcp/hook tail): non-recursive, best-effort, glob characters literal. Sys gains RealPath (subprocess-visible path mapping under an injected test root) and Pipe serializes a shared non-file stderr writer (the two tar legs would otherwise race on the test buffer).

Commit 5 of the moat-init rewrite. socat, Xvfb, and dockerd stay targeted long-lived subprocesses, spawned exactly as the shell backgrounded them — same process group (non-interactive sh has job control off, so no Setpgid; signal delivery to the container's foreground group must reach them identically), reaped opportunistically so the kill -0 liveness check matches the shell's view of a dead background job, and surviving the final exec because the handoff replaces the process image without touching children. Go owns the surrounding decisions: SSH socket dir/mode/ownership, the 20x0.1s socket wait with the two exact warnings; Xvfb :99 fire-and-forget with DISPLAY exported even on spawn failure; the docker mutex error, the dind readiness poll (30x1s, socket AND docker info — the info probe gets a 2s per-attempt timeout so a hang cannot eat the loop budget), the exact error/tail-20 diagnostics, dind group setup, and host-socket GID detection via an in-container stat (no GNU stat -c) with groupadd/usermod as best-effort subprocesses. The pre-run hook ports the root->gosu->sh -c branch verbatim (Go selects the branch; gosu runs the hook), confines the non-root hook's cwd to the child, and preserves the framed diagnostic plus the hook's literal exit code (issue #372). Cmd gains LogFile (dockerd's 2>&1 redirect) and Timeout; signal-terminated children report 128+n like $?.

…p + env scrub Commit 6 of the moat-init rewrite: the terminal handoff. Go computes the dispatch — non-root execs argv directly; root with moatuser execs 'gosu moatuser "$@"'; root without moatuser fails closed with the exact multi-line remediation — and syscall.Exec replaces the process image (PID preserved, detached children reparent exactly as under the shell's exec; fork+wait is prohibited). gosu owns setgroups/setgid/setuid and re-reads /etc/group at exec time, so groups added mid-run by the docker phases are picked up with no native group-resolution code. Both handoff paths build the exec environment by explicitly removing MOAT_INIT_FILES from the inherited environment (defense in depth over the init-files phase's unset); nothing else is scrubbed (parity). A failed exec maps to shell codes (127 not-found / 126 otherwise). Pipeline tests now drive Run() end-to-end to the recorded handoff, pin the full phase order (X-ORDER-GLOBAL), the pre_run literal exit-code passthrough, and the hook/exec branch-selection consistency (EXEC-14).

Commit 7 of the moat-init rewrite. 'moat-init --plan' prints the ordered actions the entrypoint would take for the current environment — one line per decision, side-effect-free (read-only stats and lookups only). It is both a permanent debugging affordance the shell could never offer and the release pipeline's positive functional gate: the privilege-drop and MOAT_INIT_FILES-scrub lines must appear, so a regenerated-but-defective binary fails the gate regardless of its checksum. Golden files pin the --plan output (full-featured and minimal environments) and the complete fatal stderr contract — every scripted error block, byte for byte, with its exit code (including the pre_run hook's literal 42).

…obes Commit 8 of the moat-init rewrite. A fixed state-dumper command emits a machine-parseable manifest — identity with the supplementary set compared order-independently, the exec'd environment with proxy auth tokens redacted to a fixed placeholder (and a direct MOAT_INIT_FILES leak scan, since both legs leaking would still diff clean), /etc/hosts, the system git config, file trees with modes+ownership (mtimes masked; the statsig dir compared as a content hash), and a census of the expected long-lived children — and the harness diffs it between MOAT_INIT_IMPL=sh and =go. The baseline scenario runs on every available runtime (Docker and Apple reparent children differently); the remaining scenarios cover the plan's coverage-gap fixtures on Docker: pre_run hook, workspace-volume populate, clipboard/Xvfb, and multi-record MOAT_INIT_FILES with a deep parent chain. Negative probes assert the failing pre_run hook's framed diagnostic + literal exit code on both implementations, the dispatcher's closed-enum fatal on an unknown MOAT_INIT_IMPL, and that the Go leg's detached children survive the exec handoff. run.Create() now forwards MOAT_INIT_IMPL/MOAT_INIT_LEGACY from the moat process's own environment — the operator-only channel the harness drives (user-supplied sources remain rejected).

…y-run Commit 9 of the moat-init rewrite (docs leg). Adds the container-startup section to the sandboxing concept page (entrypoint responsibilities, the dual-ship dispatcher, reserved MOAT_INIT_IMPL/MOAT_INIT_LEGACY controls, and the --plan dry-run with a working moat exec invocation) and the CHANGELOG entry. The default implementation deliberately stays 'sh': the plan gates the cutover flip on the e2e parity harness passing on real runtimes plus a documented manual macOS sign-off (Docker Desktop DNS + Apple reaping legs are untestable in CI). The flip is a one-line dispatcher change (moat-init-dispatch.sh default) once those gates pass.

Confirmed findings from the multi-agent review (4/23 upheld), plus cheap parity gaps the refuters dismissed on materiality only: - Close the moat.yaml secrets: bypass of the reserved dispatcher-env guard — secret KEYS are user-chosen and land in the container env verbatim, so validateReservedEnv now covers cfg.Secrets too - Implement the plan-promised stub gates instead of claiming them: internal/initbin/gate (goreleaser before hook) refuses stub/stale blobs and execs the regenerated binary's --plan as the positive functional check; make test-e2e now regenerates the binaries first; the parity harness skips its go legs when the test binary embeds a stub; the initbin doc comment now describes what actually exists - Normalize Docker's per-container /etc/hosts self-entry out of the parity manifest (the [hosts] section could never match across legs) - Empty user command: exec dispatch no longer panics — non-root exits 0 (the script's bare exec is a no-op and the script ends), root path hands gosu its own usage error - Shell-style path concatenation for TARGET_HOME destinations, so an empty HOME yields the script's root-anchored /.claude, never a cwd-relative path - pre_run hook inherits the container's stdin (Cmd.Stdin), matching the shell; empty dockerd log tails print nothing - Companion tests: whitespace-only pre_run runs (EXEC-01), empty excludes copy everything (WS-06), secrets-guard cases; the parity exit-code assertion now requires 'exit code 7', not a bare '7'

The in-image Go implementation moves from /usr/local/bin/moat-init-go to /usr/local/bin/moat-commit (context file, COPY, chmod, dispatcher exec target, tests, and docs). The dispatcher path (/usr/local/bin/moat-init), the shell leg (moat-init-sh), the cmd/moat-init source package, and the embedded blob names are unchanged.

… fixes Adds a differential harness that runs the REAL moat-init.sh and the REAL compiled Go entrypoint side by side against ~20 crafted environments on the non-root branch — comparing exit codes, contract stderr (byte-for-byte where scripted), the child-visible environment, and full HOME trees (paths, types, modes, content hashes). Covers all four agent staging blocks, multi-record init files with the env scrub, clipboard DISPLAY export, git system config (identical config-file bytes, quoted identity values, insteadOf both gates), docker mutex/dind/populate/chown guards, pre_run success/literal-42/signal-143/whitespace hooks, malformed extra-hosts, the ~5s resolve-failure fatal (byte-identical three-line error), exit-code passthrough, and 127 on a missing command. Plus live-shell differential corpora for the IFS=<tab> record splitter and base64 acceptance, and the documented invalid-base64 residue divergence (shell leaves a truncated file; Go leaves nothing). The harness caught one real parity bug, fixed here: coreutils base64 -d REJECTS carriage returns while Go's decoder silently strips them, so a CRLF-joined MOAT_INIT_FILES would fail closed under the shell but decode under Go — decodeInitContent now rejects CR like base64 -d. Also pins the edge cases fixed in the review pass: empty argv (non-root bare-exec no-op exits 0; root hands bare 'gosu moatuser' to gosu) and empty HOME staying root-anchored (/.codex, never cwd-relative).

Correcting the earlier moat-commit rename (a typo for the intended name). The Go implementation is now installed at /usr/local/bin/moat-init, which matches its cmd/moat-init source package and is the path the entrypoint takes over directly at cutover. Because the migration-window dispatcher had occupied that path, the three in-image pieces are re-laid-out: /usr/local/bin/moat-init Go entrypoint (was moat-commit) /usr/local/bin/moat-init.sh legacy shell entrypoint (was moat-init-sh) /usr/local/bin/moat-init-dispatch dispatcher, the ENTRYPOINT (was moat-init) The dispatcher content change re-keys the image cache on its own (the tag folds in the dispatcher bytes), so no additional salt bump is needed. At cutover the dispatcher and shell are dropped and the ENTRYPOINT becomes /usr/local/bin/moat-init directly. Updated writeEntrypoint, the dispatcher script, the dispatch/entrypoint tests, the differential harness's temp binary name, the volume-ownership-helper comment (now path-agnostic), docs, and the CHANGELOG.

…nary is the sole entrypoint The Go moat-init entrypoint has been validated working, so the migration scaffolding is retired. Deletes: - internal/deps/scripts/moat-init.sh (611-line legacy entrypoint) - internal/deps/scripts/moat-init-dispatch.sh (the sh/go selector) - MoatInitScript / MoatInitDispatcher embeds - MOAT_INIT_IMPL / MOAT_INIT_LEGACY and the whole reserved-env guard (internal/run/envguard.go + operatorInitEnv), now dead with the vars gone writeEntrypoint now COPYs the arch-matched moat-init binary to /usr/local/bin/moat-init and sets it as the ENTRYPOINT directly — no dispatcher, no shell leg. The cache-key component drops to the binary hash under a bumped moat-init-v3 salt (so images cached under the v2 dual-ship scheme re-key once). Test surface follows the code: - deleted the strings.Contains(MoatInitScript, ...) shell-marker tests (git identity/proxy-auth/pre-run/volume-populate) — their behavior is covered by internal/moatinit unit + integration tests - the sh-vs-go differential harness (shellparity_test.go) is trimmed to the two live-tool oracles that don't need the shell entrypoint (IFS=<tab> read splitting vs sh; base64 accept/reject vs base64 -d) - the e2e parity harness becomes a single-impl acceptance harness (entrypoint_acceptance_test.go): runs the Go entrypoint in a real container and asserts privilege drop, staged file modes/ownership, env scrub, DISPLAY export, and Xvfb surviving the exec handoff - the deps dispatch test becomes moat_init_entrypoint_test.go Docs + CHANGELOG updated to describe a single Go entrypoint. Net: +509 / -2126 lines.

…gainst committing them Removes the footgun where 'make build-cli' (which runs go generate to cross-compile the real ~2.5 MB moat-init binaries over the committed stubs) left those artifacts as tracked-file modifications, one 'git add' away from being committed. - build, build-cli, and test-e2e now regenerate the binaries, run their work, then restore the committed stubs — even if the build/test fails (the stubs are restored regardless, and the failure exit code still surfaces). The real binaries are baked into the built artifact at compile time, so reverting the source blobs afterward does not change what was built. generate-init / restore-init-stubs remain for the manual 'go test -tags=e2e' path. - TestCommittedBlobsAreStubs is a commit guard: it fails if the tracked embed blobs are real binaries rather than stubs. This closes the gap TestEmbeddedChecksums leaves — that test passes when a real binary is committed alongside its regenerated checksum (exactly the state the footgun produced); the guard fails on it. A clean checkout (CI, any commit) always carries stubs, so the guard only fires on the mistake. Verified: build-cli produces ./moat and leaves embed/ clean; a failing build still restores the stubs; the guard fails on real binaries and passes on stubs.

…rite # Conflicts: # CHANGELOG.md

claude · 2026-07-17T21:16:43Z

📦 Previous Review (superseded by newer review)

Claude finished @dpup's task in 2m 20s —— View job

Reviewing this PR

Gather context (read CLAUDE.md, diff, changed files)
Understand the request (code review of the Go entrypoint rewrite)
Confirm go build ./... and go vet ./... are clean on this branch
Deep-dive review across core phases (hosts/ssh/docker/volume), agent/initfiles/git/exec phases, phase orchestration + embed/gate/build system, and integration + e2e wiring (running in parallel)
Synthesize and verify findings
Post review feedback

claude · 2026-07-18T01:47:23Z

Claude finished @dpup's task in 2m 24s —— View job

Reviewing this PR

Gather context (read CLAUDE.md, diff, changed files)
Understand the request (code review of the Go entrypoint rewrite)
Confirm go build ./... and go vet ./... are clean on this branch
Deep-dive review across core phases (hosts/ssh/docker/volume/exec), agent/initfiles/git/hook phases, phase orchestration + embed/gate/build system, and e2e/docs (running in parallel)
Synthesize and verify findings
Post review feedback

dpup added 18 commits July 16, 2026 05:14

Merge remote-tracking branch 'origin/main' into feat/moat-init-go-rew…

114f0e2

…rite # Conflicts: # CHANGELOG.md

docs(changelog): fill the moat-init entrypoint PR link

73f8b99

dpup force-pushed the feat/moat-init-go-rewrite branch from 49ff6c5 to 73f8b99 Compare July 18, 2026 01:46

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

feat(moatinit): rewrite the container entrypoint in Go and remove the shell#441

feat(moatinit): rewrite the container entrypoint in Go and remove the shell#441
dpup wants to merge 18 commits into
mainfrom
feat/moat-init-go-rewrite

dpup commented Jul 17, 2026

Uh oh!

claude Bot commented Jul 17, 2026 •

edited by github-actions Bot

Loading

Reviewing this PR

Uh oh!

claude Bot commented Jul 18, 2026 •

edited

Loading

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

Uh oh!

Conversation

dpup commented Jul 17, 2026

What

Why

How

Testing

Build note (footgun-proofed)

Notes for reviewers

Uh oh!

claude Bot commented Jul 17, 2026 • edited by github-actions Bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Reviewing this PR

Uh oh!

claude Bot commented Jul 18, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Reviewing this PR

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

claude Bot commented Jul 17, 2026 •

edited by github-actions Bot

Loading

claude Bot commented Jul 18, 2026 •

edited

Loading