Skip to content

fix: make unattended authority consistent across the shipping workflow - #305

Merged
amitpaz1 merged 12 commits into
mainfrom
fix/unattended-workflow
Sep 10, 2026
Merged

fix: make unattended authority consistent across the shipping workflow#305
amitpaz1 merged 12 commits into
mainfrom
fix/unattended-workflow

Conversation

@amitpaz1

@amitpaz1 amitpaz1 commented Sep 10, 2026

Copy link
Copy Markdown
Member

Summary

One user-directed correction for the entire unattended workflow: startup, external-input authority, real child inheritance, background tests, diagnostics, optional bookkeeping, clarification/supervisor human gates, and authorized shipping. Builder: Codex/operator with isolated helper worktrees, outside AgentRig. The existing AgentRig session 86ce6bc6 and its separate PR #304 are preserved.

  • Explicit YOLO/skip-permissions now selects trusted SDK unattended mode. Existing policy allows survive external input; unresolved asks are denied without human callbacks. Children inherit the live parent mode.
  • Explicit base/live denials, audit boundaries, tool restrictions, sandbox enforcement, MCP definition pins and project trust remain. No automatic escape, auto-trust, fabricated answers or grants from transcript content.
  • Required questions use only separately configured automated policies in unattended runs; no default automatic choice. Supervisor keeps automated guidance and controls without its human escalation prompt.
  • Implicit diagnostics locate trusted project-local TypeScript using Node instead of depending on PATH; explicit checker configuration is unchanged.
  • Shipping instructions carry explicit upfront task-specific merge authorization through reviews, exact-head CI, merge and post-merge CI. Optional suggestions do not create endless repair rounds.

Authorization and scope

Latest human request: “Fix it. Cover the entire workflow.”

Standing end-to-end instruction: “Then when everything is green and everything passes, you merge the PR.”

These apply to this corrective PR for the unattended harness workflow, not unrelated work. Tool permission/YOLO alone is not merge authorization; later revocation or narrowing wins.

Contract amendment / plan departures

This is a direct user-requested correction, not an R13c or R17 roadmap-row implementation. It deliberately supersedes R13c's mandatory fresh human consent under explicit operator YOLO. Default interactive behavior is unchanged. The amendment is recorded in PLAN, DEFAULTS, R13c's historical contract and docs/plans/unattended-workflow.md.

Risk: explicit operator blanket authority remains broad even after untrusted content. Provenance is not promoted, but this increases prompt-injection exposure. It applies consistently to explicitly YOLO-configured MCP serving; do not expose that authority to untrusted clients. Scoped/default serving retains the fresh-consent restriction. No security-default flip.

Flag meanings changed: --yolo and its --dangerously-skip-permissions spelling now actually suppress human gates across the workflow, preserving denials and isolation restrictions. No new flag, model or budget default.

Verification

  • Initial core regressions failed before the change. Scripted-provider integration runs real edits, background Node/test collection, child external reads/exec, memory attempt writes and explicit parent/child write denials.
  • Targeted mutation controls cover effective parent/child mode, trust-limited compiler discovery, configured answer-policy forwarding, false human answer provenance, supervisor dialog suppression and shipping authorization guidance. All restored. Prose tests do not claim to prove a real GitHub merge.
  • Final updated-main head a7912434db1d95fd14e3413488b96da011cf98bd: pnpm build, pnpm typecheck and TMPDIR=/var/tmp/agentrig-unattended-tests.KEBvMC pnpm test all exited zero; 3,834 passed / 4 skipped across 252 files.
  • Independent Claude Code full review and Codex full review passed, including independent full suites and load-bearing mutations. Focused fixture delta and final-head main integration checks also passed. No open defects. Claude's optional TUI automated-answer-policy suggestion is not an acceptance criterion; the documented no-human-dialog behavior is intentional, and headless explicit policies work.
  • Initial macOS and Windows CI exposed the same scoped fixture alias mismatch. Reproduced fail-first on Linux using an aliased TMPDIR; fixture-only realpath canonicalization fixes it without widening reads or dropping assertions. Both reviewers independently reversed it and caught the failure. No blind rerun or production permission change.
  • Main PR fix(cli): support --version from the installed package manifest #304 is included (only STATUS conflicted; both entries preserved). Its post-merge CI and structure check passed. Final exact-head a7912434db1d95fd14e3413488b96da011cf98bd Ubuntu/macOS/Windows CI and structure check all passed.
  • Merged and verified: 93b65ee7c4c54fc51432904939a6e37f69c105a0, tree identical to the reviewed head. Post-merge Ubuntu/macOS/Windows CI and post-merge structure all passed on the first attempt. Local main fast-forwarded and rebuilt, 24 installed-tree smoke tests passed; personal profile resolves YOLO on, checkpoints off, trusted local TypeScript path. No user configuration was changed; the completed version branch and session are preserved. No open PRs remain.
  • Added Windows coverage for the new workflow, diagnostics, questions, supervisor and guidance fixtures. No live-model quality claim or claim of future hosted CI success.

@amitpaz1

Copy link
Copy Markdown
Member Author

Independent Codex review — session 01a08b81-bdb9-7320-98b6-c8b4b18add59, exclusive worktree, head d7014ede71dd81ae36e707ae5b1c66e580eff9c9.

PASS — no blockers found for PR #305 at exact head d7014ede71dd81ae36e707ae5b1c66e580eff9c9.

Read the review skill fully, PR body, and complete git diff origin/main...HEAD. Verified the prepared detached worktree was clean with dependencies installed. The authorized unattended amendment is consistent with the revised contract. Explicit denials, sandbox refusal, project trust, MCP pinning, automated answer provenance and live child inheritance remain intact.

Advisory: No additional findings. Scripted workflow and shipping-prose tests establish their stated contracts; they do not prove live-model behavior or an actual GitHub merge.

All verification used TMPDIR=/var/tmp/agentrig-unattended-tests.KEBvMC:

Separate command Result
pnpm build Exit 0
pnpm test Exit 0; 251 files, 3,830 passed, 4 skipped
pnpm typecheck Exit 0

Four sequential mutants were tested with pnpm exec vitest run <file>:

Mutation Test file Result
Restore old fresh-consent gate packages/cli/test/unattended-workflow.test.ts Exit 1; 2 failed
Remove live child-mode inheritance packages/core/test/external-expansion.test.ts Exit 1; 1 failed
Remove compiler realpath containment packages/cli/test/diagnostic-resolution.test.ts Exit 1; outside-symlink case failed
Accept human-labelled automated replies packages/cli/test/unattended-questions.test.ts Exit 1; claimed negative control failed as expected

Every mutant and restoration received a full pnpm build, all exit 0. After restoration, running those four test files together passed 49 tests, exit 0. Mutation commands and restoration script retained in the local review artifacts.

Original bytes match HEAD exactly; tracked, index and untracked state are clean. All review subprocesses completed. /tmp/.git and sibling worktrees were untouched.

PR head and remote main remained unchanged; main is 9591b1be0c6ac2ef44da43e96bc10b0bfc5bcdad, already an ancestor of HEAD. Exact-head scripted-structure passed; Ubuntu, macOS and Windows CI remains in progress. Pending CI is a delivery prerequisite, not a code defect; further monitoring belongs to the conductor.

@amitpaz1

Copy link
Copy Markdown
Member Author

Independent Claude Code review — session 20b5dc99-2cd6-491d-8e9b-a40f9870a14c.

Review of PR #305 at head d7014ed. Verdict: pass, no blockers. Base origin/main stayed at 9591b1b for the whole review, so there was no delta to re-review.

Blockers

None. I found no concrete correctness or security regression against the revised unattended contract. Explicit denies, sandbox refusal, MCP definition pinning, project trust, automated-answer provenance and live child inheritance all hold in code and are each pinned by a test that a targeted mutant fails.

Advisory (optional, not defects against the contract)

  • TUI/ACP under a YOLO profile lose ask_user entirely. The TUI and ACP wire only the human question handler, never the unattended seam, so with the user's personal profile setting yolo a single required question ends the session with reason "error" while a human is sitting at the terminal. This matches the documented contract in docs/QUESTIONS.md, so it is a design consequence, not a bug. If that proves too blunt in practice, a TUI-level answer policy would be the fix, not re-enabling the dialog.
  • No arbiter verdict block. The PR body's amendment section carries no arbiter block or session id as the review skill's contract-fidelity rule expects. The brief states this amendment is human-authorized, so I do not raise it as a HIGH; noting it for the record.
  • PR body is forward-looking. It says final-head results and CI receipts "will be recorded here." That is a conductor task before landing, not a code issue.
  • Test counts. STATUS cites 3,818 tests pre-review; I observed 3,830 passed plus 4 skipped at this head, consistent with the later fixtures the text mentions.

Evidence

Worktree confirmed: HEAD d7014ed, porcelain clean, node_modules present, all runs with the requested TMPDIR.

Command Exit Result
pnpm build 0 all packages
pnpm test 0 251 files, 3830 passed, 4 skipped
pnpm typecheck 0 all packages

Mutants, run one at a time with the relevant test file, each restored by byte-copy and hash-verified, core dist rebuilt after restore:

  1. tool-execution.ts: unattended ask falls through to onAsk. Killed by "unattended mode never overrides base ask".
  2. tools/subagent.ts: drop live approvalMode inheritance. Killed by "real child inherits unattended mode even when its factory supplies an interactive asker".
  3. tool-execution.ts: accept human-labelled replies on the unattended seam. Killed by the human-source case in unattended-questions.test.ts.
  4. diagnostic-resolution.ts: remove the trusted-root containment check. This re-runs the PR's claimed "trust-limited compiler discovery" control. Killed by the outside-trust symlink case.
  5. tool-execution.ts: unattended sandbox escalation reaches onAsk. Killed by "unattended mode refuses sandbox escape".

Other checks: no new event types, so the zod/renderEvent rule is untouched. Serving clients can only lower budgets, so YOLO on an MCP server remains an operator setting. The diagnostics tool classifies by exec class, not executable name, so swapping tsc for the Node binary changes nothing in permissions or sandbox. Startup trust uses only user, profile and argv state, and an explicit --no-yolo restores the prompt.

PR body claims verified: fail-first core regression behavior (via mutants 1, 2, 5), configured answer-policy forwarding and human-provenance rejection, MCP pinning refusal without dialog, supervisor escalation omission, trust suppression without project config load. Not verifiable here: Windows coverage and the live-session fix for the original ENOENT symptom.

CI on the exact head at report time: scripted-structure pass, ubuntu pass, macOS and Windows still pending. Pending CI is not a code defect.

Final state: HEAD unchanged at d7014ed, tracked tree and index clean, all mutant originals restored, no agents spawned, nothing pushed, no PRs or issues created. Worktree cleanup left to the conductor.

@amitpaz1

Copy link
Copy Markdown
Member Author

Focused CI fixture correction — e24fbd2

The macOS and Windows failures both identified the scoped MCP fixture's aliased temporary root. Reproduced the same list_sessions refusal locally under an aliased TMPDIR; canonicalizing only the fixture root fixes it. No production permission or assertion changes. Full build/typecheck/test pass again (3,830 passed / 4 skipped).

Claude Code delta

Delta verdict for d7014ed..e24fbd2: pass.

The delta is exactly the described change in packages/cli/test/mcp-serve-cli.test.ts: a realpath import and canonicalizing the fixture cwd before deriving the root, home and memory paths. No assertion, permission, sandbox or production source changed. The fixture still runs with a scoped write allow rather than blanket yolo, so the scoped-read check it exercises stays in force.

Reproduction with the aliased TMPDIR symlink:

Variant Exit Result
HEAD e24fbd2, realpath in place 0 1 passed
realpath removed 1 list_sessions returns "Request refused" and the session id is missing

The failure without realpath is the same lexical scoped-read mismatch reported from macOS CI: the CLI's canonical cwd and the aliased --root spell the same directory differently, so the read policy refuses it. Canonicalizing the fixture is the right fix; it does not widen what the server may read.

Restored state: the test file hash matches its pre-mutant value, porcelain is clean, HEAD is unchanged at e24fbd2. Nothing pushed, no agents spawned. CI on the updated head is left to the conductor.

Codex delta

The first resumed runtime check lacked the original writable review launch setting and stopped at EROFS before executing tests. It did not count as verification. Relaunched in the same exclusive writable setup, with no code or test waiver:

Focused delta PASS at e24fbd22d9f942571b273b98c1d818b3c30a6ba4. No blockers or advisory findings.

Ran each state with:

TMPDIR=/var/tmp/agentrig-unattended-tests-alias pnpm exec vitest run packages/cli/test/mcp-serve-cli.test.ts
  • Fixed fixture: exit 0, 1 test passed.
  • realpath removed: exit 1, reproduced the list_sessions refusal at line 55.
  • Exact bytes restored: exit 0, 1 test passed.

Static verdict unchanged: no assertion, permission, production or sandbox weakening. HEAD is unchanged, fixture bytes match HEAD exactly, working/index state is clean, and all subprocesses completed. No broader review, agents, pushes or merges.

@amitpaz1

Copy link
Copy Markdown
Member Author

Final updated-main integration — a791243

Main PR #304 merged as 030e35f. Only STATUS conflicted; both delivery records are retained. Full local build/typecheck/test exited zero at the integrated head: 3,834 passed / 4 skipped in 252 files. Exact-head and main CI are still running.

Claude Code (same independent reviewer session)

Integration verdict for merge head a791243: pass, integration is clean.

The merge has exactly the expected parents: the reviewed branch head e24fbd2 and main's PR #304 commit 030e35f, which is current origin/main. Comparing the merge against each parent, the branch side reproduces the previously approved unattended diff byte for byte, and the main side reproduces PR #304's version support byte for byte, excluding STATUS. The only differences are hunk line offsets in program.ts and vitest.windows.config.ts, which is what two independent edits to different regions of the same files look like.

STATUS was the sole conflict. It resolves to both delivery entries: the unattended entry first, the PR #304 version entry second, with the earlier sections intact and no text lost from either side.

Check Result
pnpm build exit 0
focused vitest (14 files: version, program, unattended x3, diagnostic resolution, MCP serve CLI and runtime, expansion, config, questions, sandbox evidence, shipping guidance) exit 0, 198 passed
built CLI --version prints the manifest version and exits 0

No resolution or semantic change beyond the STATUS concatenation. Nothing from the merge touches the permission, sandbox, question, trust or MCP paths reviewed earlier, and the version code from main does not intersect them.

Final state: HEAD unchanged at a791243, porcelain clean, nothing pushed, no agents spawned. Exact-head and post-merge CI remain conductor-owned and pending; this verdict covers the local integration only.

Codex (same independent reviewer session)

Final-head integration PASS at a7912434db1d95fd14e3413488b96da011cf98bd.

  • Merge parents match approved e24fbd2 and main’s 030e35fa.
  • Five-file delta contains only the expected version integration. Approved unattended behavior and canonicalized MCP fixture remain intact.
  • STATUS preserves both entries exactly; no unexpected resolution or semantic changes.
  • pnpm build: exit 0.
  • Focused version, program, unattended workflow/questions/supervisor and MCP CLI tests, using aliased TMPDIR: 48 tests passed across 6 files, exit 0.

HEAD unchanged; working tree and index clean. No blockers or advisory findings. Exact-head/post-merge CI remains pending and conductor-owned; no future CI success claimed.

@amitpaz1
amitpaz1 merged commit 93b65ee into main Sep 10, 2026
4 checks passed
@amitpaz1

Copy link
Copy Markdown
Member Author

Completed — merged, green main, local build updated

  • Reviewed head: a7912434db1d95fd14e3413488b96da011cf98bd.
  • Squash merge: 93b65ee7c4c54fc51432904939a6e37f69c105a0. Direct tree comparison to the reviewed head exits zero.
  • Exact-head CI and structure: green.
  • Actual merge-commit Ubuntu/macOS/Windows CI and structure: all green, first attempt.
  • Independent Claude Code session 20b5dc99-2cd6-491d-8e9b-a40f9870a14c and Codex session 01a08b81-bdb9-7320-98b6-c8b4b18add59: full review, targeted CI-fixture delta and final integration checks passed; no unresolved findings. Receipts linked in PR body.
  • Final full local trio: 3,834 tests passed / 4 skipped, build/typecheck exit zero. Load-bearing mutation controls failed as intended and were restored.
  • Local /home/amit/agentrig is clean on merged main, rebuilt successfully; 24 focused installed-tree smoke tests passed. Actual personal-profile resolution shows yolo: true, checkpoints: false, and TypeScript via /usr/bin/node plus the installed compiler path. No user configuration edits.
  • Existing AgentRig session 86ce6bc6 completed its own PR fix(cli): support --version from the installed package manifest #304 with green post-merge CI before this PR landed. Its branch is preserved. No open PRs or new issues remain.

Existing processes retain their loaded runtime: restart the old session with /quit, then agentrig --profile personal. No additional permission flags or query/build session choice is needed.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant