diff --git a/.claude-plugin/marketplace.json b/.claude-plugin/marketplace.json index 6751c501..b97c0930 100644 --- a/.claude-plugin/marketplace.json +++ b/.claude-plugin/marketplace.json @@ -9,7 +9,7 @@ "name": "pstack", "source": "./plugins/pstack", "description": "if you want to go fast, go deep first. pstack helps you write less, but higher quality code. rigorous agent workflows you can parallelize with confidence.", - "version": "1.8.0", + "version": "1.15.0-adam.20261019", "author": { "name": "Lauren Tan (original)" }, diff --git a/.claude/skills/verify-open-pstack/SKILL.md b/.claude/skills/verify-open-pstack/SKILL.md index 5620592c..4a924637 100644 --- a/.claude/skills/verify-open-pstack/SKILL.md +++ b/.claude/skills/verify-open-pstack/SKILL.md @@ -25,7 +25,11 @@ EVIDENCE="$SESSION/live" # must not exist yet .claude/skills/verify-open-pstack/scripts/verify.sh run --pr "$PR" --feature claude:project-skill --self-test --output "$EVIDENCE" ``` -Pick the features that exercise the PR's changed behavior and pass each as `--feature :`, repeatable. List the same features in the PR body, where review sees them. A runtime change needs at least one `--feature`; the run fails before any install without one. The run exercises only the named features, plus `project-skill` in both harnesses under `--self-test`. It never falls back to the classified set, and a full sweep is never the default. +Pick the features that exercise the PR's changed behavior and pass each as `--feature :`, repeatable. List the same features in the PR body, where review sees them. The run exercises only the named features, plus `project-skill` in both harnesses when classification reaches it or under `--self-test`. Before any install or GitHub write, `verify.sh run` makes three checks: + +- Scope: a runtime change needs at least one `--feature`, and every named feature must be in the PR's classified reach. The refusal names each refused feature and the reachable set. A manifest change that only bumps the version reaches only `hook:startup` in that manifest's harness. +- Count: the run prints the number of headless sessions the named features and automatic `project-skill` checks will start. +- Limit: above 12 sessions, the run refuses unless you pass `--wide ""`. The reason must be printable ASCII, at most 160 characters, without backticks or angle brackets. The evidence comment records the planned count and the reason. Reuse the Claude Code and Codex logins you already have; no extra login or credential source flags. Omit `--self-test` for ordinary plugin verification; it is mandatory for this skill's delivery. An agent runs and judges the whole gate from a non-TTY shell; nothing prompts a person. `--runner-route PROVIDER:MODEL@EFFORT`, repeatable, replaces the runner recipe's default route, a role default from the installed candidate's model catalog, for each parent whose provider differs from the route's. @@ -41,7 +45,7 @@ Temporary, GitHub, and fixture state remain run-owned; Git configuration is disa ## Drive -Read `features/README.md` and the document for every named feature. Path classification still runs: it decides whether the PR has a runtime change, and an unmapped path fails closed. It does not choose the features; the agent does. `no runtime change` launches no harness unless `--self-test` is requested. +Read `features/README.md` and the document for every named feature. Path classification computes the features the PR can reach, and an unmapped path fails closed. It does not choose the features; the agent does. `no runtime change` launches no harness unless `--self-test` is requested. Create detached exact-head candidate checkouts and run-owned temporary/fixture state, only for harnesses with a named feature. Every named feature resolves to a recipe in `features/recipes.ts` before the first session starts: one or more cases, each with a fixture, a prompt, and an assertion the trusted parent evaluates. A feature without a recipe fails as `missing-recipe:/`; `assets:codex` fails as `unsupported-native-consumer` until it has a recipe. Each case runs as one fresh headless session, sequentially, and setup never runs alongside anything. @@ -71,10 +75,10 @@ Each headless session exits on its own. Verify any setup restoration before publ - `scripts/verify.sh doctor --output `: parent capability and login-state report, no installation/publication. - `scripts/verify.sh doctor --candidate --output `: candidate-safe self-test probe; no publisher authentication probing. -- `scripts/verify.sh run --pr --feature :... [--self-test] [--runner-route PROVIDER:MODEL@EFFORT]... --output `: headless exact-head verification of the named features with machine-checked recipes; no terminal or prompts. The evidence comment lists only the features that ran. +- `scripts/verify.sh run --pr --feature :... [--self-test] [--runner-route PROVIDER:MODEL@EFFORT]... [--wide ] --output `: headless exact-head verification of the named features with machine-checked recipes; no terminal or prompts. The evidence comment lists only the features that ran. - `features/recipes.ts`: per-feature headless cases and assertions. - `bun run test` and `bun run typecheck`: development checks after a separate frozen dependency install, not live proof. -- `features/registry.json`: maintained path ownership; it decides runtime versus no runtime change, and unknown runtime paths block. +- `features/registry.json`: maintained path ownership; it decides which features a PR can reach, and unknown runtime paths block. Read receipts as data, never shell input. Do not put credentials in prompts or public artifacts. @@ -99,3 +103,13 @@ Read receipts as data, never shell input. Do not put credentials in prompts or p - scripts/verify.sh - scripts/verify.test.ts - scripts/verify.ts + +## Subscription account routing + +When the operator uses the harness account router, pass `--account-router /absolute/path/to/account-router` and one explicit `--native-model HARNESS:MODEL@EFFORT` for each selected harness. On Adam's model sheet these are `claude:opus@high` and `codex:gpt-6-astra@xhigh`. The router must have its matching `pstack-verify-run` launcher installed beside it. Missing support stops the run; never retry native sessions outside the router. + +The launcher receives one private request per native case, chooses the account through the router, validates the final isolated configuration against that admitted subscription, and links Codex authentication only for the admitted session. Plugin installation needs no authentication. Each case retains its account and terminal receipts privately; publication contains only their hashes. A failed or uncertain session retains candidate state for inspection. Cancellation asks the router to stop its child and write terminal evidence. + +This route supports ordinary single-session recipes with fixed native permissions. Setup, external runner exercises, and agent preload fail before the first session; they need a separate admitted contract. The poteto-mode invocation exercise resolves a supplied model sheet without dispatch. Formal creation and maintenance use the bounded capacity fixture described in `features/skill-invocation.md`. + +Routed Claude sessions use the launcher's native permission channel for Write/Edit of the formal fixtures' `.claude/skills/verify-capacity/SKILL.md`. The operator must approve that scoped policy before enabling it. Other permission requests are denied and decisions stay in the private terminal receipt. Use native file tools for that skill; a denied edit fails the case. The maintenance assertion still requires the actual skill's sync commit to advance. diff --git a/.claude/skills/verify-open-pstack/features/README.md b/.claude/skills/verify-open-pstack/features/README.md index 9f7d0a28..67b289cb 100644 --- a/.claude/skills/verify-open-pstack/features/README.md +++ b/.claude/skills/verify-open-pstack/features/README.md @@ -1,13 +1,13 @@ # Maintained feature map -The agent running the verifier picks the features that exercise the PR's changed behavior, passes each as `--feature :`, and lists them in the PR body, where review sees them. The run exercises only those features, plus `project-skill` in both harnesses under `--self-test`. A full sweep of every feature the paths touch is never the default. The ownership table below says which surfaces a changed path can affect, to guide that choice; it does not select them. +The ownership table below says which features a changed path reaches. `SKILL.md` says how to pick the named features from that reach and how `verify.sh run` checks them. -`registry.json` is the machine-readable ownership map. Classification decides whether a PR has a runtime change, which requires at least one `--feature`. `core.ts` supports exact paths and terminal `/**` only. Unknown paths fail closed; rename classification includes both paths. Runtime markdown, references, playbooks, hooks, manifests, and scripts are runtime even when their extension looks documentary. +`registry.json` is the machine-readable ownership map. Classification turns the changed paths into the reachable `:` set; an empty set is `no runtime change`. `core.ts` supports exact paths and terminal `/**` only. Unknown paths fail closed; rename classification includes both paths. Runtime markdown, references, playbooks, hooks, manifests, and scripts are runtime even when their extension looks documentary. | Ownership | Surfaces a change can affect | | --- | --- | | `skills` | One `skill-invocation:` exercise per selected skill, in both harnesses | -| `shared` | All listed skills, setup, runner, and shipped tools in both harnesses; `hook:startup` for `plugins/pstack/hooks/**` | +| `shared` | All listed skills, setup, runner, shipped tools, and `hook:startup` in both harnesses, and Claude's `agent-preload:poteto-agent`; a version-only manifest change reaches less, as described below | | `catalog` | The generated model catalog and native agents, and the registry sources: `catalog`, `setup`, and `runner`, and no skill check | | `assets` | Installed UI/resource surface for each actual consumer found in the pinned plugin manifests | | `setup` | Installed setup/model configuration with byte-exact daily Claude file restoration before publication | @@ -16,11 +16,15 @@ The agent running the verifier picks the features that exercise the PR's changed | `project` | Native discovery/invocation of this verifier in both harnesses | | `nonRuntime` | Explicit ordinary repository documentation, policy, non-shipped assets, and unit-test paths; no plugin harness launch | +A change to `plugins/pstack/.claude-plugin/plugin.json`, `.claude-plugin/marketplace.json`, or `plugins/pstack/.codex-plugin/plugin.json` is version-only when the merge base and the head hold the same regular file with the same mode, both sides parse as JSON objects, at least one version string changed (top-level `version`, or `plugins[].version` in the marketplace), and nothing else differs. A version-only change reaches only `hook:startup`: the two Claude files in Claude, the Codex manifest in Codex. `.agents/plugins/marketplace.json` has no version and always takes the shared expansion. + Asset ownership is consumer-driven, not blanket shared ownership. Inspect the exact candidate's plugin manifests. At present, `plugins/pstack/assets/logo.png` is referenced only by `plugins/pstack/.codex-plugin/plugin.json`, so a logo-only change selects the installed Codex asset surface and does not invent a Claude exercise. New shipped assets must be added to `assets` and must have a real manifest consumer; missing, ambiguous, or unknown consumption fails closed. +A registered skill's `agents/openai.yaml` controls Codex invocation and selects that skill's Codex surface. Other metadata filenames remain unmapped. The formal verification creation and maintenance skills each have their own entry in `registry.json`. + `bootstrap.ts` belongs to `tools`; exercise its actual consumers, `orch` and `watch-pr`, including dependency bootstrapping rather than the unrelated external runner. -Catalog paths take precedence over `shared` and over the skill match, so `references/model-catalog.md` and the generated agents select only `catalog`, `setup`, and `runner`. Each generated agent is listed by exact path; a new one is unmapped until it is added to `catalog`. Skill-owned `SKILL.md`, `references/**`, and `playbooks/**` select their skill. Additional executable paths must be registered explicitly. `core.ts` assigns the Codex transcript resolver, `poteto-mode/scripts/resolve-codex-transcript.ts`, and its launcher, `poteto-mode/scripts/resolve-codex-transcript`, to `reflect` by their exact paths, so a change to it selects `skill-invocation:reflect` rather than the unrelated shipped-tool recipe. Shared consumed mappings select every dependent skill conservatively. The coverage test inventories tracked plugin files; new unmapped files block CI and runs. Deleted skill surfaces cannot silently disappear from evidence: choose a surviving documented entry point demonstrating the intended removal, or report blocked. +Catalog paths take precedence over `shared` and over the skill match, so `references/model-catalog.md` and the generated agents select only `catalog`, `setup`, and `runner`. Each generated agent is listed by exact path; a new one is unmapped until it is added to `catalog`. Skill-owned `SKILL.md`, `references/**`, and `playbooks/**` select their skill. poteto-mode's `SKILL.md` also reaches Claude's `agent-preload:poteto-agent`, because poteto-agent preloads it. Additional executable paths must be registered explicitly. `core.ts` assigns the Codex transcript resolver, `poteto-mode/scripts/resolve-codex-transcript.ts`, and its launcher, `poteto-mode/scripts/resolve-codex-transcript`, to `reflect` by their exact paths, so a change to it selects `skill-invocation:reflect` rather than the unrelated shipped-tool recipe. Shared consumed mappings select every dependent skill conservatively. The coverage test inventories tracked plugin files; new unmapped files block CI and runs. Deleted skill surfaces cannot silently disappear from evidence: choose a surviving documented entry point demonstrating the intended removal, or report blocked. The feature documents below define sub-features, user entry points, driving instructions, and gotchas. Review each changed sub-feature in the diff. Record all changed sub-feature actions in the feature's transcript and artifact. Do not run destructive application work in the maintainer checkout. @@ -35,6 +39,6 @@ The feature documents below define sub-features, user entry points, driving inst - `recipes.ts`: the headless cases and machine-checked assertions for every selectable feature. A named feature without one fails the run as `missing-recipe` before any session starts. A recipe proves a lookup or a read with a fixture value the model cannot derive from the prompt, such as a per-run `crypto.randomUUID()` marker, compared in the structured result. A command's text can show which tool the parent called, but it never proves what that tool read or produced, so do not add command-text or tool-output pattern checks for that. - `agent-preload:poteto-agent` (Claude only): the parent dispatches `pstack:poteto-agent` through the Agent tool and asks it for the first Core principle of the preloaded poteto-mode skill without invoking `Skill` or reading files. The check needs that Agent call, `Laziness Protocol` in the reply the parent writes to `result.md`, and no `Skill` call or `Read` of poteto-mode's `SKILL.md` among the child's own tool calls. -Consumed `AGENTS.md`/`CLAUDE.md` instructions and `tests/skill-collision-repro.sh` select `project-skill`. `.mergify.yml` is explicitly `nonRuntime`. There are no blanket no-runtime exemptions for `.github/**`, `tests/**`, or `scripts/**`: unregistered instruction/enforcement paths fail closed until their ownership is reviewed and mapped. +Consumed `AGENTS.md`/`CLAUDE.md` instructions, `tests/skill-collision-repro.sh`, `.github/workflows/ci.yml` and the root `package.json`, whose scripts CI runs, select `project-skill`. `.mergify.yml` is explicitly `nonRuntime`. There are no blanket no-runtime exemptions for `.github/**`, `tests/**`, or `scripts/**`: unregistered instruction/enforcement paths fail closed until their ownership is reviewed and mapped. -Explicit non-runtime repository changes are `no runtime change`. Changes to this verifier are runtime changes and must run from the candidate head. `--self-test` adds `project-skill` in both harnesses, deduplicated with a named one, and is mandatory for this skill's delivery PR. Plugin-runtime proof and local CLI proof remain separate. +Explicit non-runtime repository changes are `no runtime change`. Plugin-runtime proof and local CLI proof remain separate. diff --git a/.claude/skills/verify-open-pstack/features/catalog.md b/.claude/skills/verify-open-pstack/features/catalog.md index 02045f95..eb52e3ed 100644 --- a/.claude/skills/verify-open-pstack/features/catalog.md +++ b/.claude/skills/verify-open-pstack/features/catalog.md @@ -10,7 +10,7 @@ A parent reads the catalog through `provider-dispatch.md` whenever a configured ## Driving it with verify-open-pstack -Give the parent a fixture model sheet, not the operator's configured one, so this feature changes no real configuration. Ask it to report each role's descriptors and routes as JSON. Compute the expected answer from the installed candidate's catalog and the fixture sheet, never from a copy in the verifier. +Give the parent a fixture model sheet, not the operator's configured one, so this feature changes no real configuration. Ask it to report each role's descriptors and routes as JSON. Compute the expected role defaults and routes from the installed candidate's catalog and the fixture sheet, never from a copy in the verifier. ## Gotchas @@ -18,4 +18,6 @@ The outcome is the evidence: the reported descriptors and routes must equal the ## Headless -The catalog recipe in `recipes.ts` writes a fixture sheet whose `judgment and prose` line takes another role's default descriptor, so only a read of the sheet reports it. The sheet has no `arena runners` or `bug-fix` line. The parent writes `routes.json` with the descriptors and routes of those three roles. The case asserts the `poteto-mode` load and that the report equals, role by role, the sheet's descriptor or the installed catalog's role default, each with the route the catalog's route table gives this parent. +The catalog recipe in `recipes.ts` writes a fixture sheet whose `judgment and prose` line takes another role's default descriptor, so only a read of the sheet reports it. The sheet has no `arena runners` or `bug-fix` line. It puts `swarm workers` on `codex:gpt-6-luna@max`, `hardest tasks` on the registered pin `claude:claude-opus-5-5@max`, `how explainer` on the versioned `claude:claude-sonnet-5-5@high`, and `how explorer` on the Devin lane `devin:swe-2@max`. The parent writes `routes.json` with the descriptors and routes of those seven roles. The case asserts the `poteto-mode` load and that the report equals, role by role, the sheet's descriptor or the installed catalog's role default, each with the route the catalog's route table gives this parent. The pin, Luna and the Devin lane stay unchanged, the Devin lane is external from both parents, and the Sonnet revision is reported as `claude:sonnet@high`. The recipe fixes those dispatched descriptors, and the catalog gives their routes. The sheet also puts `feature, refactoring` on `codex:gpt-6.1-sol@max` and holds the setting line `codex writer sandbox: full-access`. The report adds that role and `writerSandbox`, the `--writer-sandbox` value the parent would pass for that role's writer: `full-access` from Claude, whose route is the external runner, and null from Codex, whose route is native `spawn_agent`. A second case, `consume-sheet-bounded`, writes the same fixture sheet with the setting line `codex writer sandbox: bounded`. The runner's `bounded-writer` case covers a sheet with no setting line. It asserts the same roles, descriptors and routes, and `writerSandbox` null from both parents, so a parent that passes full access without the opt-in fails. + +On Claude, a second case, `native-agents`, has the parent dispatch `pstack:pstack-sonnet-max` and `pstack:pstack-opus-5-5-max` through `Agent`. Each child reads its own fixture nonce and writes it to its own output file. The `Agent` call returns only a launch notice, so the case asserts each dispatch, at least one tool call recorded under it, and that the output file holds that agent's nonce. Codex has no Claude-native agents, and its session record does not show `spawn_agent`, so this case runs on Claude only. diff --git a/.claude/skills/verify-open-pstack/features/formal.ts b/.claude/skills/verify-open-pstack/features/formal.ts new file mode 100644 index 00000000..36c095c9 --- /dev/null +++ b/.claude/skills/verify-open-pstack/features/formal.ts @@ -0,0 +1,127 @@ +import { mkdir, readFile, writeFile } from 'node:fs/promises'; +import { join } from 'node:path'; +import { sha256 } from '../scripts/io.ts'; +import type { Case, Fixture } from './recipes.ts'; +import type { SessionRecord } from '../scripts/types.ts'; + +const ATOMIC = 'export async function reserve(state) {\n if (state.used >= state.limit) return false;\n state.used++;\n return true;\n}\n'; +const SPLIT = 'export async function reserve(state) {\n const allowed = state.used < state.limit;\n await Promise.resolve();\n if (!allowed) return false;\n state.used++;\n return true;\n}\n'; +// The checker exhausts all schedules of two one-shot clients at capacity one. +export const CHECKER = `import { readFileSync } from 'node:fs'; +const model = JSON.parse(readFileSync('verification/model.json', 'utf8')); +if (model.clients !== 2 || model.limit !== 1 || typeof model.splitReservation !== 'boolean') throw new Error('unsupported bounds'); +const queue = [{ used:0, pc:[0,0], allowed:[false,false], trace:[] }], seen = new Set(); +while(queue.length) { + const s = queue.shift(); + const key = JSON.stringify([s.used,s.pc,s.allowed]); + if(seen.has(key)) continue; seen.add(key); + if(s.used > model.limit) { console.log(JSON.stringify({verdict:'bug',states:seen.size,counterexample:s.trace,used:s.used})); process.exit(1); } + for(let i=0;i { + await mkdir(join(f.dir, path, '..'), { recursive:true }); + await writeFile(join(f.dir,path),text); +} +async function git(f: Fixture,...args:string[]):Promise { + return f.run(['git','-c','user.name=Fixture','-c','user.email=fixture@example.invalid','-c','commit.gpgsign=false',...args],{cwd:f.dir}); +} +async function commit(f:Fixture, message:string, paths:string[]):Promise { + await git(f,'add','--',...paths); await git(f,'commit','-qm',message); return (await git(f,'rev-parse','HEAD')).trim(); +} +function request(f:Fixture, skill:string, task:string):string { + const invoke=f.harness==='claude'?`/pstack:${skill}`:`$pstack:${skill}`; + return `${invoke} ${task} Work only in ${f.dir}. This is one local verification exercise. Do not launch other model sessions or subagents, contact GitHub, or change product code. Use native ${f.harness==='claude'?'Write/Edit':'file-editing tools'} to create or update the project SKILL.md; if permission is denied, report the denial and stop that edit. Write your report to result.md and complete all work before your final reply.`; +} +function loaded(record:SessionRecord,skill:string):void { + if(!record.skills.includes(`pstack:${skill}`)) throw new Error(`skill-not-loaded:${skill}`); +} +function observed(record:SessionRecord, verdict:string):boolean { + return record.tools.some(t=>!t.parent && JSON.stringify(t.input).includes('verification/check.mjs') && t.result?.includes(`"verdict":"${verdict}"`)); +} + +export function maintainFormal():Case { + return { + id:'stale-model', + async prepare(f) { + await git(f,'init','-q','-b','main'); + await put(f,'capacity.mjs',ATOMIC);await put(f,'replay.mjs',REPLAY); + const sync=await commit(f,'Reserve capacity atomically',['capacity.mjs','replay.mjs']); + await put(f,'verification/check.mjs',CHECKER); + await put(f,'verification/model.json',JSON.stringify({clients:2,limit:1,splitReservation:false})+'\n'); + await put(f,'.claude/skills/verify-capacity/SKILL.md',`---\nname: verify-capacity\ndescription: Model-check reserve capacity after changes to capacity.mjs.\n---\n# Verify capacity\n\nProperty: used never exceeds limit. Oversubscription assigns a slot twice.\nCoverage: capacity.mjs reserve, all writes to state.used.\nSync commit: ${sync}\nRun \`bun verification/check.mjs\`. The model describes one atomic check-and-increment when splitReservation is false, and separate check and commit steps when true. The interpreter explores every schedule of two one-shot clients at capacity one, with no releases or crashes. Model state corresponds to used, the allowed decision, and reserve's position before or after its await.\nCompare covered code with the sync commit before trusting a pass. Update the model from the code, keep the capacity property and bounds, then rerun. Replay any failing schedule with \`bun replay.mjs\`. Report a real bug without editing product code. Advance sync only after checking the model against the current commit.\n`); + await commit(f,'Record bounded capacity verification',['verification/check.mjs','verification/model.json','.claude/skills/verify-capacity/SKILL.md']); + await put(f,'capacity.mjs',SPLIT); + const head=await commit(f,'Yield during capacity reservation',['capacity.mjs']); + await writeFile(`${f.dir}.formal-baseline.json`,JSON.stringify({head,product:sha256(SPLIT),checker:sha256(CHECKER)})); + }, + prompt:f=>request(f,'maintain-formal-verification-skill','Maintain .claude/skills/verify-capacity/SKILL.md. A change to reserve landed after its sync commit. The existing checker is an exhaustive bounded-state interpreter; keep verification/check.mjs and its two-client, capacity-one bounds unchanged. Update the model from the current code, run the check before and after, replay any counterexample with bun replay.mjs, and report whether the old pass was stale and whether the current code has a bug.'), + async assert(record,f) { + loaded(record,'maintain-formal-verification-skill'); + const baseline=JSON.parse(await readFile(`${f.dir}.formal-baseline.json`,'utf8')); + if(sha256(await readFile(join(f.dir,'capacity.mjs'))) !== baseline.product) throw new Error('formal-product-changed'); + if(sha256(await readFile(join(f.dir,'verification/check.mjs'))) !== baseline.checker) throw new Error('formal-checker-weakened'); + const model=JSON.parse(await readFile(join(f.dir,'verification/model.json'),'utf8')); + if(!Bun.deepEquals(model,{clients:2,limit:1,splitReservation:true})) throw new Error('formal-model-still-stale-or-bounds-changed'); + if(!observed(record,'pass') || !observed(record,'bug')) throw new Error('formal-before-after-not-observed'); + const replay=record.tools.some(t=>JSON.stringify(t.input).includes('replay.mjs') && t.result?.includes('"used":2')); + if(!replay) throw new Error('formal-counterexample-not-replayed'); + const skill=await readFile(join(f.dir,'.claude/skills/verify-capacity/SKILL.md'),'utf8'); + if(!skill.includes(baseline.head)) throw new Error('formal-sync-commit-not-updated'); + const result=await readFile(join(f.dir,'result.md'),'utf8'); + if(!/bug|oversubscri/i.test(result) || !/stale|drift/i.test(result)) throw new Error('formal-finding-not-reported'); + // The captain independently runs the unmodified checker on the maintained model. + const output=await f.run(['bun','-e',"const p=Bun.spawnSync(['bun','verification/check.mjs']);process.stdout.write(p.stdout);if(p.exitCode!==1)process.exit(2)"],{cwd:f.dir}); + if(!output.includes('"verdict":"bug"')) throw new Error('formal-counterexample-missing'); + return {assertions:['skill-loaded','stale-pass-observed','model-updated','bounds-preserved','counterexample-replayed','product-unchanged','sync-updated'],files:['result.md','verification/model.json','.claude/skills/verify-capacity/SKILL.md'].map(p=>join(f.dir,p))}; + }, + }; +} + +export function createFormal():Case { + return { + id:'create-and-mutate', + async prepare(f) { + await git(f,'init','-q','-b','main'); + await put(f,'capacity.mjs',ATOMIC);await put(f,'replay.mjs',REPLAY); + await put(f,'verification/check.mjs',CHECKER); + await put(f,'verification/model.json',JSON.stringify({clients:2,limit:1,splitReservation:false})+'\n'); + await put(f,'README.md','# Capacity reservation\n\nPromise: used never exceeds limit. Each reserve caller can acquire at most one slot. The bundled dependency-free bounded model checker enumerates every schedule of two one-shot callers at capacity one. Its splitReservation switch must match whether capacity.mjs checks and increments atomically or yields between the check and increment. The checker is available infrastructure; this repository has no project verification skill.\n'); + const head=await commit(f,'Atomic capacity module and bounded verification tool',['capacity.mjs','replay.mjs','verification/check.mjs','verification/model.json','README.md']); + await writeFile(`${f.dir}.formal-baseline.json`,JSON.stringify({head,product:sha256(ATOMIC),checker:sha256(CHECKER)})); + }, + prompt:f=>request(f,'create-formal-verification-skill',`Create .claude/skills/verify-capacity/SKILL.md for reserve's capacity invariant. Use the bundled exhaustive bounded model checker (bun verification/check.mjs); keep its interpreter and two-client, capacity-one scope unchanged. Confirm the model matches capacity.mjs. The only authorized product edit is the skill's disposable mutation test: split reserve's check and increment with an await, reflect that split in the model, observe the failure and replay it, then restore both code and model. Record the checked commit, coverage, assumptions, bounds, command and how to distinguish model drift from a bug. Follow the generated skill and report the evidence. The scratch mutation is explicitly permitted by this fixture.`).replace('or change product code.', 'or change product code except for the explicitly permitted scratch mutation.'), + async assert(record,f) { + loaded(record,'create-formal-verification-skill'); + const baseline=JSON.parse(await readFile(`${f.dir}.formal-baseline.json`,'utf8')); + if(sha256(await readFile(join(f.dir,'capacity.mjs'))) !== baseline.product) throw new Error('formal-mutation-not-restored'); + if(sha256(await readFile(join(f.dir,'verification/check.mjs'))) !== baseline.checker) throw new Error('formal-checker-weakened'); + const model=JSON.parse(await readFile(join(f.dir,'verification/model.json'),'utf8')); + if(!Bun.deepEquals(model,{clients:2,limit:1,splitReservation:false})) throw new Error('formal-model-not-restored'); + if(!observed(record,'pass') || !observed(record,'bug')) throw new Error('formal-mutation-not-observed'); + if(!record.tools.some(t=>JSON.stringify(t.input).includes('replay.mjs') && t.result?.includes('"used":2'))) throw new Error('formal-mutation-not-replayed'); + const path=join(f.dir,'.claude/skills/verify-capacity/SKILL.md'), skill=await readFile(path,'utf8'); + if(!/^---\n[\s\S]*name:\s*verify-capacity[\s\S]*description:/m.test(skill) || !skill.includes(baseline.head) || !skill.includes('capacity.mjs') || !skill.includes('verification/check.mjs') || !/bound|two|2/i.test(skill)) throw new Error('formal-skill-contract-incomplete'); + const output=await f.run(['bun','verification/check.mjs'],{cwd:f.dir}); + if(!output.includes('"verdict":"pass"')) throw new Error('formal-final-check-failed'); + return {assertions:['skill-loaded','project-skill-created','coverage-and-sync-recorded','mutation-caught','mutation-replayed','code-and-model-restored','bounded-check-passes'],files:[path,join(f.dir,'result.md'),join(f.dir,'verification/model.json')]}; + }, + }; +} diff --git a/.claude/skills/verify-open-pstack/features/recipes.ts b/.claude/skills/verify-open-pstack/features/recipes.ts index 1df70da7..d4749fbb 100644 --- a/.claude/skills/verify-open-pstack/features/recipes.ts +++ b/.claude/skills/verify-open-pstack/features/recipes.ts @@ -1,8 +1,11 @@ // Headless recipes for every selectable feature. The trusted parent runs each case as one fresh // native session and evaluates `assert` against the parsed record and the fixture it created. +import { createFormal, maintainFormal } from './formal.ts'; import { Database } from 'bun:sqlite'; import { lstat, mkdir, readFile, realpath, rm, writeFile } from 'node:fs/promises'; +import { homedir } from 'node:os'; import { basename, join } from 'node:path'; +import { pathToFileURL } from 'node:url'; import { sha256, type Command } from '../scripts/io.ts'; import type { Harness, SessionRecord } from '../scripts/types.ts'; @@ -35,10 +38,16 @@ export interface Case { export const CLAUDE_TOOLS = 'Skill,Read,Glob,Grep,Write,Edit,Bash,Agent,Task,TodoWrite'; /** Without --runner-route, the runner recipe dispatches the first lane of this role's default in the installed catalog. */ export const DEFAULT_ROLES: Record = { claude: 'bug-fix', codex: 'judgment and prose' }; -export const ROUTE = /^(claude|codex|grok):([A-Za-z0-9._-]+)@([a-z]+)$/; +/** Any provider's route; the installed candidate's catalog decides whether the provider exists and is external. */ +export const ROUTE = /^([a-z][a-z0-9-]*):([A-Za-z0-9._/-]+)@([a-z]+)$/; const SETUP_DESCRIPTOR = 'claude:opus@high'; -const DESCRIPTOR = /\b(?:claude|codex|grok):[A-Za-z0-9._-]+@[a-z]+/g; +const DESCRIPTOR = /\b[a-z][a-z0-9-]*:[A-Za-z0-9._/-]+@[a-z]+/g; const SHEET_HEADER = '# pstack model configuration\n\n'; +/** The model sheet's one setting line. It is not a role, so role-line comparisons leave it out. */ +const SETTING = 'codex writer sandbox: '; +const FULL_ACCESS_SETTING = `${SETTING}full-access`, BOUNDED_SETTING = `${SETTING}bounded`; +/** The writer role and lane that the opt-in reaches on a Claude parent's external runner. */ +const WRITER_ROLE = 'feature, refactoring', WRITER_LANE = 'codex:gpt-6.1-sol@max'; /** The installed candidate's registry, as its generated model catalog renders it. */ export interface Catalog { @@ -70,6 +79,11 @@ export async function readCatalog(location: string): Promise { } const sheetText = (lines: [string, string[]][]): string => SHEET_HEADER + lines.map(([role, lanes]) => `${role}: ${lanes.join(', ')}`).join('\n') + '\n'; +const roleLines = (text: string): string[] => text.split('\n').filter(line => /^[a-z][^:]*: \S/.test(line) && !line.startsWith(SETTING)); +/** Setup writes exactly one setting line after it asks: bounded on the default answer, full-access on a yes. */ +const settingLines = (text: string): string[] => text.split('\n').filter(line => line.startsWith(SETTING)); +/** One literal POSIX sh argument. */ +const sh = (text: string): string => `'${text.replaceAll("'", `'\\''`)}'`; const rules = (dir: string): string => `Work only inside ${dir}. Do not push, post, open, merge, or modify pull requests, issues, or remote branches, and do not write outside ${dir}. Do not edit any skill, plugin, or configuration file outside ${dir}. Do not ask questions: where the skill would ask, choose its documented default and continue. This is a headless session that ends at your first final reply and stops any background work still running, so run every command and subagent in the foreground, even where the skill says to use the background, and finish all work before you reply.`; @@ -297,6 +311,60 @@ const writer: Case = { }, }; +const CHROME = '/Applications/Google Chrome.app/Contents/MacOS/Google Chrome'; +/** Chrome, else a Playwright Chromium. Neither on the host leaves Chrome's path, and the child's launch fails. */ +async function browser(): Promise { + if (await lstat(CHROME).catch(() => undefined)) return CHROME; + const cache = join(homedir(), 'Library/Caches/ms-playwright'); + try { + for await (const path of new Bun.Glob('chromium-*/*/*.app/Contents/MacOS/*').scan({ cwd: cache })) return join(cache, path); + } catch { /* No Playwright cache. */ } + return CHROME; +} + +// A Codex writer from a Claude parent, opted in or bounded. The prompt names the fixture sheet and the role, never the +// runner flag, so the receipt's sandbox shows the parent applied the sheet. Opted in, the child writes one nonce under +// `.agents/` and reads the other from a page only a browser renders; the bounded sandbox blocks both on macOS. Bounded, +// the sheet has no setting line and the child writes its nonce to a plain file, which workspace-write allows. +const BOUNDED_WRITER_LANE = 'codex:gpt-6.1-sol@low'; +function codexWriter(optIn: boolean): Case { + const name = optIn ? 'full-access' : 'bounded', descriptor = optIn ? WRITER_LANE : BOUNDED_WRITER_LANE; + const [sandbox, argvSandbox] = optIn ? ['full-access', 'danger-full-access'] : ['bounded', 'workspace-write']; + const at = (f: Fixture, file: string) => join(f.dir, `${name}.${file}`), lane = (f: Fixture) => join(f.dir, `${name}-lane`); + const result = (f: Fixture) => join(lane(f), optIn ? '.agents/result.json' : 'result.json'); + let fileNonce = '', pageNonce = ''; + return { + id: `${name}-writer`, + async prepare(f) { + const installed = await readCatalog(f.location); + if (installed.route('claude', descriptor) !== 'external') throw new Error(`runner-route-not-external:${descriptor}`); + fileNonce = crypto.randomUUID(); pageNonce = crypto.randomUUID(); + await mkdir(lane(f)); + await writeFile(at(f, 'sheet.md'), sheetText([...installed.defaults].map(([role, lanes]) => [role, role === WRITER_ROLE ? [descriptor] : lanes])) + (optIn ? `${FULL_ACCESS_SETTING}\n` : '')); + if (!optIn) { + await writeFile(at(f, 'prompt.md'), `Create the file result.json in the current directory containing only the JSON {"written":"${fileNonce}"}, and reply DONE.\n`); + return; + } + // The file holds the nonce only base64-encoded; the DOM a browser dumps holds it decoded. + await writeFile(at(f, 'page.html'), `

\n`); + await writeFile(at(f, 'prompt.md'), `Run this command and read the text of the element with id "nonce" in the DOM it prints: ${sh(await browser())} --headless --no-first-run ${sh(`--user-data-dir=${at(f, 'browser-profile')}`)} --dump-dom ${sh(pathToFileURL(at(f, 'page.html')).href)}\nThen create the file .agents/result.json in the current directory containing only the JSON {"written":"${fileNonce}","browser":""}, and reply DONE.\n`); + }, + prompt: f => `${invoke('claude', 'poteto-mode', `Do no engineering work. For this task the pstack model sheet is ${at(f, 'sheet.md')}; use it in place of any configured sheet. Dispatch exactly one lane and nothing else: the first descriptor of its "${WRITER_ROLE}" role as an isolated-write writer, on the route and with the launcher arguments provider dispatch gives this parent. Use the prompt file ${at(f, 'prompt.md')}, the dedicated directory ${lane(f)} as its cwd, the output ${at(f, 'output.md')} and the receipt ${at(f, 'receipt.json')}, and wait for the lane to exit.`)} ${rules(f.dir)}`, + async assert(record, f) { + if (!record.commands.some(c => c.includes('pstack-runner'))) throw new Error('runner-not-run'); + const receipt = JSON.parse(await readFile(at(f, 'receipt.json'), 'utf8')), [, provider, model, effort] = ROUTE.exec(descriptor)!; + if (receipt.status !== 'complete') throw new Error(`receipt-status:${receipt.status}`); + if (receipt.parent !== 'claude' || receipt.provider !== provider || receipt.model !== model || receipt.effort !== effort || receipt.mode !== 'isolated-write' || receipt.cwd !== lane(f)) throw new Error('receipt-route-mismatch'); + if (receipt.outputPath !== at(f, 'output.md') || !(Date.parse(receipt.startedAt) >= record.started)) throw new Error('receipt-not-this-session'); + const flag = receipt.argv.indexOf('--sandbox'); + if (receipt.writerSandbox !== sandbox || receipt.argv[flag + 1] !== argvSandbox || (!optIn && receipt.argv.includes('danger-full-access'))) throw new Error(`writer-sandbox:${receipt.writerSandbox}`); + const effect = await written(record, result(f)); + if (!Bun.deepEquals(parseJson(await readFile(result(f), 'utf8')), optIn ? { written: fileNonce, browser: pageNonce } : { written: fileNonce })) throw new Error(`${name}-effect-mismatch`); + return { assertions: ['runner-ran', 'receipt-complete', `writer-sandbox:${sandbox}`, effect, ...(optIn ? ['browser-nonce'] : [])], files: [at(f, 'receipt.json'), result(f)] }; + }, + }; +} + function runner(harness: Harness, routes: string[]): Case[] { const chosen = routes.filter(route => !route.startsWith(`${harness}:`)); const cases = (chosen.length ? chosen : ['']).map((fixed, index): Case => { @@ -306,9 +374,10 @@ function runner(harness: Harness, routes: string[]): Case[] { id: `route-${index + 1}`, codexSandbox: 'danger-full-access', async prepare(f) { - route ||= (await readCatalog(f.location)).defaults.get(DEFAULT_ROLES[harness])?.[0] ?? ''; + const installed = await readCatalog(f.location); + route ||= installed.defaults.get(DEFAULT_ROLES[harness])?.[0] ?? ''; const parts = ROUTE.exec(route); - if (!parts || parts[1] === harness) throw new Error(`runner-route-not-external:${route || DEFAULT_ROLES[harness]}`); + if (!parts || installed.route(harness, route) !== 'external') throw new Error(`runner-route-not-external:${route || DEFAULT_ROLES[harness]}`); [, provider = '', model = '', effort = ''] = parts; await writeFile(lane(f, 'prompt.md'), 'Reply with the single word PONG.\n'); }, @@ -324,19 +393,25 @@ function runner(harness: Harness, routes: string[]): Case[] { }, }; }); - return harness === 'codex' ? [...cases, writer] : cases; + return harness === 'codex' ? [...cases, writer] : [...cases, codexWriter(true), codexWriter(false)]; } const setup: Case[] = [{ id: 'configure', codexSandbox: 'danger-full-access', claudePermission: 'bypassPermissions', - prompt: f => `${invoke(f.harness, 'setup-pstack', `Use these answers and ask nothing else. Role assignments: change every role assigned to a model-matrix family to the Opus family, and keep inherit-parent and auto roles unchanged. Opus requested effort: high. I confirm every write. Write the setup report to ${join(f.dir, 'result.md')}.`)} Do not push, post, or contact GitHub.`, + async prepare(f) { + // Setup keeps an existing setting line without asking, so drop it and keep every other byte; setup then asks. + const sheet = join(f.configHome, 'pstack-models.md'), text = await readFile(sheet, 'utf8').catch(() => undefined); + if (text !== undefined) await writeFile(sheet, text.split('\n').filter(line => !line.startsWith(SETTING)).join('\n')); + }, + prompt: f => `${invoke(f.harness, 'setup-pstack', `Use these answers and ask nothing else. Role assignments: change every role assigned to a model-matrix family to the Opus family, and keep inherit-parent and auto roles unchanged. Opus requested effort: high. When setup asks whether Codex writer lanes should run with full access, answer yes. I confirm every write. Write the setup report to ${join(f.dir, 'result.md')}.`)} Do not push, post, or contact GitHub.`, async assert(record, f) { const sheet = join(f.configHome, 'pstack-models.md'), assertions = [loaded(record, 'pstack:setup-pstack'), await written(record, sheet)]; - const descriptors = (await readFile(sheet, 'utf8')).match(DESCRIPTOR) ?? []; + const text = await readFile(sheet, 'utf8'), descriptors = text.match(DESCRIPTOR) ?? []; if (!descriptors.length || descriptors.some(d => d !== SETUP_DESCRIPTOR)) throw new Error('sheet-descriptors-mismatch'); - return { assertions: [...assertions, `sheet:${SETUP_DESCRIPTOR}`], files: [sheet] }; + if (!Bun.deepEquals(settingLines(text), [FULL_ACCESS_SETTING])) throw new Error('setting-not-full-access'); + return { assertions: [...assertions, `sheet:${SETUP_DESCRIPTOR}`, `setting:${FULL_ACCESS_SETTING}`], files: [sheet] }; }, }, { id: 'consume', @@ -346,7 +421,7 @@ const setup: Case[] = [{ if (!(await readFile(path, 'utf8')).includes(SETUP_DESCRIPTOR)) throw new Error('consumer-route-mismatch'); return { assertions: [...assertions, 'consumer-route'], files: [path] }; }, -}, keptPredecessor(), { +}, keptPredecessor(), newFamilies(), readOnlyProvider(), writerRefusal(), keepSetting(), { id: 'first-run', codexSandbox: 'danger-full-access', claudePermission: 'bypassPermissions', @@ -358,12 +433,32 @@ const setup: Case[] = [{ prompt: f => `${invoke(f.harness, 'setup-pstack', `Use these answers and ask nothing else. Keep the first-run role assignments and every proposed effort. I confirm every write. Write the setup report to ${join(f.dir, 'result.md')}.`)} Do not push, post, or contact GitHub.`, async assert(record, f) { const sheet = join(f.configHome, 'pstack-models.md'), assertions = [loaded(record, 'pstack:setup-pstack'), await written(record, sheet)]; - const lines = (await readFile(sheet, 'utf8')).split('\n').filter(line => /^[a-z][^:]*: \S/.test(line)); - if (!Bun.deepEquals(lines, [...(await readCatalog(f.location)).defaults].map(([role, lanes]) => `${role}: ${lanes.join(', ')}`))) throw new Error('first-run-not-role-defaults'); - return { assertions: [...assertions, 'sheet:role-defaults'], files: [sheet] }; + const text = await readFile(sheet, 'utf8'); + if (!Bun.deepEquals(roleLines(text), [...(await readCatalog(f.location)).defaults].map(([role, lanes]) => `${role}: ${lanes.join(', ')}`))) throw new Error('first-run-not-role-defaults'); + bounded(text); + return { assertions: [...assertions, ...await mirrored(f, sheet), 'sheet:role-defaults', `setting:${BOUNDED_SETTING}`], files: [sheet] }; }, }]; +// A case that takes the default answer, on a sheet without a setting line, gets the bounded setting. +function bounded(text: string): void { + if (!Bun.deepEquals(settingLines(text), [BOUNDED_SETTING])) throw new Error('setting-not-bounded'); +} + +// On Codex, setup's parent integration is one bounded AGENTS.md block: the begin marker and a newline, the sheet's +// exact bytes, then the end marker. A case that removes the block before the run checks that setup wrote it back. +const MIRROR_BEGIN = '', MIRROR_END = ''; +export async function mirrored(f: Fixture, sheet: string): Promise { + if (f.harness !== 'codex') return []; + const agents = await readFile(join(f.configHome, 'AGENTS.md'), 'utf8').catch(() => ''); + const once = (marker: string) => agents.split(marker).length === 2; + const from = agents.indexOf(MIRROR_BEGIN) + MIRROR_BEGIN.length, to = agents.indexOf(MIRROR_END); + if (!once(MIRROR_BEGIN) || !once(MIRROR_END) || to < from || agents.slice(from, to) !== `\n${await readFile(sheet, 'utf8')}`) { + throw new Error('codex-mirror-mismatch'); + } + return ['codex-mirror']; +} + // Releases before v1.5.0 wrote this pair; setup must offer its family's current model and keep it on a decline. const KEPT = 'grok:grok-4.6', KEPT_EFFORT = 'high'; function keptPredecessor(): Case { @@ -389,10 +484,135 @@ function keptPredecessor(): Case { prompt: f => `${invoke(f.harness, 'setup-pstack', `Use these answers and ask nothing else. Keep every role assignment and the requested effort ${KEPT_EFFORT}. When setup proposes replacing ${KEPT}, decline and keep it. I confirm every write. Write to ${proposal(f)} only JSON of the form {"kept":"","proposed":""} naming the kept descriptor and the replacement setup proposed. Write the setup report to ${join(f.dir, 'result.md')}.`)} Do not push, post, or contact GitHub.`, async assert(record, f) { const sheet = join(f.configHome, 'pstack-models.md'), assertions = [loaded(record, 'pstack:setup-pstack'), await written(record, sheet), await written(record, proposal(f))]; - const lines = (await readFile(sheet, 'utf8')).split('\n').filter(line => /^[a-z][^:]*: \S/.test(line)).sort(); - if (!Bun.deepEquals(lines, expected)) throw new Error('kept-sheet-mismatch'); + const text = await readFile(sheet, 'utf8'); + if (!Bun.deepEquals(roleLines(text).sort(), expected)) throw new Error('kept-sheet-mismatch'); + bounded(text); if (!Bun.deepEquals(parseJson(await readFile(proposal(f), 'utf8')), { kept: `${KEPT}@${KEPT_EFFORT}`, proposed })) throw new Error('proposal-mismatch'); - return { assertions: [...assertions, `kept:${KEPT}@${KEPT_EFFORT}`, `proposed:${proposed}`], files: [sheet, proposal(f)] }; + return { assertions: [...assertions, ...await mirrored(f, sheet), `kept:${KEPT}@${KEPT_EFFORT}`, `proposed:${proposed}`, `setting:${BOUNDED_SETTING}`], files: [sheet, proposal(f)] }; + }, + }; +} + +// A rerun on a sheet that assigns Luna, a versioned Sonnet, the registered Opus 5.5 pin and an older Opus revision. Setup +// keeps Luna and the pin, and writes each revision as its rolling alias. +const NEW_FAMILY_LANES: Record = { + 'swarm workers': ['codex:gpt-6-luna@max', 'codex:gpt-6-luna@max'], + 'hardest tasks': ['claude:claude-opus-5-5@max', 'claude:claude-opus-5-5@max'], + 'how explainer': ['claude:claude-sonnet-5-5@max', 'claude:sonnet@max'], + 'judgment and prose': ['claude:claude-opus-5@max', 'claude:opus@max'], +}; +function newFamilies(): Case { + let expected: string[] = []; + return { + id: 'new-families', + codexSandbox: 'danger-full-access', + claudePermission: 'bypassPermissions', + async prepare(f) { + const roles = [...(await readCatalog(f.location)).defaults]; + const lane = (role: string, index: 0 | 1, lanes: string[]) => role in NEW_FAMILY_LANES ? [NEW_FAMILY_LANES[role]![index]] : lanes; + for (const role of Object.keys(NEW_FAMILY_LANES)) if (!roles.some(([name]) => name === role)) throw new Error(`catalog-role-missing:${role}`); + expected = roles.map(([role, lanes]) => `${role}: ${lane(role, 1, lanes).join(', ')}`).sort(); + await writeFile(join(f.configHome, 'pstack-models.md'), sheetText(roles.map(([role, lanes]) => [role, lane(role, 0, lanes)]))); + // Codex mirrors the sheet in a run-owned AGENTS.md block; without it the sheet is the only source. + if (f.harness === 'codex') await rm(join(f.configHome, 'AGENTS.md'), { force: true }); + }, + prompt: f => `${invoke(f.harness, 'setup-pstack', `Use these answers and ask nothing else. Keep every role assignment and every family's current effort. I confirm every write. Write the setup report to ${join(f.dir, 'result.md')}.`)} Do not push, post, or contact GitHub.`, + async assert(record, f) { + const sheet = join(f.configHome, 'pstack-models.md'), assertions = [loaded(record, 'pstack:setup-pstack'), await written(record, sheet)]; + const text = await readFile(sheet, 'utf8'); + if (!Bun.deepEquals(roleLines(text).sort(), expected)) throw new Error('new-families-sheet-mismatch'); + bounded(text); + return { + assertions: [...assertions, ...await mirrored(f, sheet), ...Object.values(NEW_FAMILY_LANES).map(([, written]) => `sheet:${written}`), `setting:${BOUNDED_SETTING}`], + files: [sheet], + }; + }, + }; +} + +// A rerun on a sheet that assigns the read-only role `how explorer` to a read-only-only provider. Setup probes it +// through the runner and keeps every role line. +const READ_ONLY_ROLE = 'how explorer', READ_ONLY_LANE = 'devin:swe-2@max'; +function readOnlyProvider(): Case { + let expected: string[] = []; + return { + id: 'read-only-provider', + codexSandbox: 'danger-full-access', + claudePermission: 'bypassPermissions', + async prepare(f) { + const installed = await readCatalog(f.location), roles = [...installed.defaults]; + if (!roles.some(([role]) => role === READ_ONLY_ROLE)) throw new Error(`catalog-role-missing:${READ_ONLY_ROLE}`); + if (installed.route(f.harness, READ_ONLY_LANE) !== 'external') throw new Error(`catalog-route-not-external:${READ_ONLY_LANE}`); + const lines = roles.map(([role, lanes]): [string, string[]] => [role, role === READ_ONLY_ROLE ? [READ_ONLY_LANE] : lanes]); + expected = lines.map(([role, lanes]) => `${role}: ${lanes.join(', ')}`).sort(); + await writeFile(join(f.configHome, 'pstack-models.md'), sheetText(lines)); + // Codex mirrors the sheet in a run-owned AGENTS.md block; without it the sheet is the only source. + if (f.harness === 'codex') await rm(join(f.configHome, 'AGENTS.md'), { force: true }); + }, + prompt: f => `${invoke(f.harness, 'setup-pstack', `Use these answers and ask nothing else. Keep every role assignment and every family's current effort. I confirm every write. Write the setup report to ${join(f.dir, 'result.md')}.`)} Do not push, post, or contact GitHub.`, + async assert(record, f) { + const sheet = join(f.configHome, 'pstack-models.md'), assertions = [loaded(record, 'pstack:setup-pstack'), await written(record, sheet)]; + const text = await readFile(sheet, 'utf8'); + if (!Bun.deepEquals(roleLines(text).sort(), expected)) throw new Error('read-only-provider-sheet-mismatch'); + bounded(text); + return { assertions: [...assertions, ...await mirrored(f, sheet), `sheet:${READ_ONLY_LANE}`, `setting:${BOUNDED_SETTING}`], files: [sheet] }; + }, + }; +} + +// Setup refuses a read-only-only provider for a writer role and changes no configuration: the sheet keeps its bytes, +// Codex's removed mirror is not written back, and setup's refusal names the role and the provider. +const REFUSED_ROLE = 'bug-fix'; +function writerRefusal(): Case { + let prepared = ''; + const refusal = (f: Fixture) => join(f.dir, 'refusal.txt'); + return { + id: 'writer-refusal', + codexSandbox: 'danger-full-access', + claudePermission: 'bypassPermissions', + async prepare(f) { + const roles = [...(await readCatalog(f.location)).defaults]; + if (!roles.some(([role]) => role === REFUSED_ROLE)) throw new Error(`catalog-role-missing:${REFUSED_ROLE}`); + prepared = sheetText(roles.map(([role, lanes]) => [role, role === REFUSED_ROLE ? [READ_ONLY_LANE] : lanes])); + await writeFile(join(f.configHome, 'pstack-models.md'), prepared); + if (f.harness === 'codex') await rm(join(f.configHome, 'AGENTS.md'), { force: true }); + await rm(refusal(f), { force: true }); + }, + prompt: f => `${invoke(f.harness, 'setup-pstack', `Use these answers and ask nothing else. Keep every role assignment and every family's current effort. If setup refuses a role assignment, write only setup's refusal message to ${refusal(f)} and stop without changing any configuration.`)} Do not push, post, or contact GitHub.`, + async assert(record, f) { + const assertions = [loaded(record, 'pstack:setup-pstack')]; + if (await readFile(join(f.configHome, 'pstack-models.md'), 'utf8') !== prepared) throw new Error('writer-refusal-sheet-changed'); + if (f.harness === 'codex' && await lstat(join(f.configHome, 'AGENTS.md')).then(() => true, () => false)) throw new Error('writer-refusal-integration-written'); + const text = await readFile(refusal(f), 'utf8').catch(() => ''); + if (!text.includes(REFUSED_ROLE) || !text.toLowerCase().includes('devin')) throw new Error('writer-refusal-missing'); + return { assertions: [...assertions, `refused:${REFUSED_ROLE}:${READ_ONLY_LANE}`], files: [refusal(f)] }; + }, + }; +} + +// A rerun on a sheet that opts Codex writers into full access. Setup keeps the setting line and does not treat it as a +// role. One alias role is left out, so the rerun must write. +function keepSetting(): Case { + let expected: string[] = []; + return { + id: 'keep-setting', + codexSandbox: 'danger-full-access', + claudePermission: 'bypassPermissions', + async prepare(f) { + const roles = [...(await readCatalog(f.location)).defaults]; + const omitted = roles.find(([, lanes]) => lanes.every(lane => !lane.includes(':')))?.[0]; + expected = roles.map(([role, lanes]) => `${role}: ${lanes.join(', ')}`).sort(); + await writeFile(join(f.configHome, 'pstack-models.md'), sheetText(roles.filter(([role]) => role !== omitted)) + `${FULL_ACCESS_SETTING}\n`); + // Codex mirrors the sheet in a run-owned AGENTS.md block; without it the sheet is the only source. + if (f.harness === 'codex') await rm(join(f.configHome, 'AGENTS.md'), { force: true }); + }, + prompt: f => `${invoke(f.harness, 'setup-pstack', `Use these answers and ask nothing else. Keep every role assignment and every family's current effort. I confirm every write. Write the setup report to ${join(f.dir, 'result.md')}.`)} Do not push, post, or contact GitHub.`, + async assert(record, f) { + const sheet = join(f.configHome, 'pstack-models.md'), assertions = [loaded(record, 'pstack:setup-pstack'), await written(record, sheet)]; + const text = await readFile(sheet, 'utf8'); + if (!Bun.deepEquals(settingLines(text), [FULL_ACCESS_SETTING])) throw new Error('setting-not-kept'); + if (!Bun.deepEquals(roleLines(text).sort(), expected)) throw new Error('keep-setting-sheet-mismatch'); + return { assertions: [...assertions, ...await mirrored(f, sheet), `setting:${FULL_ACCESS_SETTING}`], files: [sheet] }; }, }; } @@ -400,11 +620,47 @@ function keptPredecessor(): Case { // The parent reads a fixture sheet and reports each role's descriptors and routes, which must equal what the // installed candidate's catalog gives for that sheet and parent. const CONFIGURED_ROLE = 'judgment and prose', MISSING_ROLES = ['arena runners', 'bug-fix']; -function catalog(harness: Harness): Case[] { +// Sheet lanes for the new families, and the descriptor the parent dispatches for each: a pin, Luna, a Devin lane and +// an OpenCode lane, whose model holds a "/", unchanged, a versioned Sonnet as its rolling alias. +const CATALOG_LANES: Record = { + 'swarm workers': ['codex:gpt-6-luna@max', 'codex:gpt-6-luna@max'], + 'hardest tasks': ['claude:claude-opus-5-5@max', 'claude:claude-opus-5-5@max'], + 'how explainer': ['claude:claude-sonnet-5-5@high', 'claude:sonnet@high'], + 'how explorer': ['devin:swe-2@max', 'devin:swe-2@max'], + 'interrogate reviewers': ['opencode:opencode/deepseek-v4.1-flash-free@max', 'opencode:opencode/deepseek-v4.1-flash-free@max'], + [WRITER_ROLE]: [WRITER_LANE, WRITER_LANE], +}; +// Each generated agent for a new Claude family runs natively: the child reads a fixture nonce and writes it to its own +// output file. The Agent call returns only a launch notice, so the evidence is the child's recorded tool calls and the +// nonce it wrote. +const NATIVE_AGENTS = ['pstack-sonnet-max', 'pstack-opus-5-5-max']; +const nativeAgents: Case = { + id: 'native-agents', + async prepare(f) { + for (const agent of NATIVE_AGENTS) await writeFile(join(f.dir, `${agent}.nonce`), `${crypto.randomUUID()}\n`); + }, + prompt: f => `Do no engineering work. ${NATIVE_AGENTS.map(agent => `Use the Agent tool once with subagent_type pstack:${agent}, and give it exactly this task: "Read ${join(f.dir, `${agent}.nonce`)} and write its exact contents to ${join(f.dir, `${agent}.out`)}. Do nothing else."`).join(' ')} Wait until both output files exist, then reply DONE. ${rules(f.dir)}`, + async assert(record, f) { + const assertions: string[] = []; + for (const agent of NATIVE_AGENTS) { + // A model override would run the agent on another model, so the call must leave the agent's own model in place. + const call = record.tools.find(t => t.name === 'Agent' && t.input.subagent_type === `pstack:${agent}` && t.input.model === undefined && !t.failed); + if (!call) throw new Error(`agent-not-dispatched:pstack:${agent}`); + if (!record.tools.some(t => t.parent === call.id)) throw new Error(`child-did-not-run:pstack:${agent}`); + const wrote = await readFile(join(f.dir, `${agent}.out`), 'utf8').catch(() => null); + if (wrote?.trim() !== (await readFile(join(f.dir, `${agent}.nonce`), 'utf8')).trim()) throw new Error(`nonce-mismatch:pstack:${agent}`); + assertions.push(`native:${agent}`); + } + return { assertions, files: NATIVE_AGENTS.map(agent => join(f.dir, `${agent}.out`)) }; + }, +}; + +// With the setting line opted in, or set to bounded, where no parent passes a writer sandbox. +function consumeSheet(harness: Harness, optIn: boolean): Case { const sheet = (f: Fixture) => join(f.dir, 'pstack-models.md'), report = (f: Fixture) => join(f.dir, 'routes.json'); - let expected: { roles: { role: string; descriptors: string[]; routes: string[] }[] } = { roles: [] }; - return [{ - id: 'consume-sheet', + let expected: { roles: { role: string; descriptors: string[]; routes: string[] }[]; writerSandbox: 'full-access' | null } = { roles: [], writerSandbox: null }; + return { + id: optIn ? 'consume-sheet' : 'consume-sheet-bounded', async prepare(f) { const installed = await readCatalog(f.location), defaults = installed.defaults.get(CONFIGURED_ROLE); // The configured role takes another role's default, so only a read of the fixture sheet reports it. @@ -413,18 +669,26 @@ function catalog(harness: Harness): Case[] { if (!defaults || !configured) throw new Error(`catalog-role-missing:${CONFIGURED_ROLE}`); const missing = MISSING_ROLES.map(role => ({ role, descriptors: installed.defaults.get(role) ?? [] })); if (missing.some(m => !m.descriptors.length)) throw new Error(`catalog-role-missing:${MISSING_ROLES.join(',')}`); + for (const role of Object.keys(CATALOG_LANES)) if (!installed.defaults.has(role)) throw new Error(`catalog-role-missing:${role}`); await writeFile(sheet(f), sheetText([...installed.defaults].filter(([role]) => !MISSING_ROLES.includes(role)) - .map(([role, lanes]) => [role, role === CONFIGURED_ROLE ? [configured] : lanes]))); - expected = { roles: [{ role: CONFIGURED_ROLE, descriptors: [configured] }, ...missing] - .map(r => ({ ...r, routes: r.descriptors.map(d => installed.route(harness, d)) })) }; + .map(([role, lanes]) => [role, role === CONFIGURED_ROLE ? [configured] : role in CATALOG_LANES ? [CATALOG_LANES[role]![0]] : lanes])) + `${optIn ? FULL_ACCESS_SETTING : BOUNDED_SETTING}\n`); + const added = Object.entries(CATALOG_LANES).map(([role, [, dispatched]]) => ({ role, descriptors: [dispatched] })); + expected = { roles: [{ role: CONFIGURED_ROLE, descriptors: [configured] }, ...missing, ...added] + .map(r => ({ ...r, routes: r.descriptors.map(d => installed.route(harness, d)) })), + // An opted-in sheet passes the flag only on an external route; a native Codex writer passes none. + writerSandbox: optIn && installed.route(harness, WRITER_LANE) === 'external' ? 'full-access' : null }; }, - prompt: f => `${invoke(f.harness, 'poteto-mode', `Do no engineering work and launch no lane. For this task the pstack model sheet is ${sheet(f)}; use it in place of any configured sheet. For each of these roles, in this order, find the descriptors this parent would dispatch and the route of each: ${expected.roles.map(r => JSON.stringify(r.role)).join(', ')}. Write to ${report(f)} only JSON of the form {"roles":[{"role":"","descriptors":[""],"routes":["native" or "external"]}]}, one route per descriptor, where "native" is this parent's own subagent primitive and "external" is the external runner.`)} ${rules(f.dir)}`, + prompt: f => `${invoke(f.harness, 'poteto-mode', `Do no engineering work and launch no lane. For this task the pstack model sheet is ${sheet(f)}; use it in place of any configured sheet. For each of these roles, in this order, find the descriptors this parent would dispatch and the route of each: ${expected.roles.map(r => JSON.stringify(r.role)).join(', ')}. Write to ${report(f)} only JSON of the form {"roles":[{"role":"","descriptors":[""],"routes":["native" or "external"]}],"writerSandbox":"" or null}, one route per descriptor, where "native" is this parent's own subagent primitive and "external" is the external runner. writerSandbox is the --writer-sandbox value this parent would pass when it dispatches the first descriptor of the ${JSON.stringify(WRITER_ROLE)} role as an isolated-write writer, or null if it would pass none.`)} ${rules(f.dir)}`, async assert(record, f) { const assertions = [loaded(record, 'pstack:poteto-mode'), await written(record, report(f))]; if (!Bun.deepEquals(parseJson(await readFile(report(f), 'utf8')), expected)) throw new Error('catalog-route-mismatch'); return { assertions: [...assertions, ...expected.roles.map(r => `role:${r.role.replaceAll(' ', '-')}`)], files: [report(f), sheet(f)] }; }, - }]; + }; +} + +function catalog(harness: Harness): Case[] { + return [consumeSheet(harness, true), consumeSheet(harness, false), ...(harness === 'claude' ? [nativeAgents] : [])]; } const shippedTools: Case = { @@ -474,6 +738,9 @@ const startupHook: Case = { /** Resolve a feature's cases, or fail with a named reason before any session starts. */ export function recipe(feature: string, harness: Harness, routes: string[] = []): Case[] { + if (feature === 'skill-invocation:create-formal-verification-skill') return [createFormal()]; + if (feature === 'skill-invocation:maintain-formal-verification-skill') return [maintainFormal()]; + if (feature === 'skill-invocation:poteto-mode') return [routingOnly]; if (feature === 'assets:codex') throw new Error('unsupported-native-consumer:assets:codex'); if (feature === 'project-skill') return [projectSkill]; if (feature === 'hook:startup') return [startupHook]; @@ -487,3 +754,18 @@ export function recipe(feature: string, harness: Harness, routes: string[] = []) if (!skill) throw new Error(`missing-recipe:${harness}/${feature}`); return [overrides[skill] ?? skillCase(skill, 'Apply it to the task in $DIR/TASK.md.')]; } + +const routingOnly: Case = { + id: 'route-without-dispatch', + async prepare(f) { + await files(f.dir, { 'pstack-models.md': '# pstack model configuration\n\njudgment and prose: inherit-parent\n' }); + }, + prompt: f => `${invoke(f.harness, 'poteto-mode', `Do no engineering work and launch no model session or subagent. Use ${join(f.dir, 'pstack-models.md')} as this exercise's model sheet. Resolve judgment and prose and write only JSON to ${join(f.dir, 'route.json')}: {"descriptor":"","route":"native" or "external"}.`)} ${rules(f.dir)}`, + async assert(record, f) { + loaded(record, 'pstack:poteto-mode'); + const path = join(f.dir, 'route.json'); + if (!Bun.deepEquals(JSON.parse(await readFile(path, 'utf8')), { descriptor: 'inherit-parent', route: 'native' })) throw new Error('router-decision-mismatch'); + if (record.tools.some(t => t.name === 'Agent' || t.name === 'Task' || /pstack-runner|codex exec|claude -p/.test(JSON.stringify(t.input)))) throw new Error('router-dispatched-unrequested-lane'); + return { assertions: ['skill-loaded', 'configured-native-route', 'no-dispatch'], files: [path] }; + }, +}; diff --git a/.claude/skills/verify-open-pstack/features/registry.json b/.claude/skills/verify-open-pstack/features/registry.json index 52aff7cc..0104186a 100644 --- a/.claude/skills/verify-open-pstack/features/registry.json +++ b/.claude/skills/verify-open-pstack/features/registry.json @@ -1,8 +1,8 @@ { "skills": [ "architect", "arena", "automate-me", "babysit", "benchmark-checklist", "blast-radius", "bro", "correct", - "create-verification-skill", "deslop", "figure-it-out", "fix-ci", "fix-merge-conflicts", - "get-pr-comments", "how", "interrogate", "ketchup", "maintain-verification-skill", + "create-formal-verification-skill", "create-verification-skill", "deslop", "figure-it-out", "fix-ci", "fix-merge-conflicts", + "get-pr-comments", "how", "in-your-own-words", "interrogate", "ketchup", "maintain-formal-verification-skill", "maintain-verification-skill", "make-pr-easy-to-review", "no-comments", "poteto-help", "poteto-mode", "principle-attack-the-premise", "principle-boundary-discipline", "principle-build-the-lever", "principle-encode-lessons-in-structure", "principle-exhaust-the-design-space", @@ -14,7 +14,7 @@ "principle-redesign-from-first-principles", "principle-separate-before-serializing-shared-state", "principle-sequence-verifiable-units", "principle-subtract-before-you-add", "principle-test-behavior-not-implementation", "principle-type-system-discipline", - "recall", "reflect", "setup-pstack", "show-me-your-work", "swarm", "tdd", "teach", + "recall", "reflect", "setup-pstack", "sharpen", "show-me-your-work", "swarm", "tdd", "teach", "technical-writing", "thermo-nuclear-code-quality-review", "typescript-best-practices", "unslop", "what-did-i-get-done", "why" ], @@ -37,7 +37,17 @@ "plugins/pstack/agents/pstack-opus-medium.md", "plugins/pstack/agents/pstack-opus-high.md", "plugins/pstack/agents/pstack-opus-xhigh.md", - "plugins/pstack/agents/pstack-opus-max.md" + "plugins/pstack/agents/pstack-opus-max.md", + "plugins/pstack/agents/pstack-sonnet-low.md", + "plugins/pstack/agents/pstack-sonnet-medium.md", + "plugins/pstack/agents/pstack-sonnet-high.md", + "plugins/pstack/agents/pstack-sonnet-xhigh.md", + "plugins/pstack/agents/pstack-sonnet-max.md", + "plugins/pstack/agents/pstack-opus-5-5-low.md", + "plugins/pstack/agents/pstack-opus-5-5-medium.md", + "plugins/pstack/agents/pstack-opus-5-5-high.md", + "plugins/pstack/agents/pstack-opus-5-5-xhigh.md", + "plugins/pstack/agents/pstack-opus-5-5-max.md" ], "assets": ["plugins/pstack/assets/**"], "setup": ["plugins/pstack/skills/setup-pstack/SKILL.md"], @@ -57,11 +67,11 @@ ], "project": [ ".claude/skills/verify-open-pstack/**", ".agents/skills/verify-open-pstack", - "AGENTS.md", "CLAUDE.md", "tests/skill-collision-repro.sh", + "AGENTS.md", "CLAUDE.md", "tests/skill-collision-repro.sh", ".github/workflows/ci.yml", "package.json", ".github/workflows/windows-hooks.yml" ], "nonRuntime": [ - "README.md", "REVIEW.md", "CHANGES.md", "LICENSE", "LICENSE.md", "UPSTREAM.md", + "README.md", "REVIEW.md", "CHANGES.md", "LICENSE", "LICENSE.md", "UPSTREAM.md", "FORK.md", "TEAM-INSTALL.md", "NOTICE.md", "README-UPSTREAM.md", "LICENSE-cursor-team-kit", "LICENSE-superpowers", ".gitignore", ".mergify.yml", "docs/**", "assets/**", "plugins/pstack/skills/poteto-mode/scripts/bootstrap.test.ts", diff --git a/.claude/skills/verify-open-pstack/features/runner.md b/.claude/skills/verify-open-pstack/features/runner.md index ce45c82a..02b4b68f 100644 --- a/.claude/skills/verify-open-pstack/features/runner.md +++ b/.claude/skills/verify-open-pstack/features/runner.md @@ -20,6 +20,8 @@ Do not add implicit timeouts or weaker-model fallbacks. A model assertion that a ## Headless -The runner recipe in `recipes.ts` has the installed `poteto-mode` parent launch the candidate `pstack-runner` for one external lane: from Claude, the first lane of the installed catalog's `bug-fix` role default, and from Codex, the first lane of its `judgment and prose` role default. The case fails before launch if that lane is native to the parent. It asserts the launcher command ran from the installed plugin, the receipt reports `complete` with the requested parent, provider, model, effort, and mode, and this session wrote the output. `verify.sh run --runner-route PROVIDER:MODEL@EFFORT` replaces the route for each parent whose provider differs. +The runner recipe in `recipes.ts` has the installed `poteto-mode` parent launch the candidate `pstack-runner` for one external lane: from Claude, the first lane of the installed catalog's `bug-fix` role default, and from Codex, the first lane of its `judgment and prose` role default. The case fails before launch if the installed catalog's route table does not list that lane's provider, or routes it natively from this parent. It asserts the launcher command ran from the installed plugin, the receipt reports `complete` with the requested parent, provider, model, effort, and mode, and this session wrote the output. `verify.sh run --runner-route PROVIDER:MODEL@EFFORT` replaces the route for each parent whose provider differs. The flag accepts any provider name; the installed catalog decides whether the route exists and is external, so a new provider such as Devin needs no verifier change. From Codex the recipe adds the `linked-worktree-writer` case: one `claude:opus@max` `isolated-write` lane whose cwd is a linked worktree of a run-owned repository. It asserts the receipt is `complete` with that route and cwd, the receipt argv's `--settings` sandbox denies writes to the repository's real shared `.git` directory, and the child wrote `WRITTEN.md` in the worktree during this session. Route overrides do not change this case. + +From Claude the recipe adds the `full-access-writer` case. Its fixture sheet puts `feature, refactoring` on `codex:gpt-6.1-sol@max` and holds the line `codex writer sandbox: full-access`. The prompt names the sheet, the role and the lane's paths, never the runner flag. The child writes a per-run nonce to `.agents/result.json` in its cwd, and adds a second nonce that it reads from the DOM headless Chrome dumps for a fixture page (Playwright's Chromium when Chrome is absent). The page file holds that nonce only base64-encoded. The case asserts the receipt is `complete` for that route and cwd, records `writerSandbox: "full-access"` and carries `--sandbox danger-full-access` in its argv, and that the `.agents/` result written this session holds both nonces. Its counterpart, `bounded-writer`, uses the same dispatch from a fixture sheet that puts `feature, refactoring` on `codex:gpt-6.1-sol@low` and has no setting line. The child writes a per-run nonce to the plain file `result.json` in its cwd, which `workspace-write` allows; it uses no browser and no `.agents/` path. The case asserts the receipt is `complete` for that route and cwd, records `writerSandbox: "bounded"`, carries `--sandbox workspace-write` and never `danger-full-access` in its argv, and that `result.json` written this session holds the nonce. Docker is not exercised. Route overrides do not change these cases. diff --git a/.claude/skills/verify-open-pstack/features/setup.md b/.claude/skills/verify-open-pstack/features/setup.md index 7dac9b02..bc2fc7a3 100644 --- a/.claude/skills/verify-open-pstack/features/setup.md +++ b/.claude/skills/verify-open-pstack/features/setup.md @@ -20,4 +20,4 @@ Claude uses the operator's normal config/login; Codex uses a run-owned home with ## Headless -The setup recipe in `recipes.ts` serves `setup` and `skill-invocation:setup-pstack`. One session runs setup with explicit answers that move every model-family role to `claude:opus@high`, and asserts the written sheet holds only that descriptor. A second session has `poteto-mode` report the configured route for one role. A third, `keep-predecessor`, writes a sheet whose every family lane is `grok:grok-4.6@high`, a kept predecessor, with one alias role left out, and asks setup to decline the replacement. It asserts the rewritten sheet keeps `grok:grok-4.6@high`, restores the missing role, and that the reported proposal is the family's current model from the installed catalog at the same effort. A fourth, `first-run`, removes the sheet (and, on Codex, the run-owned `AGENTS.md` mirror) and asserts that setup writes a sheet whose role lines equal the installed catalog's role defaults. The parent snapshots the daily files before the first session and restores and verifies them after the last, on success or failure. +The setup recipe in `recipes.ts` serves `setup` and `skill-invocation:setup-pstack`. One session, `configure`, runs setup with explicit answers that move every model-family role to `claude:opus@high` and answer yes when setup asks whether Codex writer lanes should run with full access. Setup keeps an existing setting line without asking, so before the session the case removes any `codex writer sandbox:` line from the operator's sheet and keeps every other byte. It asserts that the written sheet holds only that descriptor and exactly one setting line, `codex writer sandbox: full-access`. A second session has `poteto-mode` report the configured route for one role. A third, `keep-predecessor`, writes a sheet whose every family lane is `grok:grok-4.6@high`, a kept predecessor, with one alias role left out, and asks setup to decline the replacement. It asserts the rewritten sheet keeps `grok:grok-4.6@high`, restores the missing role, and that the reported proposal is the family's current model from the installed catalog at the same effort. A fourth, `new-families`, writes the installed catalog's role defaults with `swarm workers` on `codex:gpt-6-luna@max`, `hardest tasks` on the registered pin `claude:claude-opus-5-5@max`, `how explainer` on the versioned `claude:claude-sonnet-5-5@max`, and `judgment and prose` on the older revision `claude:claude-opus-5@max`, and asks setup to keep every assignment and effort. On Codex it removes the run-owned `AGENTS.md` mirror first. It asserts that every role line of the rewritten sheet equals the prepared line, except that the versioned revisions are written as `claude:sonnet@max` and `claude:opus@max`. A fifth, `read-only-provider`, writes the installed catalog's role defaults with the read-only role `how explorer` on `devin:swe-2@max`, a provider whose route table lists only `read-only`, and asks setup to keep every assignment and effort. On Codex it removes the run-owned `AGENTS.md` mirror first. It asserts that every role line of the rewritten sheet equals the prepared line. A sixth, `writer-refusal`, assigns the writer role `bug-fix` to `devin:swe-2@max` and asks setup to keep every assignment. It asserts that the sheet keeps its exact bytes, that Codex's removed mirror is not written back, and that setup's refusal names the role and the provider. A seventh, `keep-setting`, writes the installed catalog's role defaults with one alias role left out and the line `codex writer sandbox: full-access`, and asks setup to keep every assignment and effort. On Codex it removes the run-owned `AGENTS.md` mirror first. It asserts that the rewritten sheet holds that setting line exactly once and that its role lines equal the role defaults. An eighth, `first-run`, removes the sheet (and, on Codex, the run-owned `AGENTS.md` mirror) and asserts that setup writes a sheet whose role lines equal the installed catalog's role defaults. Every role-line comparison leaves out the setting line. Setup asks about full access when the sheet has no setting line, and declining is the default. So `keep-predecessor`, `new-families`, `read-only-provider` and `first-run`, which start without a setting line and take the default answers, each assert that the written sheet holds exactly one setting line, `codex writer sandbox: bounded`. A missing setting line or an unrequested `full-access` fails them. `writer-refusal` keeps its exact-bytes check: its prepared sheet has no setting line, and setup writes nothing. Every case that removes the Codex `AGENTS.md` mirror also asserts that setup wrote it back: exactly one begin marker and a newline, the rewritten sheet's exact bytes, and one end marker. The parent snapshots the daily files before the first session and restores and verifies them after the last, on success or failure. diff --git a/.claude/skills/verify-open-pstack/features/skill-invocation.md b/.claude/skills/verify-open-pstack/features/skill-invocation.md index 672e1b00..a2d0c2b2 100644 --- a/.claude/skills/verify-open-pstack/features/skill-invocation.md +++ b/.claude/skills/verify-open-pstack/features/skill-invocation.md @@ -25,3 +25,11 @@ A generic prompt about a skill is not native invocation evidence. A list of disc `recipes.ts` gives every `skill-invocation:` one default case: a disposable Git fixture holding a small task in `TASK.md`, a native invocation (`/pstack:` in Claude, the Skill tool for principle leaves, `$pstack:` in Codex), and an instruction to write the result to `result.md` in the fixture. It asserts that the native record shows the `pstack:` skill load and that the session wrote a nonempty `result.md`. Skills whose normal use would push, merge, post, or write daily state override it: `fix-ci`, `babysit`, and `make-pr-easy-to-review` run against a local fixture with no remote and assert a local effect, and `automate-me` writes its draft into the fixture instead of the daily skills folder. `setup-pstack` uses the setup recipe. `project-skill` runs from the candidate checkout and asserts the skill load, the doctor command, and the canonical-path `doctor.json`. `reflect` has two cases in each harness that exercise step 1 only, against run-owned synthetic transcripts with metadata before the opening user message. The prompts name the opening prompt and, for Codex, the known chat ID and a fixture-only `CODEX_HOME`, or, for Claude, the fixture's per-project directory. They do not name the resolver or the transcript file, so the installed active-transcript mapping must lead the session there. In `locate-transcript` the transcript's opener matches. The case asserts the opening-message metadata and that no digest was written. In `digest-fallback` the opener differs, and the case asserts a nonblank digest of at most 2,000 words. Each run uses a per-run nonce that no prompt contains. For Codex, it is in the fixture rollout's file name. A completed command must name the installed resolver's launcher or script by absolute path, and its harness-captured output must parse to exactly the resolver JSON for this run, key order aside. Failed or pending commands and output from another fixture or run do not count, and a report file the model writes is never the Codex execution evidence. For Claude, the nonce is the opening user record's `uuid`, which only the transcript holds. In both cases the report must equal the expected JSON exactly, including that `uuid`: under `opening_user_message` when the opener matches, and beside the fallback status when it does not. A report without the `uuid`, or with another run's, does not count. These cases do not exercise reviewer fan-out, synthesis, or skill edits. + +## Formal verification skills + +`create-formal-verification-skill` receives an atomic capacity module and an available bounded state checker, but no project skill. It must create the skill with coverage and sync commit, follow it on a scratch split-check mutation, replay oversubscription, then restore code and model. The parent reruns the checker and requires its passing result. + +`maintain-formal-verification-skill` receives a real project skill and a stale atomic model. The current committed code yields between checking capacity and reserving it. The stale check passes. The session must observe that pass, update the model without changing its bounds or interpreter, observe the counterexample, replay it on the real code, and record the current sync commit. Product edits, a no-op model, changed bounds, missing before/after evidence or missing replay fail the recipe. The parent reruns the preserved checker independently. This proves maintenance of a bounded model for two one-shot callers at capacity one; it is not an unbounded proof. + +`skill-invocation:poteto-mode` resolves the supplied `inherit-parent` role to `native`, without launching a lane. It checks routing and skill loading without an unadmitted descendant session. diff --git a/.claude/skills/verify-open-pstack/package.json b/.claude/skills/verify-open-pstack/package.json index 5d275464..c5b7eaf4 100644 --- a/.claude/skills/verify-open-pstack/package.json +++ b/.claude/skills/verify-open-pstack/package.json @@ -2,6 +2,6 @@ "name": "@open-pstack/project-verification", "private": true, "type": "module", - "scripts": { "test": "bun test scripts", "typecheck": "tsc --noEmit --strict" }, + "scripts": { "test": "USER=\"${USER:-$(id -un)}\" LOGNAME=\"${LOGNAME:-$(id -un)}\" bun test scripts", "typecheck": "tsc --noEmit --strict" }, "devDependencies": { "bun-types": "1.4.0", "typescript": "5.9.3" } } diff --git a/.claude/skills/verify-open-pstack/scripts/catalog.test.ts b/.claude/skills/verify-open-pstack/scripts/catalog.test.ts index 09e06220..9a01f3ba 100644 --- a/.claude/skills/verify-open-pstack/scripts/catalog.test.ts +++ b/.claude/skills/verify-open-pstack/scripts/catalog.test.ts @@ -1,8 +1,9 @@ import { afterEach, expect, test } from 'bun:test'; -import { mkdtemp, readFile, realpath, rm, writeFile } from 'node:fs/promises'; +import { chmod, mkdir, mkdtemp, readFile, realpath, rm, writeFile } from 'node:fs/promises'; import { tmpdir } from 'node:os'; import { join, resolve } from 'node:path'; -import { readCatalog, recipe, type Case, type Fixture } from '../features/recipes.ts'; +import { fileURLToPath, pathToFileURL } from 'node:url'; +import { readCatalog, recipe, type Case, type Catalog, type Fixture } from '../features/recipes.ts'; import { command } from './io.ts'; import type { Harness, SessionRecord } from './types.ts'; @@ -19,6 +20,26 @@ const record = (harness: Harness, skill: string): SessionRecord => const sheet = async (path: string): Promise> => new Map((await readFile(path, 'utf8')).split('\n') .filter(line => /^[a-z][^:]*: \S/.test(line)).map((line): [string, string[]] => [line.slice(0, line.indexOf(': ')), line.slice(line.indexOf(': ') + 2).split(', ')])); const setupCase = (id: string): Case => recipe('setup', 'claude').find(c => c.id === id)!; +const mirror = (f: Fixture, text: string) => + writeFile(join(f.dir, 'AGENTS.md'), `notes\n\n${text}\n`); +const BOUNDED = 'codex writer sandbox: bounded\n', FULL_ACCESS = 'codex writer sandbox: full-access\n'; +/** A default-answer case fails on an unrequested full-access line or a missing setting line, then gets `text` back. */ +async function rejectsUnbounded(c: Case, f: Fixture, text: string): Promise { + const path = join(f.dir, 'pstack-models.md'); + for (const wrong of [text.replace(BOUNDED, FULL_ACCESS), text.replace(BOUNDED, '')]) { + await writeFile(path, wrong); + await expect(c.assert(record(f.harness, 'pstack:setup-pstack'), f)).rejects.toThrow('setting-not-bounded'); + } + await writeFile(path, text); +} +// What a correct parent reports for the consume-sheet cases: the sheet's line, else the role default, with a Sonnet +// revision normalized to its rolling alias, routed by the catalog. +const promptRoles = (c: Case, f: Fixture): string[] => JSON.parse(`[${/the route of each: (.*?)\. Write to/.exec(c.prompt(f))![1]}]`); +const dispatched = (d: string) => d.replace(/^claude:claude-sonnet-[0-9-]+@/, 'claude:sonnet@'); +const reportRoles = (roles: string[], lines: Map, installed: Catalog, harness: Harness) => roles.map(role => { + const descriptors = (lines.get(role) ?? installed.defaults.get(role)!).map(dispatched); + return { role, descriptors, routes: descriptors.map(d => installed.route(harness, d)) }; +}); test('reads role defaults, kept predecessors and routes from the generated catalog', async () => { const installed = await readCatalog(location); @@ -27,9 +48,10 @@ test('reads role defaults, kept predecessors and routes from the generated catal for (const harness of ['claude', 'codex'] as const) { expect(installed.route(harness, `${harness}:any@max`)).toBe('native'); expect(installed.route(harness, 'grok:any@max')).toBe('external'); + expect(installed.route(harness, 'devin:any@max')).toBe('external'); expect(installed.route(harness, 'inherit-parent')).toBe('native'); } - expect(() => installed.route('claude', 'devin:any@max')).toThrow('catalog-route-missing'); + expect(() => installed.route('claude', 'kiro:any@max')).toThrow('catalog-route-missing'); }); for (const harness of ['claude', 'codex'] as const) { @@ -37,23 +59,69 @@ for (const harness of ['claude', 'codex'] as const) { const f = await fixture(harness), [c] = recipe('catalog', harness), installed = await readCatalog(location); await c!.prepare!(f); const lines = await sheet(join(f.dir, 'pstack-models.md')); - const roles: string[] = JSON.parse(`[${/the route of each: (.*?)\. Write to/.exec(c!.prompt(f))![1]}]`); + const roles = promptRoles(c!, f); expect(roles.length).toBeGreaterThan(0); - // What a correct parent reports: the sheet's line, else the role default, routed by the catalog. - const report = { roles: roles.map(role => { - const descriptors = lines.get(role) ?? installed.defaults.get(role)!; - return { role, descriptors, routes: descriptors.map(d => installed.route(harness, d)) }; - }) }; + // The sheet opts Codex writers into full access, which only Claude's external runner route passes on. + expect(lines.get('codex writer sandbox')).toEqual(['full-access']); + const report = { roles: reportRoles(roles, lines, installed, harness), writerSandbox: harness === 'claude' ? 'full-access' : null }; expect(report.roles[0]!.descriptors).not.toEqual(installed.defaults.get(report.roles[0]!.role)); - expect(roles.slice(1)).toEqual(['arena runners', 'bug-fix']); - for (const role of roles.slice(1)) expect(lines.has(role)).toBe(false); + expect(roles.slice(1, 3)).toEqual(['arena runners', 'bug-fix']); + for (const role of roles.slice(1, 3)) expect(lines.has(role)).toBe(false); + expect(roles.slice(3)).toEqual(['swarm workers', 'hardest tasks', 'how explainer', 'how explorer', 'interrogate reviewers', 'feature, refactoring']); + expect(roles.slice(3).map(role => lines.get(role))).toEqual([ + ['codex:gpt-6-luna@max'], ['claude:claude-opus-5-5@max'], ['claude:claude-sonnet-5-5@high'], ['devin:swe-2@max'], + ['opencode:opencode/deepseek-v4.1-flash-free@max'], ['codex:gpt-6.1-sol@max'], + ]); + expect(report.roles.slice(3).map(r => [r.descriptors, r.routes])).toEqual([ + [['codex:gpt-6-luna@max'], [harness === 'codex' ? 'native' : 'external']], + [['claude:claude-opus-5-5@max'], [harness === 'claude' ? 'native' : 'external']], + [['claude:sonnet@high'], [harness === 'claude' ? 'native' : 'external']], + [['devin:swe-2@max'], ['external']], + [['opencode:opencode/deepseek-v4.1-flash-free@max'], ['external']], + [['codex:gpt-6.1-sol@max'], [harness === 'codex' ? 'native' : 'external']], + ]); await writeFile(join(f.dir, 'routes.json'), JSON.stringify(report)); expect((await c!.assert(record(harness, 'pstack:poteto-mode'), f)).assertions.slice(0, 2)).toEqual(['skill-loaded', 'file:routes.json']); + // A parent that leaves the Sonnet revision unnormalized, normalizes the pin to opus, or reports the role default + // in place of the Devin lane, fails. + for (const [role, wrong] of [['how explainer', 'claude:claude-sonnet-5-5@high'], ['hardest tasks', 'claude:opus@max'], ['how explorer', installed.defaults.get('how explorer')![0]]]) { + const changed = { roles: report.roles.map(r => r.role === role ? { ...r, descriptors: [wrong!] } : r) }; + await writeFile(join(f.dir, 'routes.json'), JSON.stringify(changed)); + await expect(c!.assert(record(harness, 'pstack:poteto-mode'), f)).rejects.toThrow('catalog-route-mismatch'); + } + + // A Claude parent that passes no writer sandbox fails, and so does a Codex parent that passes one to its native writer. + await writeFile(join(f.dir, 'routes.json'), JSON.stringify({ ...report, writerSandbox: harness === 'claude' ? null : 'full-access' })); + await expect(c!.assert(record(harness, 'pstack:poteto-mode'), f)).rejects.toThrow('catalog-route-mismatch'); + + // A parent that routes the Devin lane natively fails. + const nativeDevin = { roles: report.roles.map(r => r.role === 'how explorer' ? { ...r, routes: ['native'] } : r) }; + await writeFile(join(f.dir, 'routes.json'), JSON.stringify(nativeDevin)); + await expect(c!.assert(record(harness, 'pstack:poteto-mode'), f)).rejects.toThrow('catalog-route-mismatch'); + report.roles[0]!.routes = report.roles[0]!.routes.map(r => r === 'native' ? 'external' : 'native'); await writeFile(join(f.dir, 'routes.json'), JSON.stringify(report)); await expect(c!.assert(record(harness, 'pstack:poteto-mode'), f)).rejects.toThrow('catalog-route-mismatch'); }); + + test(`${harness}: the bounded consume-sheet case takes the same sheet with the bounded setting line and requires no writer sandbox`, async () => { + const opted = await fixture(harness), f = await fixture(harness), installed = await readCatalog(location); + const [optIn, c] = recipe('catalog', harness); + expect(c!.id).toBe('consume-sheet-bounded'); + await optIn!.prepare!(opted); await c!.prepare!(f); + const path = join(f.dir, 'pstack-models.md'); + expect(await readFile(path, 'utf8')).toBe((await readFile(join(opted.dir, 'pstack-models.md'), 'utf8')).replace(FULL_ACCESS, BOUNDED)); + const roles = promptRoles(c!, f); + expect(roles).toEqual(promptRoles(optIn!, opted)); + const report = { roles: reportRoles(roles, await sheet(path), installed, harness), writerSandbox: null }; + await writeFile(join(f.dir, 'routes.json'), JSON.stringify(report)); + expect((await c!.assert(record(harness, 'pstack:poteto-mode'), f)).assertions.slice(0, 2)).toEqual(['skill-loaded', 'file:routes.json']); + + // A parent that passes full access without the opt-in fails, from either parent. + await writeFile(join(f.dir, 'routes.json'), JSON.stringify({ ...report, writerSandbox: 'full-access' })); + await expect(c!.assert(record(harness, 'pstack:poteto-mode'), f)).rejects.toThrow('catalog-route-mismatch'); + }); } test('the kept-predecessor case requires every loaded role, the materialized role and the proposed replacement', async () => { @@ -62,10 +130,11 @@ test('the kept-predecessor case requires every loaded role, the materialized rol const path = join(f.dir, 'pstack-models.md'), prepared = await readFile(path, 'utf8'); const omitted = [...installed.defaults.keys()].find(role => !prepared.includes(`\n${role}: `))!; expect(installed.defaults.get(omitted)).toEqual(['inherit-parent']); - const rerun = `${prepared}${omitted}: inherit-parent\n`; + const rerun = `${prepared}${omitted}: inherit-parent\n${BOUNDED}`; await writeFile(path, rerun); await writeFile(join(f.dir, 'proposal.json'), JSON.stringify({ kept: 'grok:grok-4.6@high', proposed: `${installed.kept.get('grok:grok-4.6')}@high` })); expect((await c.assert(record('claude', 'pstack:setup-pstack'), f)).assertions).toContain('kept:grok:grok-4.6@high'); + await rejectsUnbounded(c, f, rerun); await writeFile(path, rerun.replaceAll('grok:grok-4.6@', `${installed.kept.get('grok:grok-4.6')}@`)); await expect(c.assert(record('claude', 'pstack:setup-pstack'), f)).rejects.toThrow('kept-sheet-mismatch'); @@ -76,6 +145,52 @@ test('the kept-predecessor case requires every loaded role, the materialized rol }); for (const harness of ['claude', 'codex'] as const) { + test(`${harness}: the new-families case keeps Luna, Sonnet and the pin, and requires the old pin as opus`, async () => { + const f = await fixture(harness), c = recipe('setup', harness).find(c => c.id === 'new-families')!, installed = await readCatalog(location); + await writeFile(join(f.dir, 'AGENTS.md'), 'mirror\n'); + await c.prepare!(f); + if (harness === 'codex') await expect(readFile(join(f.dir, 'AGENTS.md'))).rejects.toThrow(); + else expect(await readFile(join(f.dir, 'AGENTS.md'), 'utf8')).toBe('mirror\n'); + const path = join(f.dir, 'pstack-models.md'), prepared = await readFile(path, 'utf8'), lines = await sheet(path); + expect([...lines.keys()]).toEqual([...installed.defaults.keys()]); + expect(['swarm workers', 'hardest tasks', 'how explainer', 'judgment and prose'].map(role => lines.get(role))).toEqual([ + ['codex:gpt-6-luna@max'], ['claude:claude-opus-5-5@max'], ['claude:claude-sonnet-5-5@max'], ['claude:claude-opus-5@max'], + ]); + for (const [role, lanes] of installed.defaults) { + if (!['swarm workers', 'hardest tasks', 'how explainer', 'judgment and prose'].includes(role)) expect(lines.get(role)).toEqual(lanes); + } + + const rerun = prepared.replace('judgment and prose: claude:claude-opus-5@max', 'judgment and prose: claude:opus@max') + .replace('how explainer: claude:claude-sonnet-5-5@max', 'how explainer: claude:sonnet@max') + BOUNDED; + await writeFile(path, rerun); + if (harness === 'codex') await mirror(f, rerun); + expect((await c.assert(record(harness, 'pstack:setup-pstack'), f)).assertions).toEqual([ + 'skill-loaded', 'file:pstack-models.md', ...(harness === 'codex' ? ['codex-mirror'] : []), + 'sheet:codex:gpt-6-luna@max', 'sheet:claude:claude-opus-5-5@max', 'sheet:claude:sonnet@max', 'sheet:claude:opus@max', + 'setting:codex writer sandbox: bounded', + ]); + await rejectsUnbounded(c, f, rerun); + + // Keeping an old revision, normalizing the registered pin, or dropping a role each fails. + for (const wrong of [prepared, rerun.replace('how explainer: claude:sonnet@max', 'how explainer: claude:claude-sonnet-5-5@max'), rerun.replace('claude:claude-opus-5-5@max', 'claude:opus@max'), rerun.replace(/\nbug-fix: [^\n]*/, '')]) { + await writeFile(path, wrong); + await expect(c.assert(record(harness, 'pstack:setup-pstack'), f)).rejects.toThrow('new-families-sheet-mismatch'); + } + await writeFile(path, rerun); + + // On Codex, the AGENTS.md block must mirror the rewritten sheet. + if (harness === 'codex') { + // A stale block, a block missing the sheet's last newline, or a second block each fails. + for (const agents of [`\n${prepared}\n`, + `\n${rerun.replace(/\n$/, '')}\n`, + `\n${rerun}\n\nold\n\n`, + `\n${rerun}\n`]) { + await writeFile(join(f.dir, 'AGENTS.md'), agents); + await expect(c.assert(record(harness, 'pstack:setup-pstack'), f)).rejects.toThrow('codex-mirror-mismatch'); + } + } + }); + test(`${harness}: the runner recipe's default route is the candidate's role default for this parent`, async () => { const f = await fixture(harness), [c] = recipe('runner', harness), installed = await readCatalog(location); await c!.prepare!(f); @@ -86,6 +201,92 @@ for (const harness of ['claude', 'codex'] as const) { }); } +for (const harness of ['claude', 'codex'] as const) { + test(`${harness}: the read-only-provider case requires every role line, with how explorer on Devin`, async () => { + const f = await fixture(harness), c = recipe('setup', harness).find(c => c.id === 'read-only-provider')!, installed = await readCatalog(location); + await writeFile(join(f.dir, 'AGENTS.md'), 'mirror\n'); + await c.prepare!(f); + if (harness === 'codex') await expect(readFile(join(f.dir, 'AGENTS.md'))).rejects.toThrow(); + else expect(await readFile(join(f.dir, 'AGENTS.md'), 'utf8')).toBe('mirror\n'); + const path = join(f.dir, 'pstack-models.md'), prepared = await readFile(path, 'utf8'), lines = await sheet(path); + expect([...lines.keys()]).toEqual([...installed.defaults.keys()]); + for (const [role, lanes] of installed.defaults) expect(lines.get(role)).toEqual(role === 'how explorer' ? ['devin:swe-2@max'] : lanes); + + await writeFile(path, prepared + BOUNDED); + if (harness === 'codex') await mirror(f, prepared + BOUNDED); + const probed = record(harness, 'pstack:setup-pstack'); + expect((await c.assert(probed, f)).assertions).toEqual([ + 'skill-loaded', 'file:pstack-models.md', ...(harness === 'codex' ? ['codex-mirror'] : []), 'sheet:devin:swe-2@max', + 'setting:codex writer sandbox: bounded', + ]); + await rejectsUnbounded(c, f, prepared + BOUNDED); + + // Dropping the Devin lane or another role fails. + for (const wrong of [prepared.replace('how explorer: devin:swe-2@max', `how explorer: ${installed.defaults.get('how explorer')!.join(', ')}`), prepared.replace(/\nbug-fix: [^\n]*/, '')]) { + await writeFile(path, wrong); + await expect(c.assert(probed, f)).rejects.toThrow('read-only-provider-sheet-mismatch'); + } + }); + + test(`${harness}: the runner recipe accepts any external provider in the installed catalog and refuses others`, async () => { + const f = await fixture(harness); + const [devin, swe16] = recipe('runner', harness, ['devin:swe-2@max', 'devin:swe-1.6@default']); + await devin!.prepare!(f); + await swe16!.prepare!(f); + expect(devin!.prompt(f)).toContain(`--parent ${harness} --provider devin --model swe-2 --effort max --mode read-only`); + expect(swe16!.prompt(f)).toContain(`--parent ${harness} --provider devin --model swe-1.6 --effort default --mode read-only`); + const [unknown] = recipe('runner', harness, ['kiro:auto@default']); + await expect(unknown!.prepare!(f)).rejects.toThrow('catalog-route-missing:kiro:auto@default'); + }); +} + +for (const harness of ['claude', 'codex'] as const) { + test(`${harness}: the writer-refusal case needs the sheet unchanged, no new mirror, and a refusal naming the role and provider`, async () => { + const f = await fixture(harness), c = recipe('setup', harness).find(c => c.id === 'writer-refusal')!; + await writeFile(join(f.dir, 'AGENTS.md'), 'mirror\n'); + await c.prepare!(f); + const path = join(f.dir, 'pstack-models.md'), prepared = await readFile(path, 'utf8'); + expect((await sheet(path)).get('bug-fix')).toEqual(['devin:swe-2@max']); + if (harness === 'codex') await expect(readFile(join(f.dir, 'AGENTS.md'))).rejects.toThrow(); + const done = record(harness, 'pstack:setup-pstack'); + await expect(c.assert(done, f)).rejects.toThrow('writer-refusal-missing'); + await writeFile(join(f.dir, 'refusal.txt'), 'Refusing bug-fix on devin:swe-2@max: Devin runs read-only lanes only.\n'); + expect((await c.assert(done, f)).assertions).toEqual(['skill-loaded', 'refused:bug-fix:devin:swe-2@max']); + + // Rewriting the sheet, or writing Codex's mirror back, fails. + await writeFile(path, prepared.replace('bug-fix: devin:swe-2@max', 'bug-fix: codex:gpt-6.1-sol@max')); + await expect(c.assert(done, f)).rejects.toThrow('writer-refusal-sheet-changed'); + await writeFile(path, prepared); + if (harness === 'codex') { + await mirror(f, prepared); + await expect(c.assert(done, f)).rejects.toThrow('writer-refusal-integration-written'); + } + }); +} + +test('the native-agents case needs each new agent dispatched, its child run, and its own nonce written', async () => { + const f = await fixture('claude'), c = recipe('catalog', 'claude').find(c => c.id === 'native-agents')!; + expect(recipe('catalog', 'codex').some(c => c.id === 'native-agents')).toBe(false); + await c.prepare!(f); + const agents = ['pstack-sonnet-max', 'pstack-opus-5-5-max']; + const tools = (names: string[], children = names) => [ + ...names.map((agent, i) => ({ name: 'Agent', id: `a${i}`, input: { subagent_type: `pstack:${agent}` }, parent: null, failed: false })), + ...children.map(agent => ({ name: 'Read', id: `r-${agent}`, input: {}, parent: `a${names.indexOf(agent)}`, failed: false })), + ]; + const run = (names: string[], children?: string[]): SessionRecord => ({ ...record('claude', 'pstack:poteto-mode'), tools: tools(names, children) }); + for (const agent of agents) await writeFile(join(f.dir, `${agent}.out`), await readFile(join(f.dir, `${agent}.nonce`), 'utf8')); + expect((await c.assert(run(agents), f)).assertions).toEqual(agents.map(agent => `native:${agent}`)); + + await expect(c.assert(run(['pstack-sonnet-max', 'pstack-opus-max']), f)).rejects.toThrow('agent-not-dispatched:pstack:pstack-opus-5-5-max'); + await expect(c.assert(run(agents, ['pstack-sonnet-max']), f)).rejects.toThrow('child-did-not-run:pstack:pstack-opus-5-5-max'); + // A model override on the dispatch would run the agent on another model. + const overridden = run(agents); + overridden.tools[1]!.input.model = 'opus'; + await expect(c.assert(overridden, f)).rejects.toThrow('agent-not-dispatched:pstack:pstack-opus-5-5-max'); + await writeFile(join(f.dir, 'pstack-sonnet-max.out'), 'guess\n'); + await expect(c.assert(run(agents), f)).rejects.toThrow('nonce-mismatch:pstack:pstack-sonnet-max'); +}); + test('the first-run case requires a sheet equal to the catalog role defaults', async () => { const f = await fixture('codex'), c = setupCase('first-run'), installed = await readCatalog(location); const path = join(f.dir, 'pstack-models.md'); @@ -94,8 +295,175 @@ test('the first-run case requires a sheet equal to the catalog role defaults', a await expect(readFile(path)).rejects.toThrow(); await expect(readFile(join(f.dir, 'AGENTS.md'))).rejects.toThrow(); const lines = [...installed.defaults].map(([role, lanes]) => `${role}: ${lanes.join(', ')}`); - await writeFile(path, `# pstack model configuration\n\n${lines.join('\n')}\n`); + const written = `# pstack model configuration\n\n${lines.join('\n')}\n${BOUNDED}`; + await writeFile(path, written); + await expect(c.assert(record('codex', 'pstack:setup-pstack'), f)).rejects.toThrow('codex-mirror-mismatch'); + await mirror(f, written); expect((await c.assert(record('codex', 'pstack:setup-pstack'), f)).assertions).toContain('sheet:role-defaults'); + await rejectsUnbounded(c, f, written); await writeFile(path, `# pstack model configuration\n\n${lines.slice(1).join('\n')}\n`); await expect(c.assert(record('codex', 'pstack:setup-pstack'), f)).rejects.toThrow('first-run-not-role-defaults'); }); + +test('the configure case answers yes to full access and requires that setting with only the Opus descriptor', async () => { + const f = await fixture('claude'), c = setupCase('configure'), installed = await readCatalog(location); + expect(c.prompt(f)).toContain('whether Codex writer lanes should run with full access, answer yes'); + const path = join(f.dir, 'pstack-models.md'); + const written = `# pstack model configuration\n\n${[...installed.defaults].map(([role, lanes]) => + `${role}: ${lanes.map(lane => lane.includes(':') ? 'claude:opus@high' : lane).join(', ')}`).join('\n')}\n${FULL_ACCESS}`; + await writeFile(path, written); + expect((await c.assert(record('claude', 'pstack:setup-pstack'), f)).assertions) + .toEqual(['skill-loaded', 'file:pstack-models.md', 'sheet:claude:opus@high', 'setting:codex writer sandbox: full-access']); + + // The default answer, a missing setting, or another descriptor fails. + for (const [wrong, reason] of [[written.replace(FULL_ACCESS, BOUNDED), 'setting-not-full-access'], [written.replace(FULL_ACCESS, ''), 'setting-not-full-access'], + [written.replace('claude:opus@high', 'claude:opus@max'), 'sheet-descriptors-mismatch']]) { + await writeFile(path, wrong!); + await expect(c.assert(record('claude', 'pstack:setup-pstack'), f)).rejects.toThrow(reason); + } +}); + +test('the configure case starts from the daily sheet with only its setting line removed, so setup asks', async () => { + const f = await fixture('claude'), c = setupCase('configure'), path = join(f.dir, 'pstack-models.md'); + await c.prepare!(f); + await expect(readFile(path)).rejects.toThrow(); + const daily = `# pstack model configuration\n\nbug-fix: codex:gpt-6.1-sol@max\n${FULL_ACCESS}hillclimb: codex:gpt-6.1-sol@max \n\n`; + for (const setting of [FULL_ACCESS, BOUNDED]) { + await writeFile(path, daily.replace(FULL_ACCESS, setting)); + await c.prepare!(f); + expect(await readFile(path, 'utf8')).toBe(daily.replace(FULL_ACCESS, '')); + } +}); + +test('the full-access writer case quotes each browser argument for sh and encodes the page as a file URL', async () => { + const base = await realpath(await mkdtemp(join(tmpdir(), 'pstack-catalog-recipe-'))); roots.push(base); + const dir = join(base, "x$(touch pwned)'q#r?s\nt"); + await mkdir(dir); + const f: Fixture = { harness: 'claude', dir, workspace, location, configHome: dir, run: command }; + await recipe('runner', 'claude').find(c => c.id === 'full-access-writer')!.prepare!(f); + // The command runs from the lead-in to the next instruction; a quoted path may hold a newline. + const prompt = await readFile(join(dir, 'full-access.prompt.md'), 'utf8'), lead = 'in the DOM it prints: '; + const line = prompt.slice(prompt.indexOf(lead) + lead.length, prompt.lastIndexOf('\nThen create the file')); + // `set --` splits the command into words as sh runs it, without starting the browser. + const child = Bun.spawn(['sh', '-c', `set -- ${line}; printf '%s\\0' "$@"`], { cwd: base, stdout: 'pipe' }); + const args = (await new Response(child.stdout).text()).split('\0').slice(0, -1); + expect(await child.exited).toBe(0); + const page = join(dir, 'full-access.page.html'), url = pathToFileURL(page).href; + expect(args.slice(1)).toEqual(['--headless', '--no-first-run', `--user-data-dir=${join(dir, 'full-access.browser-profile')}`, '--dump-dom', url]); + expect(url).toContain('%23'); expect(url).toContain('%3F'); expect(url).toContain('%0A'); + expect(fileURLToPath(url)).toBe(page); + await expect(readFile(join(base, 'pwned'))).rejects.toThrow(); +}); + +for (const harness of ['claude', 'codex'] as const) { + test(`${harness}: the keep-setting case requires the setting line and every role line after a rerun`, async () => { + const f = await fixture(harness), c = recipe('setup', harness).find(c => c.id === 'keep-setting')!, installed = await readCatalog(location); + await writeFile(join(f.dir, 'AGENTS.md'), 'mirror\n'); + await c.prepare!(f); + if (harness === 'codex') await expect(readFile(join(f.dir, 'AGENTS.md'))).rejects.toThrow(); + else expect(await readFile(join(f.dir, 'AGENTS.md'), 'utf8')).toBe('mirror\n'); + const path = join(f.dir, 'pstack-models.md'), prepared = await readFile(path, 'utf8'); + expect(prepared).toEndWith('\ncodex writer sandbox: full-access\n'); + const omitted = [...installed.defaults.keys()].find(role => !prepared.includes(`\n${role}: `))!; + expect(installed.defaults.get(omitted)).toEqual(['inherit-parent']); + const rerun = prepared.replace('codex writer sandbox: full-access\n', `${omitted}: inherit-parent\ncodex writer sandbox: full-access\n`); + await writeFile(path, rerun); + if (harness === 'codex') await mirror(f, rerun); + expect((await c.assert(record(harness, 'pstack:setup-pstack'), f)).assertions) + .toEqual(['skill-loaded', 'file:pstack-models.md', ...(harness === 'codex' ? ['codex-mirror'] : []), 'setting:codex writer sandbox: full-access']); + + // Dropping the setting, changing it, or dropping a role fails. + for (const [wrong, reason] of [ + [rerun.replace('codex writer sandbox: full-access\n', ''), 'setting-not-kept'], + [rerun.replace('sandbox: full-access', 'sandbox: bounded'), 'setting-not-kept'], + [rerun.replace(/\nbug-fix: [^\n]*/, ''), 'keep-setting-sheet-mismatch'], + ]) { + await writeFile(path, wrong!); + await expect(c.assert(record(harness, 'pstack:setup-pstack'), f)).rejects.toThrow(reason); + } + }); +} + +// Stands in for Codex: logged in, and the model call writes the result the test hands it to the file it names. +const FAKE_CODEX = `#!/bin/sh +if [ "$1" = "login" ]; then echo 'Logged in using ChatGPT'; exit 0; fi +cat > /dev/null +mkdir -p "$(dirname "$FAKE_RESULT_FILE")" && printf '%s' "$FAKE_RESULT" > "$FAKE_RESULT_FILE" +echo '{"type":"item.completed","item":{"type":"agent_message","text":"DONE"}}' +`; + +/** Runs the installed runner as a Claude parent's Codex writer on case `name`'s prompt and lane, and returns its receipt. */ +async function launchWriter(f: Fixture, name: string, effort: string, flags: string[], receipt: string, result: object, file: string): Promise> { + const bin = join(f.dir, 'bin'); + await mkdir(bin, { recursive: true }); + await writeFile(join(bin, 'codex'), FAKE_CODEX); await chmod(join(bin, 'codex'), 0o755); + const args = ['--parent', 'claude', '--provider', 'codex', '--model', 'gpt-6.1-sol', '--effort', effort, '--mode', 'isolated-write', ...flags, + '--prompt', join(f.dir, `${name}.prompt.md`), '--cwd', join(f.dir, `${name}-lane`), '--output', join(f.dir, `${receipt}.md`), '--receipt', join(f.dir, `${receipt}.json`)]; + const child = Bun.spawn([join(location, 'skills/poteto-mode/scripts/runner/pstack-runner'), ...args], { stdout: 'pipe', stderr: 'pipe', + env: { ...process.env, PATH: `${bin}:${process.env.PATH}`, FAKE_RESULT: JSON.stringify(result), FAKE_RESULT_FILE: file } }); + const [code, stderr] = await Promise.all([child.exited, new Response(child.stderr).text()]); + expect(code, stderr).toBe(0); + return JSON.parse(await readFile(join(f.dir, `${receipt}.json`), 'utf8')); +} + +test('the full-access writer case requires the opted-in receipt and both nonces in the child result', async () => { + const f = await fixture('claude'), c = recipe('runner', 'claude').find(c => c.id === 'full-access-writer')!; + await c.prepare!(f); + // The parent gets the sheet and the role, never the flag; the sheet holds the opt-in and the writer lane. + expect(c.prompt(f)).not.toContain('--writer-sandbox'); + const lines = await sheet(join(f.dir, 'full-access.sheet.md')); + expect([lines.get('feature, refactoring'), lines.get('codex writer sandbox')]).toEqual([['codex:gpt-6.1-sol@max'], ['full-access']]); + // Only the browser's DOM holds the page nonce; the child prompt holds the file nonce. + const page = await readFile(join(f.dir, 'full-access.page.html'), 'utf8'); + const pageNonce = atob(/atob\('([^']+)'\)/.exec(page)![1]!), fileNonce = /"written":"([0-9a-f-]+)"/.exec(await readFile(join(f.dir, 'full-access.prompt.md'), 'utf8'))![1]!; + expect(page).not.toContain(pageNonce); + + const launch = (sandbox: string, name: string) => + launchWriter(f, 'full-access', 'max', ['--writer-sandbox', sandbox], name, { written: fileNonce, browser: pageNonce }, '.agents/result.json'); + const run = { ...record('claude', 'pstack:poteto-mode'), commands: ['"$RUNNER/pstack-runner" --provider codex'] }; + // The case reads the receipt and output at full-access.receipt.json and full-access.output.md. + const receipt = await launch('full-access', 'full-access.receipt'); + await rm(join(f.dir, 'full-access.receipt.md')); + await writeFile(join(f.dir, 'full-access.receipt.json'), JSON.stringify({ ...receipt, outputPath: join(f.dir, 'full-access.output.md') })); + expect((await c.assert(run, f)).assertions).toEqual(['runner-ran', 'receipt-complete', 'writer-sandbox:full-access', 'file:result.json', 'browser-nonce']); + + // A bounded writer fails, and so does a result without the browser's nonce. + const bounded = await launch('bounded', 'bounded.receipt'); + await writeFile(join(f.dir, 'full-access.receipt.json'), JSON.stringify({ ...bounded, outputPath: join(f.dir, 'full-access.output.md') })); + await expect(c.assert(run, f)).rejects.toThrow('writer-sandbox:bounded'); + await writeFile(join(f.dir, 'full-access.receipt.json'), JSON.stringify({ ...receipt, outputPath: join(f.dir, 'full-access.output.md') })); + await writeFile(join(f.dir, 'full-access-lane/.agents/result.json'), JSON.stringify({ written: fileNonce, browser: 'unrendered' })); + await expect(c.assert(run, f)).rejects.toThrow('full-access-effect-mismatch'); +}); + +test('the bounded writer case requires a bounded receipt from a sheet without the setting line, and the nonce in a plain file', async () => { + const f = await fixture('claude'), c = recipe('runner', 'claude').find(c => c.id === 'bounded-writer')!; + await c.prepare!(f); + // The parent gets the sheet and the role, never the flag; the sheet has no setting line and a low-effort writer lane. + expect(c.prompt(f)).not.toContain('--writer-sandbox'); + const path = join(f.dir, 'bounded.sheet.md'); + expect(await readFile(path, 'utf8')).not.toContain('codex writer sandbox:'); + expect((await sheet(path)).get('feature, refactoring')).toEqual(['codex:gpt-6.1-sol@low']); + // The child writes its nonce to a plain file in its cwd, which workspace-write allows; no browser, no `.agents/`. + const prompt = await readFile(join(f.dir, 'bounded.prompt.md'), 'utf8'), fileNonce = /"written":"([0-9a-f-]+)"/.exec(prompt)![1]!; + expect(prompt).not.toContain('.agents/'); expect(prompt).not.toContain('--dump-dom'); + + const launch = (flags: string[], name: string) => launchWriter(f, 'bounded', 'low', flags, name, { written: fileNonce }, 'result.json'); + const place = (receipt: Record) => + writeFile(join(f.dir, 'bounded.receipt.json'), JSON.stringify({ ...receipt, outputPath: join(f.dir, 'bounded.output.md') })); + const run = { ...record('claude', 'pstack:poteto-mode'), commands: ['"$RUNNER/pstack-runner" --provider codex'] }; + // A parent passes no flag for a sheet without the setting line, and the runner's writer is bounded. + const receipt = await launch([], 'bounded.receipt'); + expect([receipt.writerSandbox, (receipt.argv as string[]).includes('workspace-write')]).toEqual(['bounded', true]); + await place(receipt); + expect((await c.assert(run, f)).assertions).toEqual(['runner-ran', 'receipt-complete', 'writer-sandbox:bounded', 'file:result.json']); + + // A full-access writer fails, so does a bounded receipt whose argv carries danger-full-access, and so does a wrong nonce. + await place(await launch(['--writer-sandbox', 'full-access'], 'opted.receipt')); + await expect(c.assert(run, f)).rejects.toThrow('writer-sandbox:full-access'); + await place({ ...receipt, argv: [...receipt.argv as string[], '--sandbox', 'danger-full-access'] }); + await expect(c.assert(run, f)).rejects.toThrow('writer-sandbox:bounded'); + await place(receipt); + await writeFile(join(f.dir, 'bounded-lane/result.json'), JSON.stringify({ written: 'guess' })); + await expect(c.assert(run, f)).rejects.toThrow('bounded-effect-mismatch'); +}); diff --git a/.claude/skills/verify-open-pstack/scripts/cli.test.ts b/.claude/skills/verify-open-pstack/scripts/cli.test.ts index 29a3cc0d..5b2a9747 100644 --- a/.claude/skills/verify-open-pstack/scripts/cli.test.ts +++ b/.claude/skills/verify-open-pstack/scripts/cli.test.ts @@ -73,7 +73,24 @@ test('credential-directory and account options are removed', () => { test('runner routes are explicit, repeatable, and distinct', () => { expect(parse(['run', '--pr', '1', '--runner-route', 'codex:gpt-6.1-sol@max', '--runner-route', 'codex:gpt-6.1-sol@ultra', '--output', '/fresh/run'])) .toMatchObject({ routes: ['codex:gpt-6.1-sol@max', 'codex:gpt-6.1-sol@ultra'] }); - for (const args of [['--runner-route', 'gpt-6.1-sol'], ['--runner-route', 'codex:a@max', '--runner-route', 'codex:a@max']]) { + // Any provider spelling parses; the installed catalog decides whether it exists before a session starts. + expect(parse(['run', '--pr', '1', '--runner-route', 'devin:swe-2@max', '--runner-route', 'devin:swe-1.6@default', '--output', '/fresh/run'])) + .toMatchObject({ routes: ['devin:swe-2@max', 'devin:swe-1.6@default'] }); + // A model may hold a slash, as OpenCode's provider-qualified models do. + expect(parse(['run', '--pr', '1', '--runner-route', 'opencode:opencode/deepseek-v4.1-flash-free@max', '--output', '/fresh/run'])) + .toMatchObject({ routes: ['opencode:opencode/deepseek-v4.1-flash-free@max'] }); + for (const args of [['--runner-route', 'gpt-6.1-sol'], ['--runner-route', 'codex:a@max', '--runner-route', 'codex:a@max'], ['--runner-route', 'Devin:swe-2@max']]) { expect(() => parse(['run', '--pr', '1', '--output', '/fresh/run', ...args])).toThrow(); } }); +test('--wide takes one publishable, nonblank reason and only for run', () => { + const run = ['run', '--pr', '1', '--output', '/fresh/run']; + expect(parse([...run, '--wide', 'Shared references changed every skill'])).toMatchObject({ wide: 'Shared references changed every skill' }); + expect(parse(run)).not.toHaveProperty('wide'); + for (const reason of [' ', 'uses `backticks`', 'has brackets', 'non-ASCII — dash', 'r'.repeat(161)]) { + expect(() => parse([...run, '--wide', reason])).toThrow('Wide reason must be nonblank printable ASCII'); + } + expect(() => parse([...run, '--wide', 'one', '--wide', 'two'])).toThrow('Duplicate option: --wide'); + expect(() => parse([...run, '--wide'])).toThrow('Missing value: --wide'); + expect(() => parse(['doctor', '--output', '/fresh/probe', '--wide', 'reason'])).toThrow('Unknown option: --wide'); +}); diff --git a/.claude/skills/verify-open-pstack/scripts/cli.ts b/.claude/skills/verify-open-pstack/scripts/cli.ts index 0906610c..a395b96b 100644 --- a/.claude/skills/verify-open-pstack/scripts/cli.ts +++ b/.claude/skills/verify-open-pstack/scripts/cli.ts @@ -4,35 +4,48 @@ import { ROUTE } from '../features/recipes.ts'; import { validateRegistry } from './core.ts'; import { doctor, LoginRequired } from './doctor.ts'; import { Publisher } from './github.ts'; +import type { NativeRouting } from './routing.ts'; import { MacDriver } from './harness.ts'; import { command, freshRoot, interruption, interruptCommands, save, session } from './io.ts'; -import { verify } from './verify.ts'; +import { publishable, Refused, verify } from './verify.ts'; type Options = { mode: 'doctor'; output: string; pr: number; selfTest: boolean; candidate: boolean } - | { mode: 'run'; output: string; pr: number; selfTest: boolean; routes: string[]; features: string[] }; + | { mode: 'run'; output: string; pr: number; selfTest: boolean; routes: string[]; features: string[]; wide?: string; routing?: NativeRouting }; export function parse(args: string[]): Options { const mode = args[0]; - if (mode !== 'doctor' && mode !== 'run') throw new Error('Usage: verify.sh doctor [--candidate] --output /fresh/path | run --pr NUMBER --feature HARNESS:FEATURE... [--self-test] [--runner-route PROVIDER:MODEL@EFFORT]... --output /fresh/path'); - let output = '', pr = 0, selfTest = false, candidate = false; + if (mode !== 'doctor' && mode !== 'run') throw new Error('Usage: verify.sh doctor [--candidate] --output /fresh/path | run --pr NUMBER --feature HARNESS:FEATURE... [--self-test] [--runner-route PROVIDER:MODEL@EFFORT]... [--wide REASON] --output /fresh/path'); + let router = ''; const models: NativeRouting['models'] = {}; + let output = '', pr = 0, selfTest = false, candidate = false, wide: string | undefined; const seen = new Set(), routes: string[] = [], features: string[] = []; for (let i = 1; i < args.length; i++) { const arg = args[i]!; - if (arg !== '--runner-route' && arg !== '--feature') { if (seen.has(arg)) throw new Error(`Duplicate option: ${arg}`); seen.add(arg); } + if (arg !== '--runner-route' && arg !== '--feature' && arg !== '--native-model') { if (seen.has(arg)) throw new Error(`Duplicate option: ${arg}`); seen.add(arg); } if (arg === '--self-test' && mode === 'run') selfTest = true; else if (arg === '--candidate' && mode === 'doctor') candidate = true; - else if (arg === '--output' || mode === 'run' && (arg === '--pr' || arg === '--runner-route' || arg === '--feature')) { + else if (arg === '--output' || mode === 'run' && (arg === '--account-router' || arg === '--native-model' || arg === '--pr' || arg === '--runner-route' || arg === '--feature' || arg === '--wide')) { const value = args[++i]; if (!value || value.startsWith('--')) throw new Error(`Missing value: ${arg}`); if (arg === '--output') output = value; - else if (arg === '--runner-route') { + else if (arg === '--account-router') { + if (!value.startsWith('/')) throw new Error('Account router must be an absolute path'); + router = value; + } else if (arg === '--native-model') { + const match = /^(claude|codex):([A-Za-z0-9._-]+)@(low|medium|high|xhigh|max)$/.exec(value); + if (!match || models[match[1] as 'claude' | 'codex']) throw new Error('Native model must be a distinct HARNESS:MODEL@EFFORT'); + models[match[1] as 'claude' | 'codex'] = { model: match[2]!, effort: match[3]! }; + } else if (arg === '--runner-route') { if (!ROUTE.test(value) || routes.includes(value)) throw new Error('Runner route must be a distinct PROVIDER:MODEL@EFFORT'); routes.push(value); } else if (arg === '--feature') { if (!/^(claude|codex):[a-z][a-z0-9:-]+$/.test(value) || features.includes(value)) throw new Error('Feature must be a distinct HARNESS:FEATURE, such as claude:agent-preload:poteto-agent'); features.push(value); + } else if (arg === '--wide') { + if (!value.trim() || !publishable(value)) throw new Error('Wide reason must be nonblank printable ASCII, at most 160 characters, without backticks or angle brackets'); + wide = value; } else { if (!/^[1-9]\d*$/.test(value) || !Number.isSafeInteger(Number(value))) throw new Error('PR must be a positive integer'); pr = Number(value); } } else throw new Error(`Unknown option: ${arg}`); } if (!output || mode === 'run' && !pr) throw new Error('Output and (for run) PR are required'); - if (mode === 'run') return { mode, output, pr, selfTest, routes, features }; + if (!!router !== !!Object.keys(models).length) throw new Error('Account router and native models are required together'); + if (mode === 'run') return { ...(router ? { routing: { router, models } } : {}), mode, output, pr, selfTest, routes, features, ...(wide ? { wide } : {}) }; return { mode, output, pr, selfTest, candidate }; } async function main(): Promise { @@ -53,7 +66,7 @@ async function main(): Promise { else { const publisherRevision = (await command(['git', 'rev-parse', 'HEAD'], { cwd: repository })).trim(); const registry = validateRegistry(JSON.parse(await readFile(join(import.meta.dir, '../features/registry.json'), 'utf8'))); - const receipt = await verify({ pr: options.pr, selfTest: options.selfTest, features: options.features, root, registry, github: new Publisher(), driver: new MacDriver(command, session, options.routes), persist: r => save(join(root!, 'receipt.json'), r), publisherRevision }); + const receipt = await verify({ pr: options.pr, selfTest: options.selfTest, features: options.features, routes: options.routes, wide: options.wide, root, registry, github: new Publisher(), driver: new MacDriver(command, session, options.routes, options.routing), persist: r => save(join(root!, 'receipt.json'), r), publisherRevision }); console.log(`Pinned ${receipt.sha}: live-gate=${receipt.status}; evidence ${receipt.commentUrl}`); } interruption.signal.throwIfAborted(); @@ -61,7 +74,7 @@ async function main(): Promise { } catch (error) { const failure = error instanceof Error ? error.message : String(error); if (root) await save(join(root, 'failure.json'), { result: 'failed', reason: failure }); - if (error instanceof LoginRequired) console.error(failure); + if (error instanceof LoginRequired || error instanceof Refused) console.error(failure); console.error(root ? 'Verification failed; inspect retained failure.json.' : 'Verification failed before evidence could be retained.'); process.exitCode = interrupted === 'SIGINT' ? 130 : interrupted === 'SIGTERM' ? 143 : 1; } finally { diff --git a/.claude/skills/verify-open-pstack/scripts/core.test.ts b/.claude/skills/verify-open-pstack/scripts/core.test.ts index 2f7bc292..35ceea99 100644 --- a/.claude/skills/verify-open-pstack/scripts/core.test.ts +++ b/.claude/skills/verify-open-pstack/scripts/core.test.ts @@ -3,7 +3,7 @@ import { execFileSync } from 'node:child_process'; import { resolve } from 'node:path'; import registryData from '../features/registry.json'; import { classify, completeEvidence, matches, newReceipt, requiredFeatures, requiredHarnesses, validateRegistry } from './core.ts'; -import { type Receipt } from './types.ts'; +import { HARNESSES, type Receipt } from './types.ts'; import codexManifest from '../../../../plugins/pstack/.codex-plugin/plugin.json'; import claudeManifest from '../../../../plugins/pstack/.claude-plugin/plugin.json'; @@ -14,6 +14,7 @@ export function receipt(selfTest = false): Receipt { r.selection = classify([{ filename: 'README.md' }], registry); return r; } +const both = (...features: string[]): string[] => HARNESSES.flatMap(h => features.map(f => `${h}:${f}`)).sort(); export function evidence(r: Receipt): void { for (const harness of requiredHarnesses(r)) { r.installations.push({ harness, sha: r.sha, cliVersion: 'test', pluginVersion: '1.5.0', @@ -50,38 +51,51 @@ describe('registry and ownership', () => { test('markdown instructions are runtime', () => { for (const filename of ['plugins/pstack/skills/architect/SKILL.md', 'plugins/pstack/skills/architect/references/guide.md']) { const s = classify([{ filename }], registry); - expect(s.features).toContain('skill-invocation:architect'); + expect(s.features).toEqual(both('skill-invocation:architect')); expect(s.noRuntime).toBe(false); } }); test('shared references select every dependent skill and executable surface', () => { const s = classify([{ filename: 'plugins/pstack/skills/poteto-mode/references/codex-tools.md' }], registry); expect(s.skills).toEqual([...registry.skills].sort()); - for (const feature of ['setup', 'runner', 'shipped-tools']) expect(s.features).toContain(feature); + for (const feature of both('setup', 'runner', 'shipped-tools', 'hook:startup')) expect(s.features).toContain(feature); + expect(s.features).toContain('claude:agent-preload:poteto-agent'); + expect(s.features).not.toContain('codex:agent-preload:poteto-agent'); + }); + test('Codex skill invocation policy reaches its native skill and rejects unknown metadata', () => { + const path = 'plugins/pstack/skills/create-formal-verification-skill/agents/openai.yaml'; + expect(classify([{ filename: path }], registry).features).toEqual(['codex:skill-invocation:create-formal-verification-skill']); + expect(() => classify([{ filename: path.replace('openai.yaml', 'unknown.yaml') }], registry)).toThrow('Unmapped path'); + expect(() => classify([{ filename: path.replace('create-formal-verification-skill', 'unregistered-skill') }], registry)).toThrow('Unmapped path'); + }); + test('poteto-mode SKILL.md also reaches the Claude agent that preloads it', () => { + const s = classify([{ filename: 'plugins/pstack/skills/poteto-mode/SKILL.md' }], registry); + expect(s.features).toEqual([...both('skill-invocation:poteto-mode'), 'claude:agent-preload:poteto-agent'].sort()); }); const CATALOG = 'plugins/pstack/skills/poteto-mode/references/model-catalog.md'; test('catalog-only changes select catalog, setup and runner and no skill check', () => { for (const filename of [CATALOG, 'plugins/pstack/agents/pstack-opus-max.md', 'plugins/pstack/agents/pstack-fable-low.md', + 'plugins/pstack/agents/pstack-sonnet-max.md', 'plugins/pstack/agents/pstack-opus-5-5-low.md', 'plugins/pstack/skills/poteto-mode/scripts/runner/registry/models.ts', 'plugins/pstack/skills/poteto-mode/scripts/runner/providers/grok.ts']) { const s = classify([{ filename }], registry); expect(s.skills).toEqual([]); - expect(s.features).toEqual(['catalog', 'runner', 'setup']); + expect(s.features).toEqual(both('catalog', 'runner', 'setup')); } }); test('a renamed or deleted catalog path keeps catalog ownership', () => { for (const previous_filename of [CATALOG, 'plugins/pstack/agents/pstack-opus-max.md']) { - expect(classify([{ filename: 'docs/moved.md', previous_filename }], registry).features).toEqual(['catalog', 'runner', 'setup']); + expect(classify([{ filename: 'docs/moved.md', previous_filename }], registry).features).toEqual(both('catalog', 'runner', 'setup')); } }); test('a catalog change mixed with a shared reference or a skill keeps their checks', () => { const shared = classify([{ filename: CATALOG }, { filename: 'plugins/pstack/skills/poteto-mode/references/provider-dispatch.md' }], registry); expect(shared.skills).toEqual([...registry.skills].sort()); - for (const feature of ['catalog', 'setup', 'runner', 'shipped-tools']) expect(shared.features).toContain(feature); + for (const feature of both('catalog', 'setup', 'runner', 'shipped-tools')) expect(shared.features).toContain(feature); const skill = classify([{ filename: 'plugins/pstack/agents/pstack-opus-max.md' }, { filename: 'plugins/pstack/skills/arena/SKILL.md' }], registry); - expect(skill.features).toEqual(['catalog', 'runner', 'setup', 'skill-invocation:arena']); + expect(skill.features).toEqual(both('catalog', 'runner', 'setup', 'skill-invocation:arena')); }); test('an unlisted generated agent fails closed', () => { - expect(() => classify([{ filename: 'plugins/pstack/agents/pstack-sonnet-max.md' }], registry)).toThrow('Unmapped path'); + expect(() => classify([{ filename: 'plugins/pstack/agents/pstack-retired-max.md' }], registry)).toThrow('Unmapped path'); expect(classify([{ filename: 'plugins/pstack/agents/poteto-agent.md' }], registry).skills).toEqual([...registry.skills].sort()); }); test('installed logo changes require only their actual Codex manifest consumer', () => { @@ -91,8 +105,8 @@ describe('registry and ownership', () => { r.selection = classify([{ filename: 'plugins/pstack/assets/logo.png' }], registry); expect(r.selection.noRuntime).toBe(false); expect(r.selection.skills).toEqual([]); - expect(r.selection.features).toEqual(['assets:codex']); - r.selection.features = ['codex:assets:codex']; + expect(r.selection.features).toEqual(['codex:assets:codex']); + r.runFeatures = ['codex:assets:codex']; expect(requiredHarnesses(r)).toEqual(['codex']); expect(requiredFeatures(r, 'claude')).toEqual([]); expect(requiredFeatures(r, 'codex')).toEqual(['assets:codex']); @@ -109,15 +123,30 @@ describe('registry and ownership', () => { expect(() => classify([{ filename: 'README.md', previous_filename: filename }], registry)).toThrow('Unmapped asset consumer'); } }); - test('manifest changes classify as shared runtime', () => { - for (const filename of ['plugins/pstack/.claude-plugin/plugin.json', 'plugins/pstack/.codex-plugin/plugin.json']) { - expect(classify([{ filename }], registry).features).toContain('setup'); + test('manifest changes classify as shared runtime unless they only bump the version', () => { + for (const filename of ['plugins/pstack/.claude-plugin/plugin.json', 'plugins/pstack/.codex-plugin/plugin.json', '.claude-plugin/marketplace.json', '.agents/plugins/marketplace.json']) { + expect(classify([{ filename }], registry).features).toContain('claude:setup'); } + // The Codex marketplace carries no version, so the flag never narrows it. + expect(classify([{ filename: '.agents/plugins/marketplace.json', versionOnly: true }], registry).features).toContain('codex:setup'); + expect(classify([{ filename: 'plugins/pstack/.claude-plugin/plugin.json', versionOnly: true }], registry).features).toEqual(['claude:hook:startup']); + expect(classify([{ filename: '.claude-plugin/marketplace.json', versionOnly: true }], registry).features).toEqual(['claude:hook:startup']); + expect(classify([{ filename: 'plugins/pstack/.codex-plugin/plugin.json', versionOnly: true }], registry).features).toEqual(['codex:hook:startup']); + }); + // The changed files of #199's release commit, e99abd7. + const RELEASE = [{ filename: '.claude-plugin/marketplace.json', versionOnly: true }, { filename: 'CHANGES.md' }, { filename: 'README.md' }, { filename: 'UPSTREAM.md' }, + { filename: 'docs/reference.md' }, { filename: 'plugins/pstack/.claude-plugin/plugin.json', versionOnly: true }, { filename: 'plugins/pstack/.codex-plugin/plugin.json', versionOnly: true }]; + test('a version-only release reaches only the startup hook in both harnesses', () => { + expect(classify(RELEASE, registry).features).toEqual(['claude:hook:startup', 'codex:hook:startup']); + const edited = RELEASE.map(f => f.filename === 'plugins/pstack/.codex-plugin/plugin.json' ? { filename: f.filename } : f); + const s = classify(edited, registry); + expect(s.skills).toEqual([...registry.skills].sort()); + for (const feature of both('setup', 'runner', 'shipped-tools', 'hook:startup')) expect(s.features).toContain(feature); }); test('asset self-test requires both harnesses without fictitious Claude asset proof', () => { const r = receipt(true); r.selection = classify([{ filename: 'plugins/pstack/assets/logo.png' }], registry); - r.selection.features = ['codex:assets:codex']; + r.runFeatures = ['codex:assets:codex']; expect(requiredHarnesses(r)).toEqual(['claude', 'codex']); expect(requiredFeatures(r, 'claude')).toEqual(['project-skill']); expect(requiredFeatures(r, 'codex')).toEqual(['assets:codex', 'project-skill']); @@ -130,22 +159,21 @@ describe('registry and ownership', () => { test('bootstrap changes require its shipped tool consumers, not the unrelated runner', () => { const s = classify([{ filename: 'plugins/pstack/skills/poteto-mode/scripts/bootstrap.ts' }], registry); expect(s.noRuntime).toBe(false); - expect(s.features).toEqual(['shipped-tools']); - expect(s.features).not.toContain('runner'); + expect(s.features).toEqual(both('shipped-tools')); }); test('the transcript resolver belongs to Reflect rather than unrelated shipped tools', () => { const path = 'plugins/pstack/skills/poteto-mode/scripts/resolve-codex-transcript.ts'; for (const file of [{ filename: path }, { filename: path.replace('.ts', '') }, { filename: 'docs/removed-resolver.md', previous_filename: path }]) { const s = classify([file], registry); expect(s.skills).toEqual(['reflect']); - expect(s.features).toEqual(['skill-invocation:reflect']); + expect(s.features).toEqual(both('skill-invocation:reflect')); } expect(() => classify([{ filename: path.replace('.ts', '.py') }], registry)).toThrow('Unmapped path'); expect(classify([{ filename: path.replace('.ts', '.test.ts') }], registry).noRuntime).toBe(true); }); test('rename and deletion preserve old runtime ownership', () => { const s = classify([{ filename: 'docs/old.md', previous_filename: 'plugins/pstack/skills/architect/SKILL.md' }], registry); - expect(s.features).toContain('skill-invocation:architect'); + expect(s.features).toContain('claude:skill-invocation:architect'); expect(classify([{ filename: 'plugins/pstack/skills/architect/SKILL.md' }], registry).noRuntime).toBe(false); }); test('unmapped paths and new executable leaves fail closed', () => { @@ -157,7 +185,7 @@ describe('registry and ownership', () => { const r = receipt(true); r.selection = classify([{ filename: '.claude/skills/verify-open-pstack/scripts/verify.ts' }], registry); expect(r.selection.noRuntime).toBe(false); - expect(r.selection.features).toEqual(['project-skill']); + expect(r.selection.features).toEqual(both('project-skill')); evidence(r); expect(() => completeEvidence(r)).not.toThrow(); }); @@ -167,10 +195,10 @@ describe('registry and ownership', () => { expect(s.features).toEqual([]); }); test('consumed instructions and known verifier enforcement require native project proof', () => { - for (const filename of ['AGENTS.md', 'CLAUDE.md', 'tests/skill-collision-repro.sh']) { + for (const filename of ['AGENTS.md', 'CLAUDE.md', 'tests/skill-collision-repro.sh', '.github/workflows/ci.yml', 'package.json']) { const s = classify([{ filename }], registry); expect(s.noRuntime).toBe(false); - expect(s.features).toEqual(['project-skill']); + expect(s.features).toEqual(both('project-skill')); expect(classify([{ filename: 'docs/moved.md', previous_filename: filename }], registry).noRuntime).toBe(false); } }); @@ -184,7 +212,7 @@ describe('registry and ownership', () => { } }); test('unregistered CI, tests, and scripts fail closed instead of bypassing runtime proof', () => { - for (const filename of ['.github/workflows/ci.yml', '.github/pull_request_template.md', 'tests/new-gate.sh', 'scripts/disable-proof.ts']) { + for (const filename of ['.github/workflows/new-gate.yml', '.github/pull_request_template.md', 'tests/new-gate.sh', 'scripts/disable-proof.ts']) { expect(() => classify([{ filename }], registry)).toThrow(); expect(() => classify([{ filename: 'README.md', previous_filename: filename }], registry)).toThrow(); } @@ -210,7 +238,7 @@ describe('receipt and evidence boundaries', () => { test('each selected skill must have its own observation', () => { const r = receipt(); r.selection = classify(registry.skills.slice(0, 2).map(s => ({ filename: `plugins/pstack/skills/${s}/SKILL.md` })), registry); - r.selection.features = registry.skills.slice(0, 2).map(s => `claude:skill-invocation:${s}`); + r.runFeatures = registry.skills.slice(0, 2).map(s => `claude:skill-invocation:${s}`); evidence(r); expect(() => completeEvidence(r)).not.toThrow(); r.observations.pop(); diff --git a/.claude/skills/verify-open-pstack/scripts/core.ts b/.claude/skills/verify-open-pstack/scripts/core.ts index f12f8d07..46c36deb 100644 --- a/.claude/skills/verify-open-pstack/scripts/core.ts +++ b/.claude/skills/verify-open-pstack/scripts/core.ts @@ -18,26 +18,36 @@ export function matches(path: string, pattern: string): boolean { return pattern.endsWith('/**') ? path.startsWith(pattern.slice(0, -2)) : path === pattern; } +/** Manifests whose version-only change reaches only the startup hook of the harness that reads them. */ +export const VERSION_MANIFESTS: Readonly> = { + 'plugins/pstack/.claude-plugin/plugin.json': 'claude', '.claude-plugin/marketplace.json': 'claude', 'plugins/pstack/.codex-plugin/plugin.json': 'codex', +}; + +/** Return every `:` the changed paths can reach. */ export function classify(files: ChangedFile[], registry: Registry): Selection { const paths = [...new Set(files.flatMap(f => [f.filename, ...(f.previous_filename ? [f.previous_filename] : [])]))].sort(); + const versionOnly = new Set(files.filter(f => f.versionOnly && !f.previous_filename).map(f => f.filename)); const skills = new Set(), features = new Set(); + const both = (...names: string[]): void => names.forEach(f => HARNESSES.forEach(h => features.add(`${h}:${f}`))); for (const path of paths) { if (!path || path.startsWith('/') || path.split('/').includes('..')) throw new Error(`Unsafe path: ${path}`); + if (versionOnly.has(path) && Object.hasOwn(VERSION_MANIFESTS, path)) { features.add(`${VERSION_MANIFESTS[path]}:hook:startup`); continue; } let covered = false; if (registry.assets.some(p => matches(path, p))) { if (path !== 'plugins/pstack/assets/logo.png') throw new Error(`Unmapped asset consumer: ${path}`); - features.add('assets:codex'); covered = true; + features.add('codex:assets:codex'); covered = true; } // Catalog paths change only model and provider facts, so they skip shared expansion and the skill match. const catalog = registry.catalog.some(p => matches(path, p)); - if (catalog) { ['catalog', 'setup', 'runner'].forEach(f => features.add(f)); covered = true; } + if (catalog) { both('catalog', 'setup', 'runner'); covered = true; } if (!catalog && registry.shared.some(p => matches(path, p))) { registry.skills.forEach(s => skills.add(s)); - ['setup', 'runner', 'shipped-tools'].forEach(f => features.add(f)); + both('setup', 'runner', 'shipped-tools', 'hook:startup'); + features.add('claude:agent-preload:poteto-agent'); covered = true; } for (const [key, feature] of [['setup', 'setup'], ['runner', 'runner'], ['tools', 'shipped-tools'], ['project', 'project-skill']] as const) { - if (registry[key].some(p => matches(path, p))) { features.add(feature); covered = true; } + if (registry[key].some(p => matches(path, p))) { both(feature); covered = true; } } // Reflect's Codex transcript lookup runs this launcher and script; no other skill consumes them. if (/^plugins\/pstack\/skills\/poteto-mode\/scripts\/resolve-codex-transcript(?:\.ts)?$/.test(path)) { skills.add('reflect'); covered = true; } @@ -47,11 +57,16 @@ export function classify(files: ChangedFile[], registry: Registry): Selection { if (leaf === 'SKILL.md' || leaf.startsWith('references/') || leaf.startsWith('playbooks/')) { skills.add(skill[1]!); covered = true; } + if (leaf === 'agents/openai.yaml') { + features.add(`codex:skill-invocation:${skill[1]}`); covered = true; + } + // poteto-agent preloads poteto-mode's SKILL.md. + if (skill[1] === 'poteto-mode' && leaf === 'SKILL.md') features.add('claude:agent-preload:poteto-agent'); } // Runtime ownership always takes precedence over documentation exemptions. if (!covered && !registry.nonRuntime.some(p => matches(path, p))) throw new Error(`Unmapped path: ${path}`); } - skills.forEach(s => features.add(`skill-invocation:${s}`)); + skills.forEach(s => both(`skill-invocation:${s}`)); return { paths, skills: [...skills].sort(), features: [...features].sort(), noRuntime: !features.size }; } @@ -59,12 +74,12 @@ export function newReceipt(pr: number, sha: string, base: string, selfTest: bool if (!Number.isSafeInteger(pr) || pr < 1 || !/^[a-f0-9]{40}$/.test(sha) || !/^[a-f0-9]{40}$/.test(base)) throw new Error('Invalid PR/SHA'); if (publisherRevision !== undefined && !/^[a-f0-9]{40}$/.test(publisherRevision)) throw new Error('Invalid publisher revision'); return { schema: 1, repo: REPO, pr, sha, base, selfTest, artifactRoot, ...(publisherRevision ? { publisherRevision } : {}), phase: 'resolve', - selection: { paths: [], skills: [], features: [], noRuntime: false }, installations: [], observations: [], + selection: { paths: [], skills: [], features: [], noRuntime: false }, runFeatures: [], sessions: 0, installations: [], observations: [], started: new Date().toISOString(), cleanup: 'Run state retained; operator owns cleanup.' }; } export function requiredFeatures(receipt: Receipt, harness: Harness): string[] { - const named = receipt.selection.features.filter(f => f.startsWith(`${harness}:`)).map(f => f.slice(harness.length + 1)); + const named = receipt.runFeatures.filter(f => f.startsWith(`${harness}:`)).map(f => f.slice(harness.length + 1)); return [...new Set([...named, ...(receipt.selfTest ? ['project-skill'] : [])])]; } diff --git a/.claude/skills/verify-open-pstack/scripts/formal.test.ts b/.claude/skills/verify-open-pstack/scripts/formal.test.ts new file mode 100644 index 00000000..42526d5b --- /dev/null +++ b/.claude/skills/verify-open-pstack/scripts/formal.test.ts @@ -0,0 +1,32 @@ +import { afterEach, expect, test } from 'bun:test'; +import { mkdtemp, rm, writeFile } from 'node:fs/promises'; +import { tmpdir } from 'node:os'; +import { join } from 'node:path'; +import { command } from './io.ts'; +import { createFormal, maintainFormal } from '../features/formal.ts'; +import type { Fixture } from '../features/recipes.ts'; +import type { SessionRecord } from './types.ts'; +const roots:string[]=[]; +afterEach(async()=>{for(const root of roots.splice(0)) await rm(root,{recursive:true,force:true});}); +async function fixture():Promise { + const dir=await mkdtemp(join(tmpdir(),'formal-proof-'));roots.push(dir); + return {dir,harness:'codex',workspace:dir,location:dir,configHome:dir,run:command}; +} +function record(skill:string):SessionRecord { return {harness:'codex',exitCode:0,started:Date.now(),skills:[`pstack:${skill}`],commands:[],tools:[],context:[]}; } +test('maintenance exposes a real bug while the stale model passes; no-op maintenance fails',async()=>{ + const f=await fixture(),c=maintainFormal();await c.prepare!(f); + expect(await command(['bun','verification/check.mjs'],{cwd:f.dir})).toContain('"verdict":"pass"'); + await expect(command(['bun','replay.mjs'],{cwd:f.dir})).rejects.toThrow(); + await expect(c.assert(record('maintain-formal-verification-skill'),f)).rejects.toThrow('still-stale'); + await writeFile(join(f.dir,'verification/model.json'),JSON.stringify({clients:2,limit:1,splitReservation:true})); + await expect(command(['bun','verification/check.mjs'],{cwd:f.dir})).rejects.toThrow(); + await expect(c.assert(record('maintain-formal-verification-skill'),f)).rejects.toThrow('before-after-not-observed'); + await writeFile(join(f.dir,'capacity.mjs'),'export async function reserve() {}'); + await expect(c.assert(record('maintain-formal-verification-skill'),f)).rejects.toThrow('product-changed'); +}); +test('creation begins with passing code but requires mutation evidence',async()=>{ + const f=await fixture(),c=createFormal();await c.prepare!(f); + expect(await command(['bun','replay.mjs'],{cwd:f.dir})).toContain('"used":1'); + expect(await command(['bun','verification/check.mjs'],{cwd:f.dir})).toContain('"verdict":"pass"'); + await expect(c.assert(record('create-formal-verification-skill'),f)).rejects.toThrow('mutation-not-observed'); +}); diff --git a/.claude/skills/verify-open-pstack/scripts/github.ts b/.claude/skills/verify-open-pstack/scripts/github.ts index a6d452c6..da03b8d7 100644 --- a/.claude/skills/verify-open-pstack/scripts/github.ts +++ b/.claude/skills/verify-open-pstack/scripts/github.ts @@ -1,6 +1,7 @@ import { mkdtemp, rm } from 'node:fs/promises'; import { tmpdir } from 'node:os'; import { join } from 'node:path'; +import { VERSION_MANIFESTS } from './core.ts'; import { command, type Command } from './io.ts'; import { REPO, type ChangedFile, type GitHub, type Pull } from './types.ts'; @@ -18,6 +19,27 @@ export function parseChangedFiles(output: string): ChangedFile[] { if (new Set(files.map(f => f.filename)).size !== files.length) throw new Error('Duplicate changed-file diff'); return files; } +type Json = Record; +const object = (value: unknown): value is Json => typeof value === 'object' && value !== null && !Array.isArray(value); +/** The manifest's version strings, and the manifest without them. */ +function versions(path: string, text: string): { versions: unknown[]; rest: Json } | undefined { + let value: unknown; + try { value = JSON.parse(text); } catch { return undefined; } + if (!object(value)) return undefined; + if (path !== '.claude-plugin/marketplace.json') { const { version, ...rest } = value; return { versions: [version], rest }; } + if (!Array.isArray(value.plugins)) return undefined; + const found: unknown[] = []; + const plugins = value.plugins.map(plugin => { if (!object(plugin)) return plugin; const { version, ...rest } = plugin; found.push(version); return rest; }); + return { versions: found, rest: { ...value, plugins } }; +} +/** True when the two manifest texts differ only in version strings, and at least one changed. */ +export function versionOnly(path: string, before: string, after: string): boolean { + const a = versions(path, before), b = versions(path, after); + if (!a || !b || !Bun.deepEquals(a.rest, b.rest)) return false; + const changed = a.versions.flatMap((v, i) => v === b.versions[i] ? [] : [[v, b.versions[i]]]); + return changed.length > 0 && changed.every(pair => pair.every(v => typeof v === 'string')); +} + export class Publisher implements GitHub { constructor(private run: Command = command) {} private async api(path: string, args: string[] = [], allowInterrupted = false): Promise { @@ -39,7 +61,19 @@ export class Publisher implements GitHub { if ((await this.run(['git', 'rev-parse', '--verify', `${sha}^{commit}`], { cwd: repository, env })).trim() !== sha) throw new Error('Pinned diff commit unavailable'); } const output = await this.run(['git', 'diff', '--no-ext-diff', '--no-textconv', '--name-status', '-z', '--find-renames', `${base}...${head}`, '--'], { cwd: repository, env }); - return parseChangedFiles(output); + const files = parseChangedFiles(output); + // The three-dot diff compares the merge base to the head, so the version check does too. + const mergeBase = (await this.run(['git', 'merge-base', base, head], { cwd: repository, env })).trim(); + const blob = async (commit: string, path: string): Promise<{ mode: string; text: string } | undefined> => { + const entry = /^(100644|100755) blob ([a-f0-9]{40})\t/.exec(await this.run(['git', 'ls-tree', '-z', commit, '--', path], { cwd: repository, env })); + return entry ? { mode: entry[1]!, text: await this.run(['git', 'cat-file', 'blob', entry[2]!], { cwd: repository, env }) } : undefined; + }; + for (const file of files) { + if (file.previous_filename || !Object.hasOwn(VERSION_MANIFESTS, file.filename)) continue; + const before = await blob(mergeBase, file.filename), after = await blob(head, file.filename); + if (before && after && before.mode === after.mode && versionOnly(file.filename, before.text, after.text)) file.versionOnly = true; + } + return files; } finally { await rm(repository, { recursive: true, force: true }); } } async comment(pr: number, body: string): Promise { diff --git a/.claude/skills/verify-open-pstack/scripts/harness.test.ts b/.claude/skills/verify-open-pstack/scripts/harness.test.ts index da101e2c..24afaa49 100644 --- a/.claude/skills/verify-open-pstack/scripts/harness.test.ts +++ b/.claude/skills/verify-open-pstack/scripts/harness.test.ts @@ -12,6 +12,7 @@ import { evidence, verify } from './verify.ts'; import { command, freshRoot, isolatedEnv, retainedFile, treeHash, type Command, type Session } from './io.ts'; import { REPO, type Harness, type Installation, type Receipt } from './types.ts'; const roots: string[] = []; +const gitEnv = { PATH: process.env.PATH ?? '', GIT_CONFIG_GLOBAL: '/dev/null', GIT_CONFIG_NOSYSTEM: '1' }; async function fixture(): Promise { const root = await realpath(await mkdtemp(join(tmpdir(), 'pstack-test-'))); roots.push(root); return root; } const originalEnv = { HOME: process.env.HOME, USER: process.env.USER, LOGNAME: process.env.LOGNAME, CLAUDE_CONFIG_DIR: process.env.CLAUDE_CONFIG_DIR, CODEX_HOME: process.env.CODEX_HOME }; @@ -125,10 +126,10 @@ describe('isolated harness boundaries', () => { await writeFile(join(repository, '.claude/skills/verify-open-pstack/SKILL.md'), 'pinned project skill'); await mkdir(join(repository, '.agents/skills'), { recursive: true }); await symlink('../../.claude/skills/verify-open-pstack', join(repository, '.agents/skills/verify-open-pstack')); - await command(['git', 'init', repository]); - await command(['git', 'add', '.'], { cwd: repository }); - await command(['git', '-c', 'user.name=Fixture', '-c', 'user.email=fixture@example.com', '-c', 'commit.gpgsign=false', 'commit', '-m', 'fixture'], { cwd: repository }); - const sha = (await command(['git', 'rev-parse', 'HEAD'], { cwd: repository })).trim(), actualGit: Command = command; + await command(['git', 'init', repository], { env: gitEnv }); + await command(['git', 'add', '.'], { cwd: repository, env: gitEnv }); + await command(['git', '-c', 'user.name=Fixture', '-c', 'user.email=fixture@example.com', '-c', 'commit.gpgsign=false', 'commit', '-m', 'fixture'], { cwd: repository, env: gitEnv }); + const sha = (await command(['git', 'rev-parse', 'HEAD'], { cwd: repository, env: gitEnv })).trim(), actualGit: Command = command; await mkdir(operatorHome, { mode: 0o700 }); const platform = Object.getOwnPropertyDescriptor(process, 'platform')!; const previous = { HOME: process.env.HOME, USER: process.env.USER, LOGNAME: process.env.LOGNAME }; @@ -237,7 +238,7 @@ describe('isolated harness boundaries', () => { await writeFile(join(config, instructions), original); if (harness === 'codex') await writeFile(join(config, 'pstack-models.md'), original); const receipt = newReceipt(111, 'a'.repeat(40), 'b'.repeat(40), false, root); - receipt.selection = { paths: [], skills: [], features: [`${harness}:setup`], noRuntime: false }; + receipt.runFeatures = [`${harness}:setup`]; receipt.installations = [{ harness, home, location: join(workspace, 'plugins/pstack'), sha: receipt.sha, cliVersion: 'test', pluginVersion: 'test', treeHash: await treeHash(join(workspace, 'plugins/pstack')), sourceHash: await sourceDigest(workspace) }]; const driver = new MacDriver(command, async args => { expect(args[0]).toBe(harness); @@ -301,7 +302,7 @@ async function candidate(harness: Harness): Promise<{ root: string; installation } function selected(root: string, installation: Installation, features: string[]): Receipt { const receipt = newReceipt(111, 'a'.repeat(40), 'b'.repeat(40), false, root); - receipt.selection = { paths: [], skills: [], features: features.map(feature => `${installation.harness}:${feature}`), noRuntime: false }; + receipt.runFeatures = features.map(feature => `${installation.harness}:${feature}`); receipt.installations = [installation]; return receipt; } diff --git a/.claude/skills/verify-open-pstack/scripts/harness.ts b/.claude/skills/verify-open-pstack/scripts/harness.ts index 41351eb4..13c84cf0 100644 --- a/.claude/skills/verify-open-pstack/scripts/harness.ts +++ b/.claude/skills/verify-open-pstack/scripts/harness.ts @@ -5,6 +5,7 @@ import { requiredFeatures, requiredHarnesses } from './core.ts'; import { command, isolatedEnv, retainedFile, save, session, treeHash, type Command, type Session } from './io.ts'; import { doctor } from './doctor.ts'; import { assertCodexAuth, linkCodexAuth, removeCodexHome, restoreSetup, snapshotSetup, type SetupSnapshot } from './isolation.ts'; +import { routedSession, type NativeRouting } from './routing.ts'; import { sourceDigest, sourceHash } from './provenance.ts'; import { publishable } from './verify.ts'; import { REPO, type Driver, type Harness, type Installation, type Observation, type Receipt, type SessionRecord } from './types.ts'; @@ -80,11 +81,13 @@ export class MacDriver implements Driver { private codexHomes: string[] = []; private setup?: SetupSnapshot; private restorationFailed = false; - constructor(private run: Command = command, private record: Session = session, private routes: string[] = []) {} + private routedIncomplete = false; + constructor(private run: Command = command, private record: Session = session, private routes: string[] = [], private routing?: NativeRouting) {} async cleanup(): Promise { if (this.setup) await restoreSetup(this.setup); if (this.restorationFailed) throw new Error('Setup restoration failed; publication forbidden'); - for (const home of this.codexHomes) await removeCodexHome(home); + if (this.routedIncomplete) throw new Error('Routed session did not confirm successful cleanup; candidate state retained'); + for (const home of this.codexHomes) await removeCodexHome(home, !!this.routing); this.codexHomes = []; } private candidate(home: string): Command { @@ -118,7 +121,7 @@ export class MacDriver implements Driver { const candidate = join(workspace, 'plugins/pstack'), expected = await treeHash(candidate, ['skills/poteto-mode/scripts/node_modules']); if (await sourceDigest(workspace) !== pinnedSource) throw new Error('Candidate files differ from trusted pinned source'); let location = candidate; - await linkCodexAuth(home); + if (!this.routing) await linkCodexAuth(home); this.codexHomes.push(home); if (harness === 'codex') { const added = await candidateRun(['codex', 'plugin', 'marketplace', 'add', workspace, '--json'], { env, cwd: home }); @@ -153,6 +156,14 @@ export class MacDriver implements Driver { // Setup rewrites the harness's model sheet, so it runs last and later checks see the default configuration. features: requiredFeatures(receipt, installation.harness).sort((a, b) => Number(setupFeature(a)) - Number(setupFeature(b))) .map(feature => ({ feature, cases: recipe(feature, installation.harness, this.routes) })) })); + if (this.routing) { + for (const { installation, features } of plans) { + if (!this.routing.models[installation.harness]) throw new Error(`native-model-missing:${installation.harness}`); + for (const { feature, cases } of features) { + if (feature === 'runner' || setupFeature(feature) || feature.startsWith('agent-preload:') || cases.some(c => c.codexSandbox || c.claudePermission)) throw new Error(`unsupported-routed-recipe:${feature}`); + } + } + } const observations: Observation[] = []; for (const { installation, features } of plans) { const { harness, home } = installation, workspace = join(home, 'workspace'); @@ -170,7 +181,12 @@ export class MacDriver implements Driver { await c.prepare?.(fixture); const stdout = join(records, `${c.id}.jsonl`), stderr = join(records, `${c.id}.stderr`), cwd = c.cwd === 'workspace' ? workspace : dir; const started = Date.now(); - const exitCode = await this.record(launch(harness, workspace, { cwd, addDirs: cwd === dir ? [] : [dir], codexSandbox: c.codexSandbox, claudePermission: c.claudePermission }), + if (this.routing) this.routedIncomplete = true; + const routed = this.routing ? await routedSession(this.routing, this.record, { + harness, home, workspace, cwd, sha: receipt.sha, prompt: c.prompt(fixture), stdout, stderr, + }) : undefined; + if (routed) { this.routedIncomplete = false; evidence.push(...routed.evidence); } + const exitCode = routed?.exitCode ?? await this.record(launch(harness, workspace, { cwd, addDirs: cwd === dir ? [] : [dir], codexSandbox: c.codexSandbox, claudePermission: c.claudePermission }), { cwd, env, input: sessionInput(harness, c.prompt(fixture)), stdout, stderr }); evidence.push(stdout, ...(await stat(stderr)).size ? [stderr] : []); let record: SessionRecord, result; @@ -201,7 +217,7 @@ export class MacDriver implements Driver { observed: publishable(passed) ? passed : `passed ${assertions.length} assertions; ids retained privately`, transcript: transcript!.path, transcriptHash: transcript!.sha256, artifacts }); } - if (harness === 'codex') await assertCodexAuth(home); + if (harness === 'codex' && !this.routing) await assertCodexAuth(home); if (await treeHash(installation.location, ['skills/poteto-mode/scripts/node_modules']) !== installation.treeHash) throw new Error('Installed tree changed during exercise'); if (await sourceDigest(workspace) !== installation.sourceHash) throw new Error('Pinned project/plugin source changed during exercise'); } diff --git a/.claude/skills/verify-open-pstack/scripts/io.ts b/.claude/skills/verify-open-pstack/scripts/io.ts index e3ec7f0b..ace3a3ad 100644 --- a/.claude/skills/verify-open-pstack/scripts/io.ts +++ b/.claude/skills/verify-open-pstack/scripts/io.ts @@ -12,10 +12,13 @@ export function interruptCommands(signal: 'SIGINT' | 'SIGTERM'): Promise { interrupting = (async () => { const children = [...activeCommands]; for (const [pid] of children) { - try { process.kill(-pid, 'SIGKILL'); } + try { process.kill(-pid, 'SIGTERM'); } catch (error) { if ((error as NodeJS.ErrnoException).code !== 'ESRCH') throw error; } } - await Promise.all(children.map(([, exited]) => exited)); + await Promise.all(children.map(async ([pid, exited]) => { + const timer = setTimeout(() => { try { process.kill(-pid, 'SIGKILL'); } catch {} }, 15000); + try { await exited; } finally { clearTimeout(timer); } + })); })(); return interrupting; } diff --git a/.claude/skills/verify-open-pstack/scripts/isolation.ts b/.claude/skills/verify-open-pstack/scripts/isolation.ts index 1723e5ff..5c85e3e4 100644 --- a/.claude/skills/verify-open-pstack/scripts/isolation.ts +++ b/.claude/skills/verify-open-pstack/scripts/isolation.ts @@ -32,8 +32,10 @@ export async function assertCodexAuth(home: string): Promise { } } -export async function removeCodexHome(home: string): Promise { - await assertCodexAuth(home); +export async function removeCodexHome(home: string, routed = false): Promise { + if (routed) { + if (await lstat(join(await codexDirectory(home), 'auth.json')).catch(() => undefined)) throw new Error('Routed authentication remains; retain candidate state for inspection'); + } else await assertCodexAuth(home); await rm(await codexDirectory(home), { recursive: true }); } diff --git a/.claude/skills/verify-open-pstack/scripts/provenance.test.ts b/.claude/skills/verify-open-pstack/scripts/provenance.test.ts index d8aae7c0..a4da99fa 100644 --- a/.claude/skills/verify-open-pstack/scripts/provenance.test.ts +++ b/.claude/skills/verify-open-pstack/scripts/provenance.test.ts @@ -9,6 +9,7 @@ const roots: string[] = []; const project = '.claude/skills/verify-open-pstack'; const plugin = 'plugins/pstack'; const alias = '.agents/skills/verify-open-pstack'; +const gitEnv = { PATH: process.env.PATH ?? '', GIT_CONFIG_GLOBAL: '/dev/null', GIT_CONFIG_NOSYSTEM: '1' }; afterEach(async () => { for (const root of roots.splice(0)) await rm(root, { recursive: true, force: true }); }); async function fixture(): Promise<{ root: string; sha: string }> { const root = await realpath(await mkdtemp(join(tmpdir(), 'pstack-provenance-'))); roots.push(root); @@ -24,8 +25,8 @@ async function fixture(): Promise<{ root: string; sha: string }> { ['git', 'init', '--quiet'], ['git', 'config', 'user.name', 'Provenance Test'], ['git', 'config', 'user.email', 'test@example.invalid'], ['git', 'add', '.'], ['git', '-c', 'commit.gpgsign=false', 'commit', '--quiet', '-m', 'fixture'], - ]) await command(args, { cwd: root }); - return { root, sha: (await command(['git', 'rev-parse', 'HEAD'], { cwd: root })).trim() }; + ]) await command(args, { cwd: root, env: gitEnv }); + return { root, sha: (await command(['git', 'rev-parse', 'HEAD'], { cwd: root, env: gitEnv })).trim() }; } test('real pinned source remains stable after both permitted dependency bootstraps, including symlinks', async () => { @@ -54,7 +55,7 @@ test('plugin code mutation, executable-bit changes and missing source fail again await writeFile(join(root, plugin, 'plugin.json'), '{}\n'); await expect(sourceHash(root, sha)).rejects.toThrow('Source differs from pinned Git tree'); await writeFile(join(root, plugin, 'plugin.json'), '{"version":"1"}\n'); - await command(['git', 'config', 'core.filemode', 'false'], { cwd: root }); + await command(['git', 'config', 'core.filemode', 'false'], { cwd: root, env: gitEnv }); await chmod(join(root, project, 'scripts/verify.sh'), 0o644); await expect(sourceHash(root, sha)).rejects.toThrow('Source differs from pinned Git tree'); await chmod(join(root, project, 'scripts/verify.sh'), 0o755); @@ -169,9 +170,9 @@ test('HEAD must match exact pinned source and committed dependency directories c await expect(sourceHash(root, 'HEAD')).rejects.toThrow('exact pinned SHA'); await mkdir(join(root, project, 'node_modules')); await writeFile(join(root, project, 'node_modules/committed.ts'), 'tracked runtime\n'); - await command(['git', 'add', '.'], { cwd: root }); - await command(['git', '-c', 'commit.gpgsign=false', 'commit', '--quiet', '-m', 'tracked dependencies'], { cwd: root }); - const changed = (await command(['git', 'rev-parse', 'HEAD'], { cwd: root })).trim(); + await command(['git', 'add', '.'], { cwd: root, env: gitEnv }); + await command(['git', '-c', 'commit.gpgsign=false', 'commit', '--quiet', '-m', 'tracked dependencies'], { cwd: root, env: gitEnv }); + const changed = (await command(['git', 'rev-parse', 'HEAD'], { cwd: root, env: gitEnv })).trim(); await expect(sourceHash(root, sha)).rejects.toThrow('HEAD differs'); await expect(sourceHash(root, changed)).rejects.toThrow('Pinned source files missing'); }); diff --git a/.claude/skills/verify-open-pstack/scripts/routing.test.ts b/.claude/skills/verify-open-pstack/scripts/routing.test.ts new file mode 100644 index 00000000..d0744baf --- /dev/null +++ b/.claude/skills/verify-open-pstack/scripts/routing.test.ts @@ -0,0 +1,32 @@ +import { afterEach, expect, test } from 'bun:test'; +import { mkdtemp, readFile, rm, writeFile } from 'node:fs/promises'; +import { tmpdir } from 'node:os'; +import { join } from 'node:path'; +import { parse } from './cli.ts'; +import { routedSession } from './routing.ts'; +import type { Session } from './io.ts'; +const roots:string[]=[]; +afterEach(async()=>{for(const root of roots.splice(0)) await rm(root,{recursive:true,force:true});}); +test('router requires explicit native models without silent defaults',()=>{ + expect(()=>parse(['run','--pr','2','--output','/proof','--account-router','/router'])).toThrow('required together'); + expect(()=>parse(['run','--pr','2','--output','/proof','--native-model','claude:opus@high'])).toThrow('required together'); + const p=parse(['run','--pr','2','--output','/proof','--account-router','/router','--native-model','claude:opus@high']); + expect(p.mode==='run' && p.routing?.models.claude?.model).toBe('opus'); +}); +test('native proof rejects an incomplete admission receipt',async()=>{ + const root=await mkdtemp(join(tmpdir(),'routed-proof-'));roots.push(root); + for(const terminal of [true,false]) { + const stdout=join(root,`${terminal}.jsonl`); + const run:Session=async(args,options)=>{ + expect(args[0]).toBe('/trusted/account-router');expect(args).toContain('/trusted/pstack-verify-run'); + expect(args).toContain('gpt-6-astra');expect(args).not.toContain('--sandbox');expect(options.input).toBe(''); + const req=args[args.indexOf('--request')+1]!,data=JSON.parse(await readFile(req,'utf8')); + expect(data.prompt).toBe('fixture prompt'); + await writeFile(`${req}.account.json`,JSON.stringify({terminal,attempt:'a',identity:'selected'})); + await writeFile(`${req}.receipt.json`,JSON.stringify({terminal:true,auth_link_restored:true,attempt:'a',identity:'selected',request_sha256:args.at(-1),sha:data.sha,model:'gpt-6-astra',effort:'xhigh',provider:'codex',cwd:data.cwd,exit_code:0})); + return 0; + }; + const proof=routedSession({router:'/trusted/account-router',models:{codex:{model:'gpt-6-astra',effort:'xhigh'}}},run,{harness:'codex',home:root,workspace:root,cwd:root,sha:'a'.repeat(40),prompt:'fixture prompt',stdout,stderr:`${stdout}.err`}); + if(terminal) expect((await proof).evidence).toHaveLength(3);else await expect(proof).rejects.toThrow('not-successfully-completed'); + } +}); diff --git a/.claude/skills/verify-open-pstack/scripts/routing.ts b/.claude/skills/verify-open-pstack/scripts/routing.ts new file mode 100644 index 00000000..199bf831 --- /dev/null +++ b/.claude/skills/verify-open-pstack/scripts/routing.ts @@ -0,0 +1,27 @@ +import { readFile } from 'node:fs/promises'; +import { join } from 'node:path'; +import { save, sha256, type Session } from './io.ts'; +import type { Harness } from './types.ts'; +export interface NativeRouting { router: string; models: Partial> } +/** Account selection precedes auth linking in the harness-owned native launcher. */ +export async function routedSession(routing: NativeRouting, record: Session, input: { + harness: Harness; home: string; workspace: string; cwd: string; sha: string; prompt: string; stdout: string; stderr: string; +}): Promise<{ exitCode: number; evidence: string[] }> { + const descriptor = routing.models[input.harness]; + if (!descriptor) throw new Error(`native-model-missing:${input.harness}`); + const request = `${input.stdout}.request.json`, account = `${request}.account.json`, receipt = `${request}.receipt.json`; + await save(request, { schema: 1, state: input.home, workspace: input.workspace, cwd: input.cwd, sha: input.sha, prompt: input.prompt }); + const digest = sha256(await readFile(request)); + const args = [routing.router, 'run', '--receipt', account, '--', join(routing.router, '..', 'pstack-verify-run'), + '--provider', input.harness, '--model', descriptor.model, '--effort', descriptor.effort, '--request', request, '--sha256', digest]; + // Only the trusted launcher receives host admission context. It constructs the candidate environment. + const env = Object.fromEntries(Object.entries(process.env).filter((entry): entry is [string, string] => entry[1] !== undefined)); + const exitCode = await record(args, { cwd: input.cwd, env, input: '', stdout: input.stdout, stderr: input.stderr }); + const admitted = JSON.parse(await readFile(account, 'utf8')), completed = JSON.parse(await readFile(receipt, 'utf8')); + if (admitted.terminal !== true || completed.terminal !== true || completed.auth_link_restored !== true || + completed.attempt !== admitted.attempt || completed.identity !== admitted.identity || + completed.request_sha256 !== digest || completed.sha !== input.sha || completed.model !== descriptor.model || + completed.effort !== descriptor.effort || completed.provider !== input.harness || completed.cwd !== input.cwd || + completed.exit_code !== 0 || exitCode !== 0) throw new Error('native-session-not-successfully-completed'); + return { exitCode, evidence: [request, account, receipt] }; +} diff --git a/.claude/skills/verify-open-pstack/scripts/runner.test.ts b/.claude/skills/verify-open-pstack/scripts/runner.test.ts index 88a40fb7..0e710d87 100644 --- a/.claude/skills/verify-open-pstack/scripts/runner.test.ts +++ b/.claude/skills/verify-open-pstack/scripts/runner.test.ts @@ -30,7 +30,7 @@ async function launch(f: Fixture, prompt: string): Promise { } test('the runner recipe adds one linked-worktree writer case for a Codex parent only', async () => { - expect(recipe('runner', 'claude').map(c => c.id)).toEqual(['route-1']); + expect(recipe('runner', 'claude').map(c => c.id)).toEqual(['route-1', 'full-access-writer', 'bounded-writer']); expect(recipe('runner', 'codex').map(c => c.id)).toEqual(['route-1', 'linked-worktree-writer']); const dir = await realpath(await mkdtemp(join(tmpdir(), 'pstack-runner-recipe-'))); roots.push(dir); diff --git a/.claude/skills/verify-open-pstack/scripts/types.ts b/.claude/skills/verify-open-pstack/scripts/types.ts index 80f66076..0ffbf561 100644 --- a/.claude/skills/verify-open-pstack/scripts/types.ts +++ b/.claude/skills/verify-open-pstack/scripts/types.ts @@ -1,4 +1,4 @@ -export const REPO = 'ericlitman/open-pstack'; +export const REPO = 'fitziboy3-source/open-pstack'; export const HARNESSES = ['claude', 'codex'] as const; export type Harness = typeof HARNESSES[number]; export type Phase = 'resolve' | 'classify' | 'prepare' | 'exercise' | 'publish' | 'failed' | 'head-moved'; @@ -6,7 +6,11 @@ export interface Pull { number: number; head: { sha: string }; base: { sha: string }; state: string; headRepo: string; } -export interface ChangedFile { filename: string; previous_filename?: string } +export interface ChangedFile { + filename: string; previous_filename?: string; + /** Set by the publisher when a manifest change only bumps its version strings. */ + versionOnly?: boolean; +} export interface Registry { skills: string[]; shared: string[]; @@ -31,7 +35,13 @@ export interface Installation { } export interface Receipt { schema: 1; repo: string; pr: number; sha: string; base: string; phase: Phase; - selection: Selection; installations: Installation[]; observations: Observation[]; + /** `features` holds every `:` the changed paths reach. */ + selection: Selection; + /** The features this run exercises: the named ones plus automatic project proof. */ + runFeatures: string[]; + /** Headless sessions the run plans, and the operator's reason for exceeding the limit. */ + sessions: number; wide?: string; + installations: Installation[]; observations: Observation[]; selfTest: boolean; artifactRoot: string; publisherRevision?: string; failure?: string; commentUrl?: string; status?: 'success' | 'failure'; cleanup: string; started: string; } diff --git a/.claude/skills/verify-open-pstack/scripts/verify.test.ts b/.claude/skills/verify-open-pstack/scripts/verify.test.ts index 7308e698..21938394 100644 --- a/.claude/skills/verify-open-pstack/scripts/verify.test.ts +++ b/.claude/skills/verify-open-pstack/scripts/verify.test.ts @@ -1,6 +1,6 @@ import { afterEach, describe, expect, test } from 'bun:test'; import { execFileSync } from 'node:child_process'; -import { mkdirSync, mkdtempSync, realpathSync, rmSync, writeFileSync } from 'node:fs'; +import { chmodSync, mkdirSync, mkdtempSync, realpathSync, renameSync, rmSync, writeFileSync } from 'node:fs'; import { tmpdir } from 'node:os'; import { join } from 'node:path'; import data from '../features/registry.json'; @@ -8,8 +8,8 @@ import { requiredFeatures, requiredHarnesses, validateRegistry } from './core.ts import { parseChangedFiles, Publisher } from './github.ts'; import { command, sha256, type Command } from './io.ts'; import { MacDriver } from './harness.ts'; -import { evidence, verify } from './verify.ts'; -import { HARNESSES, REPO, type Driver, type GitHub, type Pull, type Receipt } from './types.ts'; +import { evidence, Refused, SESSION_LIMIT, verify } from './verify.ts'; +import { HARNESSES, REPO, type ChangedFile, type Driver, type GitHub, type Pull, type Receipt } from './types.ts'; const SHA = 'a'.repeat(40), BASE = 'b'.repeat(40), NEXT = 'c'.repeat(40), HASH = 'd'.repeat(64), REVISION = 'e'.repeat(40); const URL = `https://github.com/${REPO}/pull/123#issuecomment-1`; @@ -36,7 +36,7 @@ function fixture(runtime = false, selfTest = false) { async prepare(r) { mark('prepare'); return requiredHarnesses(r).map(harness => ({ harness, sha: r.sha, cliVersion: 'test', pluginVersion: '1.5.0', treeHash: HASH, location: `/isolated/${harness}/plugin`, home: `/isolated/${harness}` })); }, async exercise(r) { mark('exercise'); return requiredHarnesses(r).flatMap(harness => requiredFeatures(r, harness).map(feature => ({ harness, feature, surface: 'native surface', action: 'invoke', observed: 'fixture changed', reviewer: 'recipe' as const, assertions: ['invoke:skill-loaded'], transcript: 'retained.log', transcriptHash: sha256('reviewed transcript'), artifacts: [{ path: 'fixture.json', sha256: sha256('reviewed artifact') }] }))); }, }; - const options = { pr: 123, root, selfTest, features: ['claude:skill-invocation:architect'], publisherRevision: REVISION, registry: validateRegistry(data), github, driver, persist: async (r: Receipt) => { saved.push(structuredClone(r)); } }; + const options = { pr: 123, root, selfTest, features: runtime ? ['claude:skill-invocation:architect'] : [], wide: undefined as string | undefined, publisherRevision: REVISION, registry: validateRegistry(data), github, driver, persist: async (r: Receipt) => { saved.push(structuredClone(r)); } }; return { options, calls, saved, comments, pull, hook: (fn: typeof hook) => { hook = fn; } }; } @@ -53,7 +53,8 @@ describe('exact-head evidence publication', () => { test('a runtime change runs only the named features, plus project-skill in both harnesses under self-test', async () => { const f = fixture(true, true), receipt = await verify(f.options); - expect(receipt.selection.features).toEqual(['claude:skill-invocation:architect']); + expect(receipt.selection.features).toEqual(['claude:skill-invocation:architect', 'codex:skill-invocation:architect']); + expect(receipt.runFeatures).toEqual(['claude:skill-invocation:architect']); expect(receipt.installations.map(record => record.harness).sort()).toEqual([...HARNESSES]); expect(receipt.observations.map(record => `${record.harness}/${record.feature}`).sort()).toEqual(['claude/project-skill', 'claude/skill-invocation:architect', 'codex/project-skill']); expect(f.comments).toHaveLength(1); expect(f.comments[0]).toContain('PASSED — named features verified'); @@ -94,8 +95,8 @@ describe('exact-head evidence publication', () => { test('verifier changes require the candidate publisher before any driver call or GitHub write', async () => { for (const revision of [REVISION, SHA]) { - const f = fixture(); f.options.publisherRevision = revision; - f.options.github.files = async () => [{ filename: '.claude/skills/verify-open-pstack/scripts/verify.ts' }]; + const f = fixture(true); f.options.publisherRevision = revision; + f.options.github.files = async () => [{ filename: '.claude/skills/verify-open-pstack/scripts/verify.ts' }, { filename: 'plugins/pstack/skills/architect/SKILL.md' }]; f.options.driver.cleanup = async () => { f.calls.push('cleanup'); }; if (revision !== SHA) { await expect(verify(f.options)).rejects.toThrow('rerun from that checkout'); @@ -184,6 +185,66 @@ describe('exact-head evidence publication', () => { } }); + // The changed files of #199's release commit, e99abd7. + const RELEASE: ChangedFile[] = [{ filename: '.claude-plugin/marketplace.json', versionOnly: true }, { filename: 'CHANGES.md' }, { filename: 'README.md' }, { filename: 'UPSTREAM.md' }, + { filename: 'docs/reference.md' }, { filename: 'plugins/pstack/.claude-plugin/plugin.json', versionOnly: true }, { filename: 'plugins/pstack/.codex-plugin/plugin.json', versionOnly: true }]; + const noWrites = (calls: string[]) => expect(calls.some(call => ['preflight', 'prepare', 'exercise', 'comment'].includes(call) || call.startsWith('status:'))).toBe(false); + + test('a named feature outside the classified reach is refused before preflight or any GitHub write', async () => { + const f = fixture(true); f.options.github.files = async () => { f.calls.push('files'); return RELEASE; }; + // The feature list copied from the 1.9.0 release onto #199. + f.options.features = ['claude:hook:startup', 'codex:hook:startup', 'claude:catalog', 'codex:catalog', 'claude:setup', 'codex:setup', 'claude:runner', 'codex:runner']; + const error = await verify(f.options).then(() => undefined, (e: unknown) => e); + expect(error).toBeInstanceOf(Refused); + expect((error as Error).message).toBe("Outside this PR's classified reach: claude:catalog, codex:catalog, claude:setup, codex:setup, claude:runner, codex:runner. Reachable: claude:hook:startup, codex:hook:startup"); + noWrites(f.calls); + const docs = fixture(); docs.options.features = ['claude:skill-invocation:architect']; + await expect(verify(docs.options)).rejects.toThrow("Outside this PR's classified reach: claude:skill-invocation:architect. Reachable: none"); + noWrites(docs.calls); + }); + + test('the release runs its two startup sessions', async () => { + const f = fixture(true); f.options.github.files = async () => RELEASE; + f.options.features = ['claude:hook:startup', 'codex:hook:startup']; + const receipt = await verify(f.options); + expect(receipt.status).toBe('success'); expect(receipt.sessions).toBe(2); + expect(f.comments[0]).toContain(`- Planned sessions: 2 (limit ${SESSION_LIMIT})`); expect(f.comments[0]).not.toContain('Wide run reason'); + }); + + test('more than the session limit is refused before preflight unless --wide gives a reason', async () => { + const limited = (...extra: string[]) => { + const f = fixture(true); f.options.github.files = async () => [{ filename: 'plugins/pstack/skills/poteto-mode/references/codex-tools.md' }]; + // Setup is eight sessions, the default Claude runner three and the startup hook one. + f.options.features = ['claude:setup', 'claude:runner', 'claude:hook:startup', ...extra]; + return f; + }; + expect(SESSION_LIMIT).toBe(12); + const exact = limited(); + expect((await verify(exact.options)).sessions).toBe(12); + const over = limited('codex:hook:startup'); + await expect(verify(over.options)).rejects.toThrow('13 planned sessions exceed the limit of 12; name fewer features or pass --wide ""'); + noWrites(over.calls); + const wide = limited('codex:hook:startup'); wide.options.wide = 'Setup and runner changed together in #196'; + const receipt = await verify(wide.options); + expect(receipt.status).toBe('success'); expect(receipt.sessions).toBe(13); expect(receipt.wide).toBe(wide.options.wide); + expect(wide.calls.indexOf('preflight')).toBeGreaterThan(-1); + expect(wide.comments[0]).toContain('- Planned sessions: 13 (limit 12)'); + expect(wide.comments[0]).toContain('- Wide run reason: Setup and runner changed together in #196'); + // A failed run records the reason too. + const failed = limited('codex:hook:startup'); failed.options.wide = 'Broad shared change'; + failed.options.driver.exercise = async () => { throw new Error('session failed'); }; + failed.options.driver.cleanup = async () => {}; + await expect(verify(failed.options)).rejects.toThrow('session failed'); + expect(failed.comments[0]).toContain('FAILED'); expect(failed.comments[0]).toContain('- Wide run reason: Broad shared change'); + }); + + test('an unsupported recipe fails before preflight', async () => { + const f = fixture(true); f.options.github.files = async () => [{ filename: 'plugins/pstack/assets/logo.png' }]; + f.options.features = ['codex:assets:codex']; + await expect(verify(f.options)).rejects.toThrow('unsupported-native-consumer:assets:codex'); + noWrites(f.calls); + }); + test('closed or foreign PR never enters publication', async () => { for (const field of ['state', 'headRepo'] as const) { const f = fixture(); f.pull[field] = 'other'; @@ -209,7 +270,7 @@ describe('publisher request boundaries', () => { }); test('fresh local git diff uses immutable base/head even when a source branch moves', async () => { - const repository = temporary(), env = { ...process.env, GIT_AUTHOR_NAME: 'test', GIT_AUTHOR_EMAIL: 'test@example.com', GIT_COMMITTER_NAME: 'test', GIT_COMMITTER_EMAIL: 'test@example.com' }; + const repository = temporary(), env = { PATH: process.env.PATH ?? '', GIT_CONFIG_GLOBAL: '/dev/null', GIT_CONFIG_NOSYSTEM: '1', GIT_AUTHOR_NAME: 'test', GIT_AUTHOR_EMAIL: 'test@example.com', GIT_COMMITTER_NAME: 'test', GIT_COMMITTER_EMAIL: 'test@example.com' }; const git = (...args: string[]) => execFileSync('git', args, { cwd: repository, env, encoding: 'utf8' }).trim(); git('init', '-q'); writeFileSync(join(repository, 'README.md'), 'base'); git('add', '.'); git('commit', '-qm', 'base'); const base = git('rev-parse', 'HEAD'); mkdirSync(join(repository, 'runtime')); writeFileSync(join(repository, 'runtime', 'skill.md'), 'immutable runtime'); git('add', '.'); git('commit', '-qm', 'runtime'); const head = git('rev-parse', 'HEAD'); @@ -217,10 +278,48 @@ describe('publisher request boundaries', () => { const calls: string[][] = []; const publisher = new Publisher(async (args, options) => { calls.push(args); if (args[0] === 'git' && args[1] === 'fetch') args = args.map(arg => arg === `https://github.com/${REPO}.git` ? repository : arg); return command(args, options); }); expect(await publisher.files(base, head)).toEqual([{ filename: 'runtime/skill.md' }]); - expect(calls.some(args => args[0] === 'gh')).toBe(false); expect(calls.at(-1)).toContain(`${base}...${head}`); + expect(calls.some(args => args[0] === 'gh')).toBe(false); expect(calls).toContainEqual(expect.arrayContaining([`${base}...${head}`])); await expect(publisher.files('main', head)).rejects.toThrow('Invalid diff SHA'); }); + test('a manifest is version-only when only its version strings changed since the merge base', async () => { + const repository = temporary(), env = { PATH: process.env.PATH ?? '', GIT_CONFIG_GLOBAL: '/dev/null', GIT_CONFIG_NOSYSTEM: '1', GIT_AUTHOR_NAME: 'test', GIT_AUTHOR_EMAIL: 'test@example.com', GIT_COMMITTER_NAME: 'test', GIT_COMMITTER_EMAIL: 'test@example.com' }; + const git = (...args: string[]) => execFileSync('git', args, { cwd: repository, env, encoding: 'utf8' }).trim(); + const CLAUDE = 'plugins/pstack/.claude-plugin/plugin.json', CODEX = 'plugins/pstack/.codex-plugin/plugin.json', MARKET = '.claude-plugin/marketplace.json'; + const write = (path: string, value: unknown, indent = 2) => { mkdirSync(join(repository, path, '..'), { recursive: true }); writeFileSync(join(repository, path), JSON.stringify(value, null, indent) + '\n'); }; + const manifests = (version: string, extra = {}) => { + write(CLAUDE, { name: 'pstack', version, description: 'd' }); + write(CODEX, { version, name: 'pstack', interface: { logo: './assets/logo.png' } }); + write(MARKET, { name: 'open-pstack', plugins: [{ name: 'pstack', version, source: './plugins/pstack' }], ...extra }); + }; + const commit = (message: string) => { git('add', '-A'); git('commit', '-qm', message); return git('rev-parse', 'HEAD'); }; + git('init', '-q', '-b', 'main'); manifests('1.0.0'); const mergeBase = commit('base'); + // main moves after the branch point; comparing the base tip would see this field as a head change. + manifests('1.0.0', { owner: 'later' }); const base = commit('main moved'); + git('checkout', '-q', '-b', 'release', mergeBase); + write(CLAUDE, { version: '1.1.0', description: 'd', name: 'pstack' }, 4); + write(CODEX, { version: '1.1.0', name: 'pstack', interface: { logo: './assets/logo.png' } }); + write(MARKET, { name: 'open-pstack', plugins: [{ name: 'pstack', version: '1.1.0', source: './plugins/pstack' }] }); + const release = commit('release'); + git('checkout', '-q', '-b', 'edited', mergeBase); + write(CLAUDE, { name: 'pstack', version: '1.1.0', description: 'changed' }); + write(CODEX, { version: '1.0.0', name: 'pstack', interface: { logo: './assets/logo.png' } }, 4); + write(MARKET, { name: 'open-pstack', plugins: [{ name: 'pstack', source: './plugins/pstack' }] }); + const edited = commit('extra field, unchanged version, removed version'); + git('checkout', '-q', '-b', 'moved', mergeBase); + renameSync(join(repository, CLAUDE), join(repository, 'plugins/pstack/.claude-plugin/old.json')); + rmSync(join(repository, CODEX)); + write(MARKET, { name: 'open-pstack', plugins: [{ name: 'pstack', version: '1.1.0', source: './plugins/pstack' }] }); chmodSync(join(repository, MARKET), 0o755); + const moved = commit('rename, deletion, mode change'); + const publisher = new Publisher(async (args, options) => command(args[0] === 'git' && args[1] === 'fetch' ? args.map(arg => arg === `https://github.com/${REPO}.git` ? repository : arg) : args, options)); + const flagged = async (head: string) => (await publisher.files(base, head)).filter(f => f.versionOnly).map(f => f.filename).sort(); + expect(await flagged(release)).toEqual([MARKET, CLAUDE, CODEX].sort()); + expect((await publisher.files(base, edited)).map(f => f.filename).sort()).toEqual([MARKET, CLAUDE, CODEX].sort()); + expect(await flagged(edited)).toEqual([]); + expect((await publisher.files(base, moved)).length).toBe(3); + expect(await flagged(moved)).toEqual([]); + }); + test('interruption blocks ordinary status writes but permits a pinned failure gate', () => { for (const signal of ['SIGINT', 'SIGTERM']) { const script = ` diff --git a/.claude/skills/verify-open-pstack/scripts/verify.ts b/.claude/skills/verify-open-pstack/scripts/verify.ts index 3cb7cbea..0016fb1f 100644 --- a/.claude/skills/verify-open-pstack/scripts/verify.ts +++ b/.claude/skills/verify-open-pstack/scripts/verify.ts @@ -1,8 +1,13 @@ -import { classify, completeEvidence, newReceipt, requiredHarnesses } from './core.ts'; +import { recipe } from '../features/recipes.ts'; +import { classify, completeEvidence, newReceipt, requiredFeatures, requiredHarnesses } from './core.ts'; import { retainedFile, sha256 } from './io.ts'; import { REPO, type Driver, type GitHub, type Pull, type Receipt, type Registry } from './types.ts'; class HeadMoved extends Error {} +/** An operator choice the verifier refuses before any install or GitHub write; its message is safe to print. */ +export class Refused extends Error {} +/** Headless sessions one run may start without `--wide ""`. */ +export const SESSION_LIMIT = 12; // Omit unsafe or oversized values whole rather than rewriting recorded evidence. export const EVIDENCE_LIMIT = 160; export const publishable = (text: string, limit = EVIDENCE_LIMIT): boolean => text.length <= limit && /^[\x20-\x7e]*$/.test(text) && !/[`<>]/.test(text); @@ -17,6 +22,8 @@ export function evidence(receipt: Receipt): string { `- Base SHA: \`${receipt.base}\``, `- Result: ${failed ? 'FAILED — verification did not complete; rerun required' : receipt.selection.noRuntime && !receipt.selfTest ? 'PASSED — no runtime change' : 'PASSED — named features verified'}`, `- Self-test: ${receipt.selfTest ? failed ? 'required in both harnesses' : 'completed in both harnesses' : 'not requested'}`, + `- Planned sessions: ${receipt.sessions} (limit ${SESSION_LIMIT})`, + ...(receipt.wide ? [`- Wide run reason: ${safe(receipt.wide)}`] : []), `- Publisher revision: ${receipt.publisherRevision ? `\`${receipt.publisherRevision}\`` : 'not supplied'}`, `- Evidence-set SHA-256: \`${sha256(JSON.stringify({ installations: receipt.installations, observations: receipt.observations }))}\``, ]; @@ -52,6 +59,10 @@ type VerifyOptions = { selfTest: boolean; /** `:` checks the agent named for the PR's changed behavior. */ features: string[]; + /** Runner routes, which decide the runner recipe's session count. */ + routes?: string[]; + /** The operator's reason for planning more than SESSION_LIMIT sessions. */ + wide?: string; root: string; registry: Registry; github: GitHub; @@ -94,14 +105,21 @@ export async function verify(options: VerifyOptions): Promise { }; await phase('classify'); receipt.selection = classify(await github.files(receipt.base, receipt.sha), options.registry); - if (receipt.selection.features.includes('project-skill') && receipt.publisherRevision !== receipt.sha) { + if (receipt.selection.features.includes('claude:project-skill') && receipt.publisherRevision !== receipt.sha) { throw new Error('Verifier-changing PRs must run from a checkout of the candidate head; rerun from that checkout'); } - if (!receipt.selection.noRuntime && !options.features.length) throw new Error('A runtime change requires at least one --feature :'); - // Only the named features run; classification gates runtime changes and unmapped paths. + if (!receipt.selection.noRuntime && !options.features.length) throw new Refused('A runtime change requires at least one --feature :'); + const refused = options.features.filter(feature => !receipt.selection.features.includes(feature)); + if (refused.length) throw new Refused(`Outside this PR's classified reach: ${refused.join(', ')}. Reachable: ${receipt.selection.features.join(', ') || 'none'}`); // Verifier and instruction changes keep their native project-skill proof whatever is named. - const project = receipt.selection.features.includes('project-skill') ? ['claude:project-skill', 'codex:project-skill'] : []; - receipt.selection.features = receipt.selection.noRuntime ? [] : [...new Set([...options.features, ...project])]; + const project = receipt.selection.features.includes('claude:project-skill') ? ['claude:project-skill', 'codex:project-skill'] : []; + receipt.runFeatures = [...new Set([...options.features, ...project])]; + for (const harness of requiredHarnesses(receipt)) { + for (const feature of requiredFeatures(receipt, harness)) receipt.sessions += recipe(feature, harness, options.routes).length; + } + console.log(`Planned headless sessions: ${receipt.sessions} (limit ${SESSION_LIMIT})`); + if (receipt.sessions > SESSION_LIMIT && !options.wide) throw new Refused(`${receipt.sessions} planned sessions exceed the limit of ${SESSION_LIMIT}; name fewer features or pass --wide ""`); + if (options.wide) receipt.wide = options.wide; if (requiredHarnesses(receipt).length) await driver.preflight(receipt); try { await current(); await persist(receipt); diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml index 126d94b4..259220ee 100644 --- a/.github/workflows/ci.yml +++ b/.github/workflows/ci.yml @@ -17,18 +17,9 @@ jobs: with: node-version: 22.23.2 - uses: oven-sh/setup-bun@v2 - with: - bun-version: 1.4.0 - - name: Install - working-directory: plugins/pstack/skills/poteto-mode/scripts - run: bun install --frozen-lockfile - name: Test - working-directory: plugins/pstack/skills/poteto-mode/scripts run: bun run test - name: Typecheck - working-directory: plugins/pstack/skills/poteto-mode/scripts run: bun run typecheck - name: Validate manifests and static invariants - run: | - bun -e 'for (const path of [".claude-plugin/marketplace.json", ".agents/plugins/marketplace.json", "plugins/pstack/.claude-plugin/plugin.json", "plugins/pstack/.codex-plugin/plugin.json"]) JSON.parse(await Bun.file(path).text())' - PSTACK_STATIC_ONLY=1 bash tests/skill-collision-repro.sh + run: bun run static diff --git a/.github/workflows/windows-hooks.yml b/.github/workflows/windows-hooks.yml index 43374a89..95a22cb8 100644 --- a/.github/workflows/windows-hooks.yml +++ b/.github/workflows/windows-hooks.yml @@ -31,6 +31,8 @@ jobs: Copy-Item (Join-Path $env:SystemRoot 'System32\hostname.exe') (Join-Path $cwd 'cmd.exe') $expected = (Get-Content -Raw "$root\hooks\session-start-context.md").Replace("`r`n", "`n").TrimEnd() $hook = (Get-Content -Raw plugins/pstack/hooks/hooks.json | ConvertFrom-Json).hooks.SessionStart[0].hooks[0].commandWindows + $flag = Join-Path $env:RUNNER_TEMP 'pstack-front-door.on' + $env:PSTACK_FRONT_DOOR_FLAG = $flag.Replace('\', '/') # Codex runs a hook through the session's shell (codex-rs/hooks/src/engine/command_runner.rs, # build_command): PowerShell gets `-NoProfile -Command ` as an argument, and cmd, or # %COMSPEC% with no session shell, gets `/c ""` unescaped. @@ -57,6 +59,11 @@ jobs: Write-Host "$(if ($Ok) { 'PASS' } else { 'FAIL' }): $Name" if (-not $Ok) { $script:failed++ } } + foreach ($shell in $shells.Keys) { + $result = Invoke-Hook $shell $hook $env:PATH + Check "$shell, absent front-door flag" ($result.Code -eq 0 -and $result.Out -eq '') + } + New-Item -ItemType File -Path $flag | Out-Null foreach ($shell in $shells.Keys) { foreach ($case in @(@{ Name = 'ordinary PATH'; Path = $env:PATH }, @{ Name = 'System32-only PATH'; Path = $system32 })) { $result = Invoke-Hook $shell $hook $case.Path diff --git a/CHANGES.md b/CHANGES.md index 3606943f..df6e7ad9 100644 --- a/CHANGES.md +++ b/CHANGES.md @@ -3,6 +3,85 @@ Mergify auto-queue is live: ready PRs to main require passing verify, Unfret and exact-head live-gate, and no unresolved review thread; workflow changes require operator queueing. This port applies the Cursor → Claude Code substitutions in skill bodies. Earlier drafts left them flagged; this revision resolves them. A later pass added a Codex build that shares the same skills; see [Codex port](#codex-port) below. +## 1.15.0-adam.20261019 adds formal verification skills + +Adam's fork combines released Open Pstack 1.15.0 with Cursor pstack 0.15.28 skill changes. It adds explicit creation and maintenance skills for formal verification, their help and setup entries, and the latest ketchup, sharpen and TypeScript guidance. The fork's front-door gate, cloud installer and model choices remain. See FORK.md for exact source commits and exclusions. + +## 1.15.0 runs Devin, agy and dsh lanes in any repository + +Open Pstack 1.15.0 keeps the Cursor pstack 0.15.25 sync. It ships PR #215, merged after 1.14.0. Default roles and panels are unchanged. + +**No repository refusals.** A read-only lane means the read-only settings pstack passes to each CLI; the operator's host, account, repository and organization configuration applies as they set it up, and the runner adds no refusal, guard or account check because of it (#203). The runner no longer refuses a Devin lane in a repository holding `.devin/`, an agy lane in one holding `.agents/`, `.agent/`, `_agents/` or `_agent/`, or a dsh lane whose working directory holds `.env`, and the home-directory rule that came with those checks is gone. Every provider now runs in any directory (#204). + +**What still loads.** The catalog notes say what each CLI loads from the working directory and its parents up to the git root, or up to `/` outside a repository: Devin's `.devin/` hooks, permissions and MCP servers; agy's hooks, MCP servers, rules and skills in its four directories, with no trust check; dsh's working-directory `.env`, whose values other than dsh's own bootstrap names reach its bash tool. Outside a repository under the home directory, Devin and agy also load the home directory's own `~/.devin/` and `~/.agents/`. dsh still refuses any version except the measured `0.2.0-rc.2`, because dsh ignores a patch row it does not have. + +## 1.14.0 adds a read-only Cursor CLI provider + +Open Pstack 1.14.0 keeps the Cursor pstack 0.15.25 sync. It ships PR #211, merged after 1.13.0. Default roles and panels are unchanged. No role uses the new provider by default, and setup refuses it for every role that writes files. + +**Read-only Cursor CLI provider.** The runner dispatches `cursor:composer-2.5` at the `default` effort and `cursor:gemini-3.7-flash` at `low`, `medium` or `high` through the signed-in `cursor-agent` CLI from either parent; `high` is Gemini's default, and the effort selects the CLI model ID, such as `gemini-3.7-flash-high`. Gemini 3.7 Flash is the model #37 asked for, which Antigravity did not serve for the measured account. Each lane runs `cursor-agent -p --mode ask --sandbox enabled --trust` with a run-owned `CURSOR_CONFIG_DIR` whose config denies writes, shell commands and every MCP call, and approves no web tool. The login check is `cursor-agent models`, never `cursor-agent status`, and empty output is inconclusive, never unauthenticated (#94). Cursor reports only a display label for the model, so receipts record `pinned-argv` evidence. Sign in with `env -u CURSOR_API_KEY cursor-agent login`: a login made while `CURSOR_API_KEY` was set could not make model calls on the measured Mac (#71). + +**Read-only means pstack's own settings.** The runner refuses no repository and adds no pre-launch check because of the operator's own Cursor configuration (#203). Every turn starts Cursor's language-server worker, which runs `npx -y typescript-language-server --stdio` in the workspace and uses a repository's own copy when it has one; the worker stays running for a few idle minutes after the turn. Enterprise, team, user and repository hooks run unsandboxed in every lane, and a repository's `.cursor/cli.json` replaces the run-owned deny list. A lane can start a Task subagent, which ran on the lane's own model under the same limits; Cursor's CLI has no setting that removes Task (#212). The catalog's Cursor notes say what each was measured to do. + +## 1.13.0 adds a read-only Amp provider + +Open Pstack 1.13.0 keeps the Cursor pstack 0.15.25 sync. It ships PR #208, merged after 1.12.0. Default roles and panels are unchanged. No role uses the new provider by default, and setup refuses it for every role that writes files. + +**Read-only Amp provider.** The runner dispatches `amp:deepseek-v4.1-flash`, Amp's official DeepSeek V4.1 Flash mode, at the `default` effort through the signed-in Amp CLI from either parent. Each lane passes Amp a run-owned settings file that enables only the `Read` tool and rejects every other tool, and turns off MCP servers, Claude Code and `~/.agents` skills, updates and notifications. The lane ignores `AGENTS.md` guidance files, runs as a private ampcode.com thread labelled `pstack-runner`, and spends Amp credits. `amp usage` is the login check. Failures and cancellations can exit 0, so the stream's `result` event decides the outcome. Amp's stream never names the served model, so receipts record `pinned-argv` evidence (#158). + +**Read-only means pstack's own settings.** The runner refuses no repository and adds no pre-launch check because of the operator's own Amp configuration (#203). Amp's own `official-modes` plugin, system plugins, personal and workspace plugins, machine and workspace managed settings, and a repository's `.amp/` settings and plugins still load in every lane, and they can enable tools, change modes or run code. The catalog's Amp notes say what each was measured to do. + +## 1.12.0 adds a read-only OpenCode provider + +Open Pstack 1.12.0 keeps the Cursor pstack 0.15.25 sync. It ships PR #202, merged after 1.11.0. Default roles and panels are unchanged. No role uses the new provider by default, and setup refuses it for every role that writes files. + +**Read-only OpenCode provider.** The runner dispatches `opencode:opencode/deepseek-v4.1-flash-free`, OpenCode Zen's free DeepSeek V4.1 Flash, which needs no key, at `low`, `high` or `max` through the OpenCode CLI from either parent. `max` is the default. Other OpenCode models run unchanged as unregistered pairs and authenticate through OpenCode's stored logins. Each lane runs OpenCode's `build` agent with a run-owned config whose permissions allow only `read`, `glob`, `grep`, `list` and `todowrite`, with `--pure`, a fixed session title, and OpenCode's switches that turn off project config, Claude Code files and external skills. A turn that ends on a rejected tool call is `child-failed`. Neither the event stream nor any other OpenCode output names the served model, so receipts record `pinned-argv` evidence (#68). + +**Read-only means pstack's own settings.** The runner refuses no repository and adds no pre-launch check because of the operator's own OpenCode configuration (#203). A managed config, the macOS managed-preferences profile, remote config from a stored well-known login or console organization, and a repository's `.opencode/plugins/` still load in every lane, and they can add MCP servers, language servers, plugins, provider routing or agent changes. The catalog's OpenCode notes say what each was measured to do with OpenCode 1.19.1. + +## 1.11.0 adds read-only DeepSeek Harness and Antigravity providers + +Open Pstack 1.11.0 keeps the Cursor pstack 0.15.25 sync. It ships PR #196, merged after 1.10.0. Default roles and panels are unchanged. No role uses the new providers by default, and setup refuses both for every role that writes files. + +**Read-only DeepSeek Harness provider.** The runner dispatches `dsh:deepseek-flash` and `dsh:deepseek-v4-pro`, each at `low`, `high` or `max`, through the signed-in DeepSeek Harness CLI from either parent. `max` is the default. Each lane runs dsh with its own `DSH_HOME`, the lane's private 0700 provider directory, so the operator's dsh profile does not load. A run-owned `--patch` file pins the model, the effort, a read-only sandbox rooted at the lane's working directory and approval `ask`. It reads the stored login in place and turns off host and project rule files, skills, the web, subagent and workflow tools, the plugin inventory, the session-title call and the session-log upload. In read-only mode dsh's bash tool keeps network access, and its `read` tool is not confined to the working directory; the catalog says so. The runner checks `dsh --version` before launch and refuses any version except the measured `0.2.0-rc.2` as `unavailable-cli`, because dsh ignores a patch row it does not have. It refuses a lane whose working directory holds a `.env` file, because dsh loads that file into its bash tool at boot. The preflight proves only that a key is stored or exported, not that the key works. A key kept only in the operator's dsh `.env` does not authenticate a lane: store it with dsh or export `DEEPSEEK_API_KEY` (#159). + +**Read-only Antigravity provider.** The runner dispatches `agy:gemini-3.8-flash` at `low`, `medium` or `high` through the signed-in Antigravity CLI from either parent. `high` is the default, and the effort selects the model ID, such as `gemini-3.8-flash-high`. The prompt goes to agy as one JSON line on stdin. Each lane runs a run-owned agent that allows only `view_file`, so the lane has no web search, URL reads, subagents, scheduling, messaging or browser. A turn that ends on a denied tool is `child-failed`, even after a reply preamble. The runner refuses a lane whose working directory, or any parent up to the git root, holds `.agents/`, `.agent/`, `_agents/` or `_agent/`, because agy runs the hooks and MCP servers there with no trust check. The operator's own `~/.gemini/GEMINI.md`, host skills and agy plugin hooks still load in every lane, and no flag turns them off. Plugin hooks the operator installs are trusted operator configuration (#197). Issue #37 asked for Gemini 3.7 Flash, which Antigravity did not list for the measured account. The runner registers Gemini 3.8 Flash and never maps a 3.7 request to it (#37). + +**Model proof.** Neither CLI reports the model it served, so both receipts record `pinned-argv` evidence. `provider-dispatch.md` now accepts pinned-argv evidence for any provider whose catalog Model proof cell says so, not only for Codex. + +**Home-directory rule.** The runner refuses a DeepSeek Harness, Antigravity or Devin lane whose project-config walk would reach the home directory. That covers a working directory under the home outside any git repository, a repository rooted at the home, and the home itself. Inside a repository below the home, the walk stops at the repository root. Claude, Codex and Grok lanes are unchanged. + +## 1.10.0 syncs to Cursor pstack 0.15.25 + +Open Pstack 1.10.0 tracks Cursor pstack 0.15.25 at `99743e9c151c8537cbbf1f98bb38f34ef3368453`. Seventeen upstream commits since 0.15.20 touch pstack. Thirteen change content, and all thirteen are applied with the port adaptations below. The other four bump only the Cursor manifests, which stay out. + +**New skills.** `/in-your-own-words` restates the user's goal and the problem before work starts. `/sharpen` approaches a task the way a top expert would, aimed at what the user is really after, without overspecifying it. Both run only when typed. soulmachine ported them in PR #188, and this release carries that work (#187). + +**Effort per spawn.** Before each subagent starts, poteto-mode and the skills it routes to start from the role's configured effort and move it for the job: one step up, never past `max`, for the hardest work, one step down for judgment, and two or more steps down for routine work. Verification lanes keep the `swarm workers` descriptor. The provider and model stay as configured, so a panel keeps its three providers. An effort the model's catalog row does not list moves to the nearest one it lists, and a configured `ultra` stays `ultra`. If a route rejects the adjusted effort, the lane runs on the configured descriptor and says so. Any other failure is a dropout, as before. + +**Changed behavior.** Autopilot-full, Autopilot-stack and the multi-phase plan run one full swarm at the code-ready head, and a rebase, a conflict fix or a minor fix gets one verifier at the new head. A defect that trunk shows without the PR does not gate the PR. Loosening a gate needs a verifier's proof and the root's countersign, and a disabled or soft-failed check always comes out. Autopilot owners run off trunk and never hold a private stack, and they stop and report a push they did not make. Autopilot-stack owners branch from the branch of the item they depend on. An audit tick or a reply that changed something catches the operator up through `/ketchup`. Orchestrate's `orch frontier set` reads the pull request base chain through `gh`, or through Origin with `--forge origin`, instead of Graphite. `watch-pr` treats a Code Review Gate that waits on an owner approval as the gate, not a failed check. `worktree-audit.sh` reads Origin pull requests when Origin can read the repository, and a draft pull request holds its worktree. `/ketchup` keeps its reply under 150 words in `/bro`'s voice, shows screenshots, videos and diagrams where a picture explains better, and lists action items as plain bullets. `principle-fix-root-causes` proposes a cheap always-on signal after a deep debugging dig. poteto-mode tells delegates where bundled skills live and passes its own path to each `poteto-agent`. + +**Restated principles.** `principle-test-behavior-not-implementation`, `principle-fix-root-causes`, `principle-attack-the-premise` and `principle-make-operations-idempotent` now state the reasoning instead of naming matchers, guards, a census or locks. `correct` step 3 asks whether a test fails when the behavior breaks and passes when only the internals change. These rewrites keep absence assertions that a contract promises, relational invariants and compile-time checks, so the port's 0.15.0 local corrections and the `00b14f1` and `9511e60` exclusions retire. + +**Shorter prompts.** Upstream cuts sections from Interrogate's lead-judgment reference, Swarm, and the type-system and migrate-callers principles. + +**Port adaptations.** The effort rule drops upstream's fast and standard speed sentences, because the port's descriptors carry no speed selector, and reads its starting effort from the model sheet or role default instead of the excluded setup budget. Arena, Interrogate and the other routed skills keep their dispatch contracts and dropout rule, without upstream's `cursor-grok-*` and `composer-*` fallback. Shipping keeps its nine landing steps and takes upstream's new rules and reasons. Autopilot-full takes upstream's seven steps, with the guarded rebase and push in step 2. Autopilot-stack takes upstream's four steps, with the fork and topology rules in step 2. The landing invariants follow the moved text. `poteto-agent` keeps the `pstack:poteto-mode` preload and takes the parent's path instead of searching `~/.cursor/plugins/`. `/ketchup` draws charts with the harness's own renderer instead of Cursor's `/visualize`. `docs/reference.md` and `codex-tools.md` no longer list Graphite as a requirement. + +**Not applied.** See `UPSTREAM.md`: Autopilot-full step 5's trunk-move rule as `ce649c0` revised it, the stuck-after runtime and first-tick stuck test, the `composer-*` fallback, the Cursor plugin search in `poteto-agent`, and the Cursor manifests and `docs/guide/`. `README-UPSTREAM.md` is the verbatim upstream README. +## 1.9.0 adds Luna, Sonnet, an exact Opus 5.5 pin, a read-only Devin provider and full-access Codex writers + +Open Pstack 1.9.0 keeps the Cursor pstack 0.15.20 sync. It ships registry units U4 to U7, merged after 1.8.0. Default roles and panels are unchanged. + +**New model families.** `codex:gpt-6-luna`, `claude:sonnet` and `claude:claude-opus-5-5` are registry entries. Luna takes `low` to `max` and defaults to `max`. Sonnet is Claude's rolling Sonnet alias. `claude-opus-5-5` is an exact pin that keeps Opus 5.5 when the `opus` alias moves. The model catalog lists all three, and Claude Code gets native `pstack-sonnet-` and `pstack-opus-5-5-` agents. Luna and Sonnet reject `ultra`, which stays on Sol only. Setup probes a new family only when one of your roles uses it. `gpt-5.6-luna` and `gpt-5.6-terra` get no entries, because Codex describes both as older and no released sheet used them (#54). + +**Model matching order.** Setup and dispatch now match a descriptor against an exact catalog row first, then a kept predecessor, then a rolling alias's revision. Before this release they normalized every versioned Fable and Opus descriptor to its alias first, which would have turned the pin into `opus`. An older versioned Opus, such as `claude:claude-opus-5`, still normalizes to `opus`, and a versioned Sonnet, such as `claude:claude-sonnet-5-5`, normalizes to `sonnet`. A setup rerun writes those migrations to the sheet and keeps Luna and the pin. The runner rejects an external Claude lane whose provider report names another model: the pin accepts only `claude-opus-5-5`, and Sonnet only a Sonnet revision (#54). + +**Read-only Devin provider.** The runner dispatches `devin:swe-2@` and `devin:swe-1.6@default` through the signed-in Devin CLI from either parent. The effort selects the Devin model ID, such as `swe-2-medium`. The reply and the model proof come from Devin's `--export` file, not stdout. Devin lanes are read-only: a lane runs `--permission-mode auto` without `--sandbox`, because Devin's `--sandbox` lets shell commands write in the working directory. The runner refuses an `isolated-write` Devin lane before launch. Setup refuses Devin for every role that writes files: `feature, refactoring`, `bug-fix`, `perf-issue`, `hillclimb`, `hardest tasks`, `arena runners`, `architect runners` and `swarm workers`. Each lane gets a run-owned `--config` file in a private 0700 directory. That file turns off subagents, host rule files such as `CLAUDE.md` and MCP tools, and denies web search and the fetch tools. The runner refuses a Devin lane whose working directory, or any parent up to the git root, holds `.devin/`, because Devin would load that repository's hooks, permissions and MCP servers. Devin gets the shared environment allowlist, without `DEVIN_PERMISSION_MODE`, `DEVIN_SANDBOX`, `DEVIN_MODEL` or `DEVIN_REFUSAL_FALLBACK`. No role uses Devin by default. Network isolation for a Devin lane is not established, and the catalog says so (#69). + +**Full access for Codex writer lanes.** The model sheet line `codex writer sandbox: full-access` runs Codex `isolated-write` lanes that the external runner launches with `--sandbox danger-full-access`. Such a writer can launch a browser and write dot-folders such as `.agents/`. Full access removes every Codex sandbox limit, including confinement to the worktree. Without the line, or with `codex writer sandbox: bounded`, writers keep `workspace-write`. Setup asks once, says what full access removes, writes `bounded` on a decline and keeps the line on reruns. The receipt records `writerSandbox` as `bounded` or `full-access`, and as null for a read-only lane. The runner refuses `--writer-sandbox full-access` on a read-only lane and on Claude, Grok and Devin. Native writers are unchanged: a Claude `Agent` writer stays native, and a Codex parent's `spawn_agent` writer keeps the parent's sandbox. Nothing turns full access on after a sandbox failure (#80). + +**Repository checks.** A root `package.json` runs CI's checks as `bun run test`, `bun run typecheck` and `bun run static`, so CI and reviewers run one definition. The plugin is unchanged (#185). + ## 1.8.0 adds a generated model catalog and fixes the Windows SessionStart hook Open Pstack 1.8.0 keeps the Cursor pstack 0.15.20 sync. It ships registry units U2 and U3 and the Windows hook fix, merged after 1.7.0, and adds the note for registry unit U1, which shipped in 1.7.0. diff --git a/FORK.md b/FORK.md index bbbf4d43..12a0ef38 100644 --- a/FORK.md +++ b/FORK.md @@ -49,3 +49,13 @@ Routines and `claude --cloud` use the same environments, so they need no extra s - **Secrets:** none. The fork is public, and anyone who can use an environment can read its setup script. - **Test:** `bash tests/cloud-setup-test.sh`. - **Codex cloud:** no step yet. OpenAI's docs say a cloud task loads skills stored in the repository and does not sync personal skills. They do not say whether an environment's install script can place skills in `$HOME/.agents/skills`. That needs a probe in a real Codex cloud task. + +## October 19, 2026 update + +This candidate uses released Open Pstack 1.15.0 at `39cb5e4a4d3c03e432a0dc840bcfdae0856166ac` and the shared skill changes from Cursor pstack 0.15.28 at `d457567332aa6989e753ea289f30ea2bfbc80bc0`. It is versioned `1.15.0-adam.20261019` to distinguish the fork from the released port. + +The import adds create-formal-verification-skill and maintain-formal-verification-skill, updates their help and setup entry points, and includes the ketchup, sharpen and TypeScript changes since 0.15.25. Both new skills remain explicit-only in Claude and Codex. The setup offer follows the port's existing model probes and behavioral smoke; the port's installation instructions and provider mapping remain. The Cursor-only make-bot-ui, plugin manifests and guide are not installed. The router's name correction is already present in the port. + +The original formal skill bodies are retained byte for byte. Their verification scope, mutation checks, reporting duties and prohibition on product edits remain intact. The shared model sheet, front-door flag, gated session hook and cloud installer are retained. + +The repository-local verifier publishes only to this fork. Its feature map includes both formal skills and their Codex invocation policies. The Windows hook check first verifies silence without the fork's flag, then creates a run-owned flag before exercising the enabled hook. It never changes the operator's flag. diff --git a/NOTICE.md b/NOTICE.md index 47667c47..ac00dabc 100644 --- a/NOTICE.md +++ b/NOTICE.md @@ -23,6 +23,7 @@ This plugin is a port of upstream MIT-licensed work. All upstream copyright noti | `plugins/pstack/skills/` (0.15.2 to 0.15.5 changes: three-model defaults, code-ready rounds, owner authority, prompt cuts, decision-trail runs, `show-me-your-work/scripts/log.sh`), `README-UPSTREAM.md` | [cursor/plugins/pstack @ 12d587d](https://github.com/cursor/plugins/tree/12d587dfb20741cafc376c42c696c5f6e2a64487/pstack) | (c) 2026 Lauren Tan | MIT | [LICENSE](LICENSE) | | `plugins/pstack/skills/` (0.15.6 to 0.15.17 changes: `benchmark-checklist`, `correct`, `ketchup`, `poteto-help`, `principle-explain-the-number`, architect red flags, fresh subagents, hourly audit tick, PR headings, performance mantras, prompt distillation), `plugins/pstack/agents/poteto-agent.md`, `README-UPSTREAM.md` | [cursor/plugins/pstack @ 978ca6e](https://github.com/cursor/plugins/tree/978ca6e9c0e012047cd5058b8e829240336e704e/pstack) | (c) 2026 Lauren Tan | MIT | [LICENSE](LICENSE) | | `plugins/pstack/skills/` (0.15.18 to 0.15.20 changes: `ketchup` before-and-after visuals, prompt cuts in `how`, `why`, poteto-mode playbooks, and four principles, the `/poteto-mode` chat as the playbook runner, `create-verification-skill` usable readiness, plan-checker fences, `watch-pr` review-thread pagination, `worktree-audit.sh -b`), `plugins/pstack/agents/poteto-agent.md`, `README-UPSTREAM.md` | [cursor/plugins/pstack @ 1b6d44f](https://github.com/cursor/plugins/tree/1b6d44ff26d4e242f4e8f68c391e6d2f7ca16def/pstack) | (c) 2026 Lauren Tan | MIT | [LICENSE](LICENSE) | +| `plugins/pstack/skills/` (0.15.21 to 0.15.25 changes: the `in-your-own-words` and `sharpen` skills, per-spawn effort in poteto-mode and the routed skills, one full swarm per PR in the autopilots and the multi-phase plan, the Graphite-free `orch` frontier, the `watch-pr` owner-approval gate, Origin PRs in `worktree-audit.sh`, `ketchup` visuals and length, restated principles, and prompt cuts), `plugins/pstack/agents/poteto-agent.md`, `README-UPSTREAM.md` | [cursor/plugins/pstack @ 99743e9](https://github.com/cursor/plugins/tree/99743e9c151c8537cbbf1f98bb38f34ef3368453/pstack) | (c) 2026 Lauren Tan | MIT | [LICENSE](LICENSE) | ## What changed in the port @@ -64,3 +65,5 @@ Files authored for this port (not derived from upstream): - `REVIEW.md` (the review panel's usage and waiver-scope mandate, #153; adapted from mastra-pilot's `REVIEW.md`, not from upstream) - `CHANGES.md` - `LICENSE-cursor-team-kit` (copied verbatim from upstream cursor-team-kit MIT) + +Adam fork supplements Open Pstack 1.15.0 with Cursor pstack skill changes through d457567332aa6989e753ea289f30ea2bfbc80bc0 (0.15.28), including create-formal-verification-skill and maintain-formal-verification-skill. The existing Cursor license applies. diff --git a/README-UPSTREAM.md b/README-UPSTREAM.md index 5e34dbff..4e8b8c2a 100644 --- a/README-UPSTREAM.md +++ b/README-UPSTREAM.md @@ -33,6 +33,8 @@ that's it. the other skills are situational; the mode skill uses them for you as use [`/poteto-mode`](./skills/poteto-mode/SKILL.md) at the start of a task. it reads your request, picks from a set of playbooks, and runs the other skills as the steps need them. +pick `/poteto-mode` from the `/` menu before you type your task, so cursor attaches the skill to the message. + ### just use [`/poteto-mode`](./skills/poteto-mode/SKILL.md) this skill is the main shortcut. i use it whenever i need the agent to do rigorous engineering work. it comes with twenty-three playbooks: @@ -111,10 +113,12 @@ to keep [`/poteto-mode`](./skills/poteto-mode/SKILL.md) on across turns, pick it |---|---| | [`/poteto-mode`](./skills/poteto-mode/SKILL.md) | default entry point for any non-trivial task. | | [`/poteto-help`](./skills/poteto-help/SKILL.md) | you're new to pstack, or unsure which skill, playbook, or principle fits. finds out what you're trying to do, answers that part, and hands you a prompt to type. runs only when you type `/poteto-help`. | +| [`/in-your-own-words`](./skills/in-your-own-words/SKILL.md) | you want the agent to play back your goal and the problem before it starts. runs only when you type `/in-your-own-words`. | +| [`/sharpen`](./skills/sharpen/SKILL.md) | you want a task approached the way a top expert would, aimed at what you're really after, without overspecifying it or deciding your choices for you. | | [`/how`](./skills/how/SKILL.md) | you want a walkthrough of how a subsystem works. | | [`/why`](./skills/why/SKILL.md) | you want to know why something was built this way. discovers available MCPs at run time and queries each evidence category in parallel (source control, issue tracker, long-form docs, real-time chat, infra observability, error tracking, analytics warehouse). | | [`/recall`](./skills/recall/SKILL.md) | you're starting or resuming work and want your recent context on a topic rebuilt from your own chat history and the shared record, handed back as a tight current-state brief. | -| [`/ketchup`](./skills/ketchup/SKILL.md) | you want everything in this chat since your last message, in plain words, with an executive summary for each action item so you can make the call. | +| [`/ketchup`](./skills/ketchup/SKILL.md) | you want what changed in this chat since your last message, in a few plain sentences and small visuals, with just enough context on each action item to make the call. | | [`/blast-radius`](./skills/blast-radius/SKILL.md) | you have a small-looking change and want to know what else it could break, with the one fact it's safe because of proven by running code, not asserted. | | [`/architect`](./skills/architect/SKILL.md) | you're about to write code that crosses a function boundary and want the caller's usage, types, and module shape settled first. | | [`/arena`](./skills/arena/SKILL.md) | you want N parallel attempts at the same thing, then to grab the best parts of each. | @@ -211,7 +215,7 @@ twenty-four short skills, one principle each. `poteto-mode` indexes them inline | [laziness-protocol](./skills/principle-laziness-protocol/SKILL.md) | core | Bias toward deletion and the smallest change that solves the problem. | | [foundational-thinking](./skills/principle-foundational-thinking/SKILL.md) | core | Apply before writing logic: choosing core types and data structures, sequencing scaffold-vs-feature work, asking what concurrent actors share. Get the data structures right so downstream code becomes obvious. | | [redesign-from-first-principles](./skills/principle-redesign-from-first-principles/SKILL.md) | core | Redesign as if the requirement had been a foundational assumption from day one, instead of bolting it on. | -| [attack-the-premise](./skills/principle-attack-the-premise/SKILL.md) | core | Apply when two or more fixes that share one premise have failed the same gate. Take a census of which actors hold the imbalance before the next fix, then question the premise instead of writing another fix that assumes it. | +| [attack-the-premise](./skills/principle-attack-the-premise/SKILL.md) | core | Apply when two or more fixes that share one premise have failed the same gate. Write the premise down and test it directly before you write another fix that assumes it. | | [subtract-before-you-add](./skills/principle-subtract-before-you-add/SKILL.md) | core | Remove dead weight, redundant validators, and stub references first, then build on the simpler base. | | [minimize-reader-load](./skills/principle-minimize-reader-load/SKILL.md) | core | Count layers between question and answer, and hidden state in the reader's head; collapse one-caller wrappers and shrink mutable scope. | | [outcome-oriented-execution](./skills/principle-outcome-oriented-execution/SKILL.md) | core | Apply during planned rewrites and migrations with explicit phase boundaries. Converge on the target architecture; don't preserve smooth intermediate states with throwaway compatibility code. | @@ -220,14 +224,14 @@ twenty-four short skills, one principle each. `poteto-mode` indexes them inline | [build-the-lever](./skills/principle-build-the-lever/SKILL.md) | core | Apply to any non-trivial work, not just bulk work: edits, migrations, analyses, checks. Build the tool that does it or proves it (codemod, script, generator, or a skill your subagents follow) instead of working by hand. The tool is the artifact a reviewer can rerun. | | [model-the-domain](./skills/principle-model-the-domain/SKILL.md) | architecture | Encode the domain in a structure instead of scattered conditionals. | | [boundary-discipline](./skills/principle-boundary-discipline/SKILL.md) | architecture | Concentrate guards at system boundaries (CLI, config, network, external APIs); trust internal types and keep business logic in pure functions. | -| [type-system-discipline](./skills/principle-type-system-discipline/SKILL.md) | architecture | Make illegal states unrepresentable, brand semantic primitives, parse external data at boundaries, refuse to lie to the compiler, exhaust variants, derive from authoritative schemas. | +| [type-system-discipline](./skills/principle-type-system-discipline/SKILL.md) | architecture | Make illegal states unrepresentable, brand semantic primitives, parse external data at boundaries, exhaust variants, derive from authoritative schemas. | | [make-operations-idempotent](./skills/principle-make-operations-idempotent/SKILL.md) | architecture | Converge to the same end state regardless of partial prior runs. | | [migrate-callers-then-delete-legacy-apis](./skills/principle-migrate-callers-then-delete-legacy-apis/SKILL.md) | architecture | Migrate callers and delete the old API in the same wave instead of preserving compatibility layers. | | [separate-before-serializing-shared-state](./skills/principle-separate-before-serializing-shared-state/SKILL.md) | architecture | Eliminate the sharing first; serialize structurally only when one shared writer is a real invariant. | | [prove-it-works](./skills/principle-prove-it-works/SKILL.md) | verification | Apply after completing a task, before declaring done. Verify against the real artifact (run the feature, read the actual value, inspect the diff), not a proxy, self-report, or 'it compiles.'. | -| [fix-root-causes](./skills/principle-fix-root-causes/SKILL.md) | verification | Trace each symptom to its root cause and fix it there; reproduce first, ask why until you reach it, resist nil-check guards that silence crashes. | +| [fix-root-causes](./skills/principle-fix-root-causes/SKILL.md) | verification | Trace each symptom back to the first place the system went wrong and fix it there. A guard is a fix only when the state it handles is legitimate. | | [sequence-verifiable-units](./skills/principle-sequence-verifiable-units/SKILL.md) | verification | Apply to multi-step work (sweeps, migrations, runs of similar edits) and to how you stack commits and PRs. Break work into small units that each end in a verifiable state, check each before the next, and order delivery so the sequence proves itself to a reviewer. | -| [test-behavior-not-implementation](./skills/principle-test-behavior-not-implementation/SKILL.md) | verification | Apply when you write, change, or keep a test. Call the code the way its users do and assert the result they observe against a literal expected value. If the test would still pass when every imported function returns undefined, rewrite the assertion or delete the test. | +| [test-behavior-not-implementation](./skills/principle-test-behavior-not-implementation/SKILL.md) | verification | Apply when you write, change, or keep a test. Assert what the code's users can observe, so the test fails when that behavior breaks and keeps passing when only the internals change. | | [explain-the-number](./skills/principle-explain-the-number/SKILL.md) | verification | Apply before you trust, report, or act on a number you measured: a speedup, a regression, a throughput, a latency, or an eval result. Find what limits it, and rule out that it measured something other than the work you think. | | [guard-the-context-window](./skills/principle-guard-the-context-window/SKILL.md) | delegation | Route bulk to subagents; keep summaries in the main thread, not raw payloads. | | [never-block-on-the-human](./skills/principle-never-block-on-the-human/SKILL.md) | delegation | Proceed, present the result, let the human course-correct after the fact; reserve confirmation for irreversible actions. | diff --git a/README.md b/README.md index f1f8d6c9..d09e6989 100644 --- a/README.md +++ b/README.md @@ -91,9 +91,9 @@ In Codex, ask: Use pstack:setup-pstack to configure pstack. ``` -Setup checks the models you can actually run, shows how each one will start, and asks before saving the choices. The default review panel uses Opus, Sol, and Grok, and setup probes only the models your roles use. Fable remains available. +Setup checks the models you can actually run, shows how each one will start, and asks before saving the choices. The default review panel uses Opus, Sol, and Grok, and setup probes only the models your roles use. Fable, Sonnet, Luna and an exact Opus pin remain available. -An older model sheet starts using the rolling aliases in memory as soon as this release is installed. Run setup once after updating to persist that migration. It replaces versioned Fable and Opus entries while preserving every role assignment and effort selection. +An older model sheet starts using the rolling aliases in memory as soon as this release is installed. Run setup once after updating to persist that migration. It replaces versioned entries of a rolling-alias family, such as Opus, while preserving every role assignment and effort selection. A registered exact pin is kept. A model sheet from an earlier release keeps its panel. To take the new defaults, delete those role lines and run setup again; setup fills missing roles from the defaults. An entry for a model that a later release replaced keeps running until the next setup run asks you to replace it. @@ -132,6 +132,8 @@ That is the main workflow. The other skills are there when poteto-mode needs the | `reflect` | A hard task is finished and its lessons should improve the next run. | | `correct` | Agents keep making the same mistake in a repository and you want a check that stops it. | | `poteto-help` | You want to know which skill, playbook, or principle fits a question. | +| `in-your-own-words` | You want the agent to restate your goal and the problem before it starts. | +| `sharpen` | You want a task approached the way a top expert would, aimed at what you are really after, without overspecifying it. | Plugin skills include `pstack:` in their name. In Claude Code, invoke a native skill such as `/pstack:architect`. In Codex, ask for the skill, such as `Use pstack:architect for this design.` See the [technical reference](docs/reference.md) for the full list. @@ -168,7 +170,7 @@ This repository also keeps: ## Staying close to Lauren's pstack -Open Pstack 1.8.0 tracks pstack 0.15.20 at Cursor commit [`1b6d44ff26d4e242f4e8f68c391e6d2f7ca16def`](https://github.com/cursor/plugins/commit/1b6d44ff26d4e242f4e8f68c391e6d2f7ca16def). +Open Pstack 1.15.0 tracks pstack 0.15.25 at Cursor commit [`99743e9c151c8537cbbf1f98bb38f34ef3368453`](https://github.com/cursor/plugins/commit/99743e9c151c8537cbbf1f98bb38f34ef3368453). The two projects have separate version numbers. The pstack version identifies Lauren's upstream content. The Open Pstack version identifies the Claude Code and Codex package built from it. diff --git a/REVIEW.md b/REVIEW.md index 7c4f94b4..ba1ff664 100644 --- a/REVIEW.md +++ b/REVIEW.md @@ -45,5 +45,7 @@ Each scoped sentence in this section names a concrete entry point or bounded sur - In `.claude/skills/verify-open-pstack/**`, a failure that needs the model under test to deliberately evade a recipe, such as a crafted shell spelling, a value hidden in a variable or a report echoed without its lookup, is out of scope even when it would change the gate decision; a wrong decision on an honest run stays in scope. - In `plugins/pstack/**`, a failure that needs a host missing a prerequisite named in `docs/reference.md` (Dependencies) or `provider-dispatch.md` (Host and parent prerequisites) is out of scope, except for the behavior those files promise on such a host. - On such a host, `plugins/pstack/**` must keep the promised behavior: `worktree-audit.sh` warns without `jq` or `rg`, a blocked external CLI is a loud dropout, and a Grok lane without Landlock or `bwrap` ends `child-failed` with Grok's refusal in its receipt. +- In `plugins/pstack/skills/poteto-mode/scripts/runner/**`, a read-only lane means the read-only settings pstack passes to the provider CLI; the runner adds no refusal, guard or account check because of the operator's host, account, repository or organization configuration, and the catalog notes disclose what that configuration can do (operator policy, 2026-10-16, #203). +- In `plugins/pstack/skills/poteto-mode/scripts/runner/**`, a failure that needs configuration from the operator's host, account, repository or organization, such as hooks, plugins, managed or remote settings, project config or stored logins, to widen a read-only lane is out of scope (operator policy, 2026-10-16, #203). - The `*.test.ts` and `*.test-helper.ts` files under `plugins/pstack/**` run only through `bun run test` in Ubuntu CI and on the operator's Mac; a test failure that needs a git configuration or host neither of them has, such as a global `protocol.file.allow=never`, is out of scope. - No exclusions apply to findings in `.github/workflows/**`, `.github/unfret.json` or `.mergify.yml`; PR data, labels, comments and API responses reaching that rail surface are untrusted, and every finding there stands on its consequence class. diff --git a/UPSTREAM.md b/UPSTREAM.md index 82e39749..ef29841b 100644 --- a/UPSTREAM.md +++ b/UPSTREAM.md @@ -8,11 +8,11 @@ open-pstack tracks [Cursor's pstack](https://github.com/cursor/plugins/tree/main | --- | --- | | Repository | `https://github.com/cursor/plugins.git` | | Path | `pstack/` | -| Commit | `1b6d44ff26d4e242f4e8f68c391e6d2f7ca16def` | -| Upstream version | `0.15.20` | -| open-pstack version | `1.8.0` | +| Commit | `99743e9c151c8537cbbf1f98bb38f34ef3368453` | +| Upstream version | `0.15.25` | +| open-pstack version | `1.15.0-adam.20261019` | -The table above is the current Cursor sync point. Open Pstack 1.7.0 imported this 0.15.20 sync, and 1.8.0 keeps it. `README-UPSTREAM.md` preserves the upstream pstack README verbatim. `CHANGES.md` and `NOTICE.md` describe the adaptations and provenance. +The table above is the current Cursor sync point. Open Pstack 1.10.0 imported this 0.15.25 sync, and 1.11.0 through 1.15.0 keep it. `README-UPSTREAM.md` preserves the upstream pstack README verbatim. `CHANGES.md` and `NOTICE.md` describe the adaptations and provenance. ## Upstream-only exclusions @@ -20,20 +20,20 @@ The table above is the current Cursor sync point. Open Pstack 1.7.0 imported thi - Four `disable-model-invocation: true` lines from `73f8be4` are not applied to `how`, `why`, `unslop`, or `typescript-best-practices`. Poteto-mode invokes those skills by name, and the flag blocks that route on Claude Code. - The default-model hunks for `bug-fix`, `perf-issue`, and `hillclimb` from `23a56e2`, `889ec4b`, and `70b2dc8` are not applied. Those frequent code-writing roles stay on `codex:gpt-6.1-sol@max`. - `5bf2b15`'s setup budget question, its `# budget` line, and its step down to a lower detected effort are not applied. Setup already asks one requested effort per assigned family, and the step-down would silently lower a requested effort. -- `12d587d`'s rule that reruns a rejected configured entry on its family default or the closest valid slug is not applied. An unavailable model stays a named dropout per `provider-dispatch.md`. -- The expected-runtime column in `70b2dc8`'s `children.tsv` and its expected-runtime stuck test are not applied. A lane is stuck only on affirmative failure evidence. +- `12d587d`'s rule that reruns a rejected configured entry on its family default or the closest valid slug is not applied, and neither is `ce649c0`'s rule that puts `cursor-grok-*` and `composer-*` entries in the Grok family for that fallback. An unavailable model stays a named dropout per `provider-dispatch.md`. +- The expected-runtime column in `70b2dc8`'s `children.tsv` and its expected-runtime stuck test are not applied, and neither are `ce649c0`'s stuck-after runtime and its rule that an owner with no push or trail row by the first tick is stuck. A lane is stuck only on affirmative failure evidence. An owner with no push or trail row at a tick is drift for that owner to record. - The explicit Grok, Opus, and Sol defaults for the Why and Reflect roles are not applied. Those roles stay on `inherit-parent` because the external runner omits the parent's MCP servers. - The Claude manifest does not take the logo field from `efa2a53` because Claude Code has no schema for it. The shared asset is exposed through the Codex manifest instead. - Commit `df58112` (pstack 0.15.15) is not applied. It drops Sol from every default, cuts the panels to Opus and Grok, and moves Opus from `max` to `xhigh`. Open Pstack keeps its own model defaults and three-model panels. -- The `disable-model-invocation: true` lines that `23e4138` puts on `benchmark-checklist` and `principle-explain-the-number` are not applied. The same flag, which upstream carried on principle leaves and `figure-it-out` before 0.15.5, stays off as in earlier syncs. Principle leaves keep `user-invocable: false`, and `figure-it-out` and `benchmark-checklist` stay model-invocable because poteto-mode routes to them by name. The typed-only `correct`, `ketchup`, and `poteto-help` keep the flag. +- The `disable-model-invocation: true` lines that `23e4138` puts on `benchmark-checklist` and `principle-explain-the-number` are not applied. The same flag, which upstream carried on principle leaves and `figure-it-out` before 0.15.5, stays off as in earlier syncs. Principle leaves keep `user-invocable: false`, and `figure-it-out` and `benchmark-checklist` stay model-invocable because poteto-mode routes to them by name. The typed-only `correct`, `ketchup`, `poteto-help`, `in-your-own-words`, and `sharpen` keep the flag. - The Built-in PR tool paragraph from `23e4138` and `00b14f1` is not applied. It describes a Cursor run tool that Claude Code and Codex do not have. -- `00b14f1`'s shorter `principle-test-behavior-not-implementation` text is not applied. It fails every restated constant, table row, or prompt string and every absence-only assertion (`not.toHaveBeenCalled`, `toBeUndefined`, `toEqual([])`), and the 0.15.0 local correction keeps both kinds when they test a real contract. The poteto-mode index line for this principle takes `00b14f1`'s format change and keeps the 0.15.0 wording. - The 40-line squash-body cutoff returns in `00b14f1` and stays omitted, as in the 0.15.0 sync. -- Autopilot-full step 5's rule from `23e4138` is not applied. That rule lets a green head with an unchanged patch ID merge without another rebase after trunk moves, when `git merge-tree` is clean and trunk touched none of the PR's paths or CI paths. Both checks pass when trunk removes an export that the PR's new code imports, so the combined tree merges without a build or test. Step 5 keeps sending a trunk move before the merge through the patch ID rule in `playbooks/shipping.md`. -- `correct` step 3 from `9511e60` says to fix or delete any test that would still pass if every function it calls returned nothing. The port sends that step to `principle-test-behavior-not-implementation`, which keeps negative-path tests whose correct result is an absence. +- Autopilot-full step 5's trunk-move rule from `23e4138`, and its revision in `ce649c0`, are not applied. The `23e4138` rule merged without another rebase when `git merge-tree` was clean and trunk touched none of the PR's paths or CI paths, and both checks pass when trunk removes an export that the PR's new code imports. `ce649c0` replaces those checks with the owner's judgment: merge when trunk's changes cannot affect the patch, and run the patch's checks on the merge result when trunk changed a caller or a dependency. An owner who misses a dependency still merges an untested tree. Step 5 keeps the rebase at merge prep and sends a trunk move before the merge through the patch ID rule in `playbooks/shipping.md`. `ce649c0`'s lockfile exception and its hold before a third rebase go with the rule. - `poteto-agent.md` does not take `is_background: true` from `2bd5dd4`. Claude Code documents `background`, not `is_background`, in agent frontmatter, and Codex defines agents in TOML and maps concurrency at dispatch. The port keeps `general-purpose` in the description and the `skills: [pstack:poteto-mode]` preload. -- The root Agent Plugins v1 `pstack/plugin.json` from `e5a02f8` and its `1b6d44f` version bump are not applied. The port keeps its current Claude manifest at `plugins/pstack/.claude-plugin/plugin.json` and Codex manifest at `plugins/pstack/.codex-plugin/plugin.json`. Agent Plugins v1 is revisited with #13. +- The root Agent Plugins v1 `pstack/plugin.json` from `e5a02f8` and its version bumps through 0.15.25 are not applied. The port keeps its current Claude manifest at `plugins/pstack/.claude-plugin/plugin.json` and Codex manifest at `plugins/pstack/.codex-plugin/plugin.json`. Agent Plugins v1 is revisited with #13. - `unslop` takes `bc697d0`'s description without its `disable-model-invocation: true` line, as for `73f8be4` above. +- `pstack/.cursor-plugin/plugin.json` and `pstack/docs/guide/` are not imported. The Cursor manifest has no Claude Code or Codex consumer, and the guide shows Cursor's interface; `poteto-help` links to the guide upstream. That covers the 0.15.21 to 0.15.25 version bumps and `e5136d8`'s guide edit. +- `poteto-agent.md` does not take `b93e811`'s search of `~/.cursor/plugins/` for poteto-mode, its highest-version rule, or its note that the skill catalog hides poteto-mode. The Claude agent preloads `pstack:poteto-mode`, the port's poteto-mode is not hidden, and Codex has no `poteto-agent` type. The agent keeps upstream's "Use the absolute path your parent gave you." ## Local port-patch ledger @@ -42,6 +42,8 @@ The table above is the current Cursor sync point. Open Pstack 1.7.0 imported thi - **#157 — role pointers**: Cursor names model descriptors in the Arena, Architect, How, Interrogate, and Swarm skills, the bug-fix, feature, hillclimb, perf-issue, and refactoring playbooks, poteto-mode's delegation defaults, and setup's probe table and first-run sheet. The port replaces each with a role pointer: the role's backticked sheet label and a link to `plugins/pstack/skills/poteto-mode/references/model-catalog.md#role-defaults`, which `registry/render.ts` generates from `plugins/pstack/skills/poteto-mode/scripts/runner/registry/models.ts`. Interrogate's reviewer table and setup's probe table and example sheet are deleted. When an upstream sync changes a model slug on one of these lines, keep the port's line and put the model change in `registry/models.ts`. `runner/registry.test.ts` enforces the rule: `ROLE_POINTERS` names each file's roles, and its text scan rejects any descriptor or versioned model name in `plugins/pstack/**/*.md`, `README.md`, and `docs/reference.md` outside the generated catalog and agents. +- **#191 — landing playbooks and per-spawn effort**: `ce649c0` cuts Shipping to six steps and both autopilots to principles. The port takes the cut's rules and reasons but keeps the landing procedure from 1.3.0: Shipping keeps steps 1 to 9, Autopilot-full step 2 holds the guarded rebase and push, and Autopilot-stack step 2 holds the fork and topology rules. Those steps disarm auto-merge and merge-queue entries before verification and before any rewrite, keep the verdict head and base apart from the landing head and base, resolve the base fetch remote apart from the head push URL, bind each push to a captured SHA, and merge only through a server-enforced expected head. The named landing invariants in `tests/skill-collision-repro.sh` check those rules by step and phrase, so a sync that moves them updates the invariants in the same change. `ce649c0`'s Match the model to the job rule is taken with port adaptations: the starting level is the role's configured effort, an unlisted effort moves to the nearest effort the catalog row lists, a configured `ultra` stays, and the fast and standard speed sentences are dropped because the port's descriptors carry no speed selector. + ## Check for changes The repository already names Cursor's repository as the `cursor` remote in the maintainer checkout. A fresh clone can add it once: @@ -54,8 +56,8 @@ Fetch and inspect only commits that touched pstack after the recorded sync point ```shell git fetch cursor main -git log --oneline 1b6d44ff26d4e242f4e8f68c391e6d2f7ca16def..cursor/main -- pstack -git diff --stat 1b6d44ff26d4e242f4e8f68c391e6d2f7ca16def..cursor/main -- pstack +git log --oneline 99743e9c151c8537cbbf1f98bb38f34ef3368453..cursor/main -- pstack +git diff --stat 99743e9c151c8537cbbf1f98bb38f34ef3368453..cursor/main -- pstack ``` No output means the tracked pstack tree has not changed. This comparison does not need a polling service or generated mirror branch. @@ -70,3 +72,7 @@ No output means the tracked pstack tree has not changed. This comparison does no 6. Merge the reviewed PR before tagging the next open-pstack release. Publish the release with `python3 scripts/release.py publish` from a clean checkout of `main`. Cursor's version and open-pstack's version are independent. Cursor's version identifies the imported content; open-pstack's version identifies the cross-harness distribution. + +## Adam fork supplemental sync + +The October 19 fork layers Cursor pstack 0.15.28 skill changes from `d457567332aa6989e753ea289f30ea2bfbc80bc0` over Open Pstack 1.15.0. `FORK.md` records the imports and translation decisions. The upstream sync table above describes the released port, not this supplemental import. diff --git a/docs/reference.md b/docs/reference.md index b5ca5abe..0ce7a8e8 100644 --- a/docs/reference.md +++ b/docs/reference.md @@ -2,7 +2,7 @@ This page contains the full skill, dependency, runtime, and porting reference. For the plain-English introduction and quick start, see the [main README](../README.md). -[Poteto](https://x.com/poteto)'s [pstack](https://github.com/cursor/plugins/tree/main/pstack), adapted to run in Claude Code and Codex without Cursor. One shared skill tree serves both harnesses; Grok remains available as a model-provider lane. Version 1.8.0 is synced to Cursor pstack v0.15.20 at `1b6d44ff26d4e242f4e8f68c391e6d2f7ca16def`. See [UPSTREAM.md](../UPSTREAM.md) for the exact sync contract. +[Poteto](https://x.com/poteto)'s [pstack](https://github.com/cursor/plugins/tree/main/pstack), adapted to run in Claude Code and Codex without Cursor. One shared skill tree serves both harnesses; Grok remains available as a model-provider lane. Version 1.15.0 is synced to Cursor pstack v0.15.25 at `99743e9c151c8537cbbf1f98bb38f34ef3368453`. See [UPSTREAM.md](../UPSTREAM.md) for the exact sync contract. Original by Lauren Tan. This distribution builds on Michael Denyer's [pstack-claude](https://github.com/michael-denyer/pstack-claude) port and retains its history and MIT attribution. It imports seven MIT-licensed skills from [cursor-team-kit](https://github.com/cursor/plugins/tree/main/cursor-team-kit): `deslop`, `thermo-nuclear-code-quality-review`, `make-pr-easy-to-review`, `fix-ci`, `fix-merge-conflicts`, `get-pr-comments`, `what-did-i-get-done`. @@ -59,11 +59,11 @@ The marketplace install is the normal user path. Direct links are only for testi ├── plugins/pstack/ # the plugin itself │ ├── .claude-plugin/plugin.json # Claude Code manifest │ ├── .codex-plugin/plugin.json # Codex manifest (skills: ./skills/) -│ ├── skills/ # 59 skills shared by Claude Code and Codex +│ ├── skills/ # 61 skills shared by Claude Code and Codex │ │ ├── poteto-mode/references/{codex-tools,provider-dispatch}.md # tool + provider routing │ │ └── poteto-mode/scripts/ # bun/bash/node tooling: watch-pr, orch, runner, resolve-codex-transcript, check-plan.mjs, worktree-audit.sh │ ├── hooks/ # SessionStart auto-fire: injects the poteto-mode mandate (Claude Code only) -│ └── agents/ # Claude subagents, including native Fable and Opus lanes at each selectable effort +│ └── agents/ # Claude subagents, including a native lane for each Claude family at each selectable effort ├── tests/skill-collision-repro.sh # native-skill package invariants and Claude invocation checks ├── LICENSE # pstack upstream MIT ├── LICENSE-cursor-team-kit # cursor-team-kit upstream MIT @@ -86,7 +86,7 @@ The Codex build shares one `skills/` tree with the Claude Code build. Nothing is - **Tool and built-in mapping.** Claude tool names and built-in skills resolve through [`codex-tools.md`](../plugins/pstack/skills/poteto-mode/references/codex-tools.md). Model execution resolves separately through [`provider-dispatch.md`](../plugins/pstack/skills/poteto-mode/references/provider-dispatch.md), so Codex can keep Sol native while invoking Claude and Grok externally. - **Subagents.** The `Agent` tool maps to Codex `spawn_agent` / `wait_agent`, enabled by `multi_agent = true`. Parallel fan-out is multiple `spawn_agent` calls in one turn. If the native Codex lane is unavailable, record that lane as a dropout; external Claude and Grok lanes still run, and no provider is silently substituted. There is no `poteto-agent` subagent type on Codex; route ad-hoc subagents by dispatching a `spawn_agent` told to read `poteto-mode` first. - **Auto-fire.** The `hooks/` SessionStart injection is Claude Code-only; Codex has no plugin hook runtime. Enter `pstack:poteto-mode` by name, or add a standing instruction to `~/.codex/AGENTS.md` if you want the same always-on routing. -- **Models.** `/setup-pstack` writes provider-qualified descriptors and asks one requested effort per assigned family (`low`, `medium`, `high`, `xhigh`, `max`, and `ultra` where the model's CLI lists it, today only Sol). The first-run sheet is the model catalog's [role defaults](../plugins/pstack/skills/poteto-mode/references/model-catalog.md#role-defaults), with a panel of Opus, Sol, and Grok; Fable stays selectable with its native agents. Fable and Opus use Claude's rolling aliases. Runtime dispatch normalizes older versioned descriptors in memory, so an installed sheet stops pinning immediately. A setup rerun persists that migration while keeping each role's family and effort. In Codex, Sol uses native `spawn_agent`; Claude and Grok use the deterministic external runner. In Claude Code, Fable and Opus use native agents; Sol and Grok use the runner. Children never detect the parent or reroute themselves. The `bug-fix`, `perf-issue`, and `hillclimb` roles stay on Sol instead of upstream's Grok default. +- **Models.** `/setup-pstack` writes provider-qualified descriptors and asks one requested effort per assigned family (`low`, `medium`, `high`, `xhigh`, `max`, and `ultra` where the model's CLI lists it, today only Sol). The first-run sheet is the model catalog's [role defaults](../plugins/pstack/skills/poteto-mode/references/model-catalog.md#role-defaults), with a panel of Opus, Sol, and Grok; Fable, Sonnet, Luna and an exact Opus pin stay selectable. Fable, Opus and Sonnet use Claude's rolling aliases; a registered exact pin keeps its own model. Runtime dispatch normalizes older versioned descriptors in memory, so an installed sheet stops pinning immediately. A setup rerun persists that migration while keeping each role's family and effort. In Codex, Codex families such as Sol use native `spawn_agent`; Claude and Grok use the deterministic external runner. In Claude Code, Claude families use native agents; Codex and Grok families use the runner. A Codex writer that the runner launches stays in Codex's `workspace-write` sandbox unless the sheet opts in with the line `codex writer sandbox: full-access`, which setup asks about once. Full access lets the writer launch a browser and write dot-folders such as `.agents/`, and removes every sandbox limit. Children never detect the parent or reroute themselves. The `bug-fix`, `perf-issue`, and `hillclimb` roles stay on Sol instead of upstream's Grok default. Verified in fresh installed Claude Code and Codex sessions: the user-facing skills are discovered and namespaced under `pstack`; both parents fan out the configured panel through the documented native/external route table, retain long-running handles without a default timeout, and cross-judge only after every candidate is terminal. The `principle-*` leaves remain available for `poteto-mode` to read by path. Claude honors their `user-invocable: false` metadata; Codex 0.149.0 does not ([#8](https://github.com/ericlitman/open-pstack/issues/8)). @@ -106,12 +106,12 @@ Nothing is declared in `plugin.json`. Install the one companion plugin yourself: Not declared as deps, but referenced in skill bodies: - **`run`, `verify`, `loop`** — Claude Code CLI built-ins (ship with the binary, always available). -- **`gh` (GitHub CLI).** This is the default forge for every stack playbook and a system-level requirement of the standalone `babysit` skill. Install it with [`brew install gh`](https://cli.github.com) and authenticate with `gh auth login`. If Origin's `origin` CLI is installed and can resolve the repository, the stack playbooks use it instead. Only the Orchestrate playbook and its `scripts/orch` frontier tooling still require `gt`. +- **`gh` (GitHub CLI).** This is the default forge for every stack playbook and a system-level requirement of the standalone `babysit` skill. Install it with [`brew install gh`](https://cli.github.com) and authenticate with `gh auth login`. If Origin's `origin` CLI is installed and can resolve the repository, the stack playbooks use it instead. No playbook requires Graphite (`gt`): Orchestrate's `orch frontier set` reads the PR base chain through `gh`, or through Origin with `--forge origin`. - **`bun`** — runs the vendored `skills/poteto-mode/scripts/` tooling (`watch-pr`, `orch`, `runner`) and the port-authored `resolve-codex-transcript` launcher. Install via [`brew install oven-sh/bun/bun`](https://bun.sh). `bootstrap.ts` installs dependencies for `watch-pr` and `orch`; the runner uses only Bun and Node built-ins, so it launches directly without an install/re-exec layer. - **`git` 2.25 or later** — the runner asks `git rev-parse --git-common-dir` for a linked worktree's shared git directory, and `worktree-audit.sh` reads `git worktree list --porcelain`. Ubuntu 20.04 LTS ships 2.25; older git is unsupported. - **Bash on Windows** — the SessionStart hook runs through `hooks/run-hook.cmd`, which looks for Git for Windows' `bash.exe` in its standard system-wide and per-user locations, then for a `bash` on `PATH` (MSYS2, Cygwin). Without one, the hook exits 0 and skips the pstack startup context, as the superpowers wrapper it comes from does. Install [Git for Windows](https://git-scm.com/download/win) to get the context. In Codex, the hook starts `C:\Windows\System32\cmd.exe` by full path, because one command must work in both PowerShell and cmd. A Windows installed outside `C:\Windows` does not run the hook. - **`node`** — runs `skills/poteto-mode/scripts/check-plan.mjs`. The checker uses only Node built-ins and does not need Bun. -- **Claude Code, Codex, and Grok Build CLIs** — the external runner uses the assigned subscribed CLI directly. Install and authenticate only the providers present in your model sheet. Same-provider work stays native; the runner refuses it. +- **Provider CLIs** — the external runner uses the assigned subscribed CLI directly, one per provider in the [model catalog's route table](../plugins/pstack/skills/poteto-mode/references/model-catalog.md#routes). Install and authenticate only the providers present in your model sheet. Some providers run read-only lanes only. The catalog notes for OpenCode, Amp, Cursor, Devin, agy and dsh say what repository and host configuration their CLIs load into a lane. Same-provider work stays native; the runner refuses it. - **`jq` and `rg` (ripgrep)** — only for `scripts/worktree-audit.sh` (the Worktree cleanup playbook). Without them the audit still runs but blanks its PR and LAST_CHAT columns, so it warns on stderr rather than returning a table that looks complete. No third-party plugins. The harsher-critique escape hatch lives in the bundled `thermo-nuclear-code-quality-review` skill (imported from cursor-team-kit), not in an external plugin. @@ -157,6 +157,8 @@ The table uses the short upstream names. Claude Code exposes each native skill w | `/correct` | turn mistakes agents keep repeating in a repo into checks that make them impossible (typed only) | | `/ketchup` | catch up on everything in this session since your last message (typed only) | | `/poteto-help` | find the skill, playbook, or principle for a question about pstack (typed only) | +| `/in-your-own-words` | play back your goal and the problem before work starts (typed only) | +| `/sharpen` | approach a task the way a top expert would, aimed at the real goal, without overspecifying it (typed only) | ## Subagents @@ -164,7 +166,7 @@ The table uses the short upstream names. Claude Code exposes each native skill w `comment-sicko` is the read-only comment reviewer the `no-comments` skill spawns. Upstream names it `Comment Sicko`; the port renames it to `comment-sicko` so the name is a valid `subagent_type`. Invoke it through `/no-comments`, not directly. -Fable and Opus each ship at `low`, `medium`, `high`, `xhigh`, and `max`. Names are `pstack--`. `pstack-fable-max` and `pstack-opus-xhigh` remain. Each file selects the rolling family alias and requested effort, runs in the background, and denies nested Agent/Task dispatch. pstack dispatches them from provider-qualified descriptors; they are not user-facing workflows. +Each Claude family in the [model catalog](../plugins/pstack/skills/poteto-mode/references/model-catalog.md) ships a generated agent at each selectable effort. Names are `pstack--`. `pstack-fable-max` and `pstack-opus-xhigh` remain. Each file selects the family's model, a rolling alias or an exact pin, and the requested effort, runs in the background, and denies nested Agent/Task dispatch. pstack dispatches them from provider-qualified descriptors; they are not user-facing workflows. ## Differences from upstream diff --git a/package.json b/package.json new file mode 100644 index 00000000..4560426a --- /dev/null +++ b/package.json @@ -0,0 +1,10 @@ +{ + "name": "open-pstack", + "private": true, + "packageManager": "bun@1.4.0", + "scripts": { + "test": "cd plugins/pstack/skills/poteto-mode/scripts && bun install --frozen-lockfile && bun run test", + "typecheck": "cd plugins/pstack/skills/poteto-mode/scripts && bun install --frozen-lockfile && bun run typecheck", + "static": "bun -e 'for (const path of process.argv.slice(1)) JSON.parse(await Bun.file(path).text())' .claude-plugin/marketplace.json .agents/plugins/marketplace.json plugins/pstack/.claude-plugin/plugin.json plugins/pstack/.codex-plugin/plugin.json && PSTACK_STATIC_ONLY=1 bash tests/skill-collision-repro.sh" + } +} diff --git a/plugins/pstack/.claude-plugin/plugin.json b/plugins/pstack/.claude-plugin/plugin.json index 5fc404ff..3e8d4cf9 100644 --- a/plugins/pstack/.claude-plugin/plugin.json +++ b/plugins/pstack/.claude-plugin/plugin.json @@ -1,7 +1,7 @@ { "name": "pstack", "displayName": "pstack", - "version": "1.8.0", + "version": "1.15.0-adam.20261019", "description": "if you want to go fast, go deep first. pstack helps you write less, but higher quality code. rigorous agent workflows you can parallelize with confidence. Ported from cursor/plugins/pstack for Claude Code and Codex.", "author": { "name": "Lauren Tan" diff --git a/plugins/pstack/.codex-plugin/plugin.json b/plugins/pstack/.codex-plugin/plugin.json index 6a1166c4..ef15a9d9 100644 --- a/plugins/pstack/.codex-plugin/plugin.json +++ b/plugins/pstack/.codex-plugin/plugin.json @@ -1,6 +1,6 @@ { "name": "pstack", - "version": "1.8.0", + "version": "1.15.0-adam.20261019", "description": "if you want to go fast, go deep first. pstack helps you write less, but higher quality code. rigorous agent workflows you can parallelize with confidence. Codex port of the Claude Code plugin; skills are shared, tool names resolve via skills/poteto-mode/references/codex-tools.md.", "author": { "name": "Lauren Tan" diff --git a/plugins/pstack/agents/poteto-agent.md b/plugins/pstack/agents/poteto-agent.md index d05459d8..a5dd78e2 100644 --- a/plugins/pstack/agents/poteto-agent.md +++ b/plugins/pstack/agents/poteto-agent.md @@ -7,4 +7,4 @@ skills: # Poteto subagent -You are operating as poteto-mode's full agent style. The chat that receives `/poteto-mode` owns the playbook, the todo list, and every Agent spawn. You are one of its delegates. Read the `poteto-mode` skill's `SKILL.md` in full before doing any work, including its inline Principles index. Navigate to a leaf `principle-*` skill whenever you apply that principle. +You are operating as poteto-mode's full agent style. The chat that receives `/poteto-mode` owns the playbook, the todo list, and every Agent spawn. You are one of its delegates. Read the `poteto-mode` skill's `SKILL.md` in full before doing any work, including its inline Principles index. Use the absolute path your parent gave you. Navigate to a leaf `principle-*` skill whenever you apply that principle. diff --git a/plugins/pstack/agents/pstack-opus-5-5-high.md b/plugins/pstack/agents/pstack-opus-5-5-high.md new file mode 100644 index 00000000..149b0ac9 --- /dev/null +++ b/plugins/pstack/agents/pstack-opus-5-5-high.md @@ -0,0 +1,12 @@ +--- +name: pstack-opus-5-5-high +description: Native Claude lane for pstack roles configured as claude:claude-opus-5-5@high. +model: claude-opus-5-5 +effort: high +background: true +disallowedTools: Agent, Task +--- + +# pstack Opus 5.5 lane + +Execute only the task and path scope the parent assigns. Read the grounding artifacts by path. Do not choose another model, spawn another agent, or start a pstack workflow. If the assignment is read-only, do not modify files. Return the requested artifact or verdict plus a concise rationale. diff --git a/plugins/pstack/agents/pstack-opus-5-5-low.md b/plugins/pstack/agents/pstack-opus-5-5-low.md new file mode 100644 index 00000000..19196970 --- /dev/null +++ b/plugins/pstack/agents/pstack-opus-5-5-low.md @@ -0,0 +1,12 @@ +--- +name: pstack-opus-5-5-low +description: Native Claude lane for pstack roles configured as claude:claude-opus-5-5@low. +model: claude-opus-5-5 +effort: low +background: true +disallowedTools: Agent, Task +--- + +# pstack Opus 5.5 lane + +Execute only the task and path scope the parent assigns. Read the grounding artifacts by path. Do not choose another model, spawn another agent, or start a pstack workflow. If the assignment is read-only, do not modify files. Return the requested artifact or verdict plus a concise rationale. diff --git a/plugins/pstack/agents/pstack-opus-5-5-max.md b/plugins/pstack/agents/pstack-opus-5-5-max.md new file mode 100644 index 00000000..7ca1c388 --- /dev/null +++ b/plugins/pstack/agents/pstack-opus-5-5-max.md @@ -0,0 +1,12 @@ +--- +name: pstack-opus-5-5-max +description: Native Claude lane for pstack roles configured as claude:claude-opus-5-5@max. +model: claude-opus-5-5 +effort: max +background: true +disallowedTools: Agent, Task +--- + +# pstack Opus 5.5 lane + +Execute only the task and path scope the parent assigns. Read the grounding artifacts by path. Do not choose another model, spawn another agent, or start a pstack workflow. If the assignment is read-only, do not modify files. Return the requested artifact or verdict plus a concise rationale. diff --git a/plugins/pstack/agents/pstack-opus-5-5-medium.md b/plugins/pstack/agents/pstack-opus-5-5-medium.md new file mode 100644 index 00000000..2447ab07 --- /dev/null +++ b/plugins/pstack/agents/pstack-opus-5-5-medium.md @@ -0,0 +1,12 @@ +--- +name: pstack-opus-5-5-medium +description: Native Claude lane for pstack roles configured as claude:claude-opus-5-5@medium. +model: claude-opus-5-5 +effort: medium +background: true +disallowedTools: Agent, Task +--- + +# pstack Opus 5.5 lane + +Execute only the task and path scope the parent assigns. Read the grounding artifacts by path. Do not choose another model, spawn another agent, or start a pstack workflow. If the assignment is read-only, do not modify files. Return the requested artifact or verdict plus a concise rationale. diff --git a/plugins/pstack/agents/pstack-opus-5-5-xhigh.md b/plugins/pstack/agents/pstack-opus-5-5-xhigh.md new file mode 100644 index 00000000..eba89da7 --- /dev/null +++ b/plugins/pstack/agents/pstack-opus-5-5-xhigh.md @@ -0,0 +1,12 @@ +--- +name: pstack-opus-5-5-xhigh +description: Native Claude lane for pstack roles configured as claude:claude-opus-5-5@xhigh. +model: claude-opus-5-5 +effort: xhigh +background: true +disallowedTools: Agent, Task +--- + +# pstack Opus 5.5 lane + +Execute only the task and path scope the parent assigns. Read the grounding artifacts by path. Do not choose another model, spawn another agent, or start a pstack workflow. If the assignment is read-only, do not modify files. Return the requested artifact or verdict plus a concise rationale. diff --git a/plugins/pstack/agents/pstack-sonnet-high.md b/plugins/pstack/agents/pstack-sonnet-high.md new file mode 100644 index 00000000..86b918f8 --- /dev/null +++ b/plugins/pstack/agents/pstack-sonnet-high.md @@ -0,0 +1,12 @@ +--- +name: pstack-sonnet-high +description: Native Claude lane for pstack roles configured as claude:sonnet@high. +model: sonnet +effort: high +background: true +disallowedTools: Agent, Task +--- + +# pstack Sonnet lane + +Execute only the task and path scope the parent assigns. Read the grounding artifacts by path. Do not choose another model, spawn another agent, or start a pstack workflow. If the assignment is read-only, do not modify files. Return the requested artifact or verdict plus a concise rationale. diff --git a/plugins/pstack/agents/pstack-sonnet-low.md b/plugins/pstack/agents/pstack-sonnet-low.md new file mode 100644 index 00000000..5c202b4f --- /dev/null +++ b/plugins/pstack/agents/pstack-sonnet-low.md @@ -0,0 +1,12 @@ +--- +name: pstack-sonnet-low +description: Native Claude lane for pstack roles configured as claude:sonnet@low. +model: sonnet +effort: low +background: true +disallowedTools: Agent, Task +--- + +# pstack Sonnet lane + +Execute only the task and path scope the parent assigns. Read the grounding artifacts by path. Do not choose another model, spawn another agent, or start a pstack workflow. If the assignment is read-only, do not modify files. Return the requested artifact or verdict plus a concise rationale. diff --git a/plugins/pstack/agents/pstack-sonnet-max.md b/plugins/pstack/agents/pstack-sonnet-max.md new file mode 100644 index 00000000..321a7ae4 --- /dev/null +++ b/plugins/pstack/agents/pstack-sonnet-max.md @@ -0,0 +1,12 @@ +--- +name: pstack-sonnet-max +description: Native Claude lane for pstack roles configured as claude:sonnet@max. +model: sonnet +effort: max +background: true +disallowedTools: Agent, Task +--- + +# pstack Sonnet lane + +Execute only the task and path scope the parent assigns. Read the grounding artifacts by path. Do not choose another model, spawn another agent, or start a pstack workflow. If the assignment is read-only, do not modify files. Return the requested artifact or verdict plus a concise rationale. diff --git a/plugins/pstack/agents/pstack-sonnet-medium.md b/plugins/pstack/agents/pstack-sonnet-medium.md new file mode 100644 index 00000000..f22f051b --- /dev/null +++ b/plugins/pstack/agents/pstack-sonnet-medium.md @@ -0,0 +1,12 @@ +--- +name: pstack-sonnet-medium +description: Native Claude lane for pstack roles configured as claude:sonnet@medium. +model: sonnet +effort: medium +background: true +disallowedTools: Agent, Task +--- + +# pstack Sonnet lane + +Execute only the task and path scope the parent assigns. Read the grounding artifacts by path. Do not choose another model, spawn another agent, or start a pstack workflow. If the assignment is read-only, do not modify files. Return the requested artifact or verdict plus a concise rationale. diff --git a/plugins/pstack/agents/pstack-sonnet-xhigh.md b/plugins/pstack/agents/pstack-sonnet-xhigh.md new file mode 100644 index 00000000..cada3fe2 --- /dev/null +++ b/plugins/pstack/agents/pstack-sonnet-xhigh.md @@ -0,0 +1,12 @@ +--- +name: pstack-sonnet-xhigh +description: Native Claude lane for pstack roles configured as claude:sonnet@xhigh. +model: sonnet +effort: xhigh +background: true +disallowedTools: Agent, Task +--- + +# pstack Sonnet lane + +Execute only the task and path scope the parent assigns. Read the grounding artifacts by path. Do not choose another model, spawn another agent, or start a pstack workflow. If the assignment is read-only, do not modify files. Return the requested artifact or verdict plus a concise rationale. diff --git a/plugins/pstack/skills/arena/SKILL.md b/plugins/pstack/skills/arena/SKILL.md index 29df2f7d..f5f51562 100644 --- a/plugins/pstack/skills/arena/SKILL.md +++ b/plugins/pstack/skills/arena/SKILL.md @@ -15,7 +15,7 @@ Every candidate gets the same prompt, so the prompt is the contract. 1. State the artifact each candidate is producing. 2. Derive the rubric, 3-6 concrete gradeable criteria for what success looks like on *this* task. It is the picker's tool in Phase D. Candidates only see the task. -3. Pick the runners. Use `arena runners` from the current harness's pstack model sheet when present. Otherwise use its [role default](../poteto-mode/references/model-catalog.md#role-defaults), one runner per entry. Spawn more runners when the arena covers multiple design directions. Use the same descriptor N times when the work is generation-bound rather than judgment-sensitive. +3. Pick the runners. Use `arena runners` from the current harness's pstack model sheet when present. Otherwise use its [role default](../poteto-mode/references/model-catalog.md#role-defaults), one runner per entry. Spawn more runners when the arena covers multiple design directions. Use the same descriptor N times when the work is generation-bound rather than judgment-sensitive. Pick each subagent's effort for the job, per the Subagents section of the **poteto-mode** skill. 4. Give each candidate its own output location (a git worktree where possible, otherwise `/tmp/arena-/candidate-/`), per the **separate-before-serializing-shared-state** principle skill. ## Phase B: Fan out diff --git a/plugins/pstack/skills/automate-me/SKILL.md b/plugins/pstack/skills/automate-me/SKILL.md index 186fdfb4..eb2ae680 100644 --- a/plugins/pstack/skills/automate-me/SKILL.md +++ b/plugins/pstack/skills/automate-me/SKILL.md @@ -29,7 +29,7 @@ Update mode changes the rest of the flow: Locate the active workspace's transcripts before fanning out. Claude Code stores them at `~/.claude/projects//*.jsonl`, where `` is the workspace's working directory with `/` → `-`. Use only that path. Don't glob across `~/.claude/projects/`. That crosses workspace boundaries and reads private chats from unrelated projects. -Survey recent agent conversations within that scope for recurring patterns. Run multiple parallel subagents across slices of history (e.g. last 2-4 weeks, split into 3 slices so each has enough material). Each slice mining subagent reads transcripts from the workspace-scoped path the parent provides, looks for the signals below, and returns a short structured list of patterns it saw with evidence pointers. Default signals worth hunting: +Survey recent agent conversations within that scope for recurring patterns. Run multiple parallel subagents across slices of history (e.g. last 2-4 weeks, split into 3 slices so each has enough material). Pick each subagent's effort for the job, per the Subagents section of the **poteto-mode** skill. Each slice mining subagent reads transcripts from the workspace-scoped path the parent provides, looks for the signals below, and returns a short structured list of patterns it saw with evidence pointers. Default signals worth hunting: - Response preferences (length, tone, format, "dumb it down" corrections) - Delegation habits (subagents, models, specialized workflows, parallelism) diff --git a/plugins/pstack/skills/correct/SKILL.md b/plugins/pstack/skills/correct/SKILL.md index 61fdcd92..815d4f1b 100644 --- a/plugins/pstack/skills/correct/SKILL.md +++ b/plugins/pstack/skills/correct/SKILL.md @@ -17,7 +17,7 @@ First, read recent commits, reverts, review comments, agent instruction files, a 1. **Eliminate it with architecture.** Give each piece of state one owner and each task one supported way. Hide internals so the wrong import fails. Replace hand-synced lists with one source of truth. Delete old ways and dead code an agent would copy. 2. **Enforce it with types so the bad state can't be written.** If bad code still compiles, add a lint or CI check whose error names the file, type, or function to use instead. If the pattern is already common, fail only when a change adds more. -3. **Test the behavior.** Fix or delete any test that a relevant defect would not make fail, per the **principle-test-behavior-not-implementation** skill. Keep negative-path tests whose correct result is an absence. +3. **Test the behavior.** Fix or delete any test that still passes when the behavior it checks breaks, or that fails when only the internals change. 4. **Write docs or agent rules last, only for judgment calls.** Nothing fails when an agent skips them. ## Fix and prove diff --git a/plugins/pstack/skills/create-formal-verification-skill/SKILL.md b/plugins/pstack/skills/create-formal-verification-skill/SKILL.md new file mode 100644 index 00000000..b97609fd --- /dev/null +++ b/plugins/pstack/skills/create-formal-verification-skill/SKILL.md @@ -0,0 +1,41 @@ +--- +name: create-formal-verification-skill +description: "Generate a project-local skill that formally verifies the code where a bug costs the most and tests reach the least, in any language, with whichever verification tool fits. Use for /create-formal-verification-skill, \"model-check this race\", \"prove this invariant\", or when concurrency, retries, state machines, parsers, or unsafe code need more than tests." +disable-model-invocation: true +--- + +# Create a formal verification skill + +A test checks the cases that someone thought to write. Formal verification checks every case in a stated scope: every order of steps in a model, every input up to a bound, or every input. This skill finds the part of the repo where that matters most, verifies one property of it, and leaves a project-local skill that lets the next agent check a change against that property. Write that skill for an agent who reads it cold, mid-change, and has never seen your work. The user must be able to trust the result and act on it without learning the verification language. + +## Choose the target from the code + +Answer from the codebase, and ask the user only what you cannot observe. If the user named an area or a bug, start there. + +A good target is a promise the code makes, where a break harms users and tests rarely reach the case that breaks it. Choose the strongest such target in the repo, not the one that is easiest to verify. + +## Let the question choose the tool + +Pick the tool that answers the question at the lowest cost: a model checker, a proof assistant, an SMT-backed verifier, or another tool. Whether events can happen in a bad order usually suits a model checker over a small model of the design, and TLA+ is the default for that in any language. Whether the code itself behaves for every input, as with memory safety, overflow, or a data-structure invariant, usually suits a verifier that runs on the code. If no verifier runs on this language, say so, and test the real code with generated inputs, not a hand model of the function. A property that must hold at every size, where a bound is not enough, can need a proof. Prefer a tool that the repo's developers can install and run, and tell the user why you chose it. + +## What makes it good + +- **True to the code.** Every part of the model or proof points to code that a reviewer can open. What the verification treats as one indivisible step is one indivisible step in the code. Names come from this repo. +- **Able to fail.** A check that you have never seen fail proves nothing. Show each property failing on a broken variant of the code it guards, such as a removed check or a reordered step. +- **Findings stay findings.** When the current code breaks the property, report it with the counterexample in the code's terms. Do not weaken the property or the model until it passes, and do not change product code unless the user asks. +- **Honest about scope.** Say what is verified, under which bounds and assumptions, and what is left out. A pass covers the model or the bound, not the whole system. +- **Readable without the tool.** State the property and the harm in one plain sentence each. Tell a counterexample as a short sequence of steps in the code's words. +- **Cheap to rerun.** One command, a reproducible tool version, and a run short enough that the next agent will use it on every relevant change. +- **Kept in step with the code.** Record which files and functions each check covers, and the commit at which you last confirmed that the verification describes them. When the covered code changes after that commit, the next agent can see that the verification needs another look. + +## Generate the verify skill + +Write a project-local skill with frontmatter that names the area and says when to use it. Without frontmatter, the skill never registers. From the skill alone, the next agent must be able to tell whether a change touches the verified code, run the check, tell a real bug from a modeling error, and update the verification when a change is intended. Put the coverage record and the sync commit in the skill, and tell the next agent to move the sync commit forward only after it confirms that the verification describes the code at the new commit. Use this repo's real commands, paths, and nouns, and leave no placeholders. + +## Prove it before you hand off + +Follow the generated skill once, as the next agent would, on a scratch change that breaks the property in the real code. It must catch the break. Then drop the scratch change and confirm that the check is back where it started: passing, or failing only on the findings you will report. A verify skill that was never run is a draft. If the skill misses the break or a step does not run, fix the skill or the verification, not the scratch change. + +## Hand off + +Tell the user what you verified, which tool you used and why, the bounds, what the check found in plain words, how to run it, and what it does not cover. Suggest a second target only if you found a close one. Point the user at `/maintain-formal-verification-skill` for bringing the verification back in line after changes that did not recheck it. diff --git a/plugins/pstack/skills/create-formal-verification-skill/agents/openai.yaml b/plugins/pstack/skills/create-formal-verification-skill/agents/openai.yaml new file mode 100644 index 00000000..5b1f887a --- /dev/null +++ b/plugins/pstack/skills/create-formal-verification-skill/agents/openai.yaml @@ -0,0 +1,2 @@ +policy: + allow_implicit_invocation: false diff --git a/plugins/pstack/skills/figure-it-out/SKILL.md b/plugins/pstack/skills/figure-it-out/SKILL.md index 4b557f69..b65058c0 100644 --- a/plugins/pstack/skills/figure-it-out/SKILL.md +++ b/plugins/pstack/skills/figure-it-out/SKILL.md @@ -27,7 +27,7 @@ Decompose into atomic, independently-landable units. Sequence riskiest-unknown-f - Build the verification harness before the work, with the baseline captured from the pre-change state, so the check reads as "old value vs new value". - For one-way-door design decisions, run the **architect** skill (it runs **arena**). Skip it for mechanical work whose shape is already concrete. A second arena over a settled design is over-engineering (the **laziness-protocol** principle skill). -- Decide what fans out. Parallelize only across seams, and give each worker its own worktree or branch (the **separate-before-serializing-shared-state** principle skill). Don't over-fan. +- Decide what fans out. Parallelize only across seams, and give each worker its own worktree or branch (the **separate-before-serializing-shared-state** principle skill). Don't over-fan. Pick each subagent's effort for the job, per the Subagents section of the **poteto-mode** skill. - Write the designed phase list down. That list is what the human reviews. Then execute the design. Add its steps to the todolist as concrete items between the Phase C and Phase D entries. Run each under the Phase C loop discipline, and weave the Phase D log through them, a row as each step lands, rather than saving the whole trail for the end. diff --git a/plugins/pstack/skills/how/SKILL.md b/plugins/pstack/skills/how/SKILL.md index 86924007..17a39961 100644 --- a/plugins/pstack/skills/how/SKILL.md +++ b/plugins/pstack/skills/how/SKILL.md @@ -7,7 +7,7 @@ description: "Use for \"how does X work\", code walkthroughs before changing som Explore the codebase to answer "how does X work?" questions. Produce architectural explanations at the level of a senior engineer onboarding onto a subsystem, enough to build a working mental model, not so much that it reads like annotated source code. -**Dispatch contract.** Resolve every configured role through [`provider-dispatch.md`](../poteto-mode/references/provider-dispatch.md). Values are provider-qualified descriptors; the parent chooses native versus external execution. Every lane runs in `read-only` mode. Explorers use the `how explorer` descriptor, and every explainer uses the `how explainer` descriptor. When the sheet lacks either line, use that role's [role default](../poteto-mode/references/model-catalog.md#role-defaults). A native lane uses the parent subagent primitive; an external lane uses the launcher directly. On Codex, resolve remaining Claude tool names via [`codex-tools.md`](../poteto-mode/references/codex-tools.md). +**Dispatch contract.** Resolve every configured role through [`provider-dispatch.md`](../poteto-mode/references/provider-dispatch.md). Values are provider-qualified descriptors; the parent chooses native versus external execution. Every lane runs in `read-only` mode. Explorers use the `how explorer` descriptor, and every explainer uses the `how explainer` descriptor. When the sheet lacks either line, use that role's [role default](../poteto-mode/references/model-catalog.md#role-defaults). A native lane uses the parent subagent primitive; an external lane uses the launcher directly. On Codex, resolve remaining Claude tool names via [`codex-tools.md`](../poteto-mode/references/codex-tools.md). Pick each subagent's effort for the job, per the Subagents section of the **poteto-mode** skill. ## 1. Assess complexity diff --git a/plugins/pstack/skills/in-your-own-words/SKILL.md b/plugins/pstack/skills/in-your-own-words/SKILL.md new file mode 100644 index 00000000..4b2a476a --- /dev/null +++ b/plugins/pstack/skills/in-your-own-words/SKILL.md @@ -0,0 +1,7 @@ +--- +name: in-your-own-words +description: "Restate in your own words what you think the user's goals are and what problem they're trying to solve. Use for /in-your-own-words." +disable-model-invocation: true +--- + +Restate in your own words what you think the user's goals are and what problem they're trying to solve. diff --git a/plugins/pstack/skills/interrogate/SKILL.md b/plugins/pstack/skills/interrogate/SKILL.md index 4d469c8d..4e3d43b5 100644 --- a/plugins/pstack/skills/interrogate/SKILL.md +++ b/plugins/pstack/skills/interrogate/SKILL.md @@ -31,7 +31,7 @@ Route each reviewer's descriptor with `read-only` access and a unique output and If a provider, login, or model is unavailable, record a dropout and continue with the completed reviewers. Never pick the closest model or silently fall back; that destroys the meaning of cross-provider agreement. -Read `references/reviewer-prompt.md` and fill in the same template for every reviewer with the stated intent, the diff or file contents, the rubric from `references/rubric.md`, and the code-quality lens from `references/code-quality-review.md`. +Read `references/reviewer-prompt.md` and fill in the same template for every reviewer with the stated intent, the diff or file contents, the rubric from `references/rubric.md`, and the code-quality lens from `references/code-quality-review.md`. Pick each subagent's effort for the job, per the Subagents section of the **poteto-mode** skill. ## Step 4, Synthesize diff --git a/plugins/pstack/skills/interrogate/references/lead-judgment.md b/plugins/pstack/skills/interrogate/references/lead-judgment.md index 5ce52609..3b41346c 100644 --- a/plugins/pstack/skills/interrogate/references/lead-judgment.md +++ b/plugins/pstack/skills/interrogate/references/lead-judgment.md @@ -1,18 +1,5 @@ # Lead Judgment Framework -You are the lead reviewer. The configured reviewers have produced their findings. Apply pragmatic engineering judgment. Don't aggregate. Filter, contextualize, and decide. - -## Why This Step Matters - -Adversarial reviewers are useful because they're aggressive. But aggression without context produces noise. The reviewers only saw a slice of the codebase and a one-paragraph intent statement. They don't know: - -- What was already tried and rejected -- What constraints exist outside the code (timeline, dependencies, migration plans) -- Which parts of the code are temporary scaffolding vs. permanent architecture -- What the next PR in the stack will address - -You have the full conversation context. Use it. - ## Filtering Principles ### Nitpick Gravity @@ -50,9 +37,3 @@ Don't dismiss findings just because they're uncomfortable. The whole point of ad - You read the finding and think "...yeah, actually" Be especially careful about dismissing security findings and correctness bugs. These deserve more scrutiny even when they come from a single model. - -## Verdict Calibration - -A good verdict is useful, not comprehensive. The user should be able to read the "Act On" section, fix those issues, and ship with confidence. If your "Act On" list has more than 5 items, you're probably not filtering hard enough. - -The "Dismissed" section is not busywork. It's a trust mechanism. Showing the user what you rejected and why lets them override your judgment where they disagree. This is more valuable than hiding the rejected findings. diff --git a/plugins/pstack/skills/ketchup/SKILL.md b/plugins/pstack/skills/ketchup/SKILL.md index 1c63f43f..49e4d94c 100644 --- a/plugins/pstack/skills/ketchup/SKILL.md +++ b/plugins/pstack/skills/ketchup/SKILL.md @@ -1,27 +1,43 @@ --- name: ketchup -description: "Catch the user up on everything in this chat since their last message, in plain words. Give each action item an executive summary with enough context to make the call. Use for /ketchup." +description: "Catch the user up on what changed in this chat since their last message, in plain words and small visuals. List every item waiting on them with what they need to decide it from the reply alone. Use for /ketchup." disable-model-invocation: true --- # Ketchup -Catch the user up on everything in this chat since the last message they wrote before `/ketchup`. Notifications and subagent results are not their messages. Write in the plain voice of [`/bro`](../bro/SKILL.md), and skip its restating step. Assume the user remembers their last message and nothing after it. Call each PR, branch, or agent by a name they would know, not by a label you invented during the work. +Catch the user up on what changed in this chat since the last message they wrote before `/ketchup`, and help them understand each change, not only where it stands. Then lay out every call waiting on them so they can make it from your reply alone. Notifications and subagent results are not their messages. Assume the user remembers their last message and nothing after it. + +Use [`/bro`](../bro/SKILL.md)'s voice, not its task: plain words, short sentences, no jargon, like one person catching up another. Say what changed for the user, not how you did it. Say "the tests pass", not "CI is green". Leave out hashes, file paths, commands, build numbers, and test counts unless the user needs one to act. Call each PR, branch, or agent by a name the user would know, not by a label you invented. If a fact needs context to make sense, give the context or leave the fact out. ## Gather what changed -Keep this turn read-only, except for visuals you make for the reply from what you already have. Do not change code, start an app or a job, or post anything to make them. Collect your own work since that message, reports from subagents, and messages from anyone else in this chat. Check the current state of each PR, CI run, or agent this chat touched. Say which are still running and which you could not check. +Keep this turn read-only. Make visuals only from what you already have: do not change code, start an app or a job, or post anything. Collect your own work since that message, reports from subagents, and messages from anyone else in this chat. Check the current state of each PR, CI run, or agent this chat touched. Check each number you will show against the facts it comes from, and say so when you correct one. + +An action item is a call or step only the user can take, such as a product choice, an approval, an irreversible action, or access you lack. Include any still waiting on the user, even one raised before their last message. Work you can still do yourself is not an action item. Never make up a call to fill the reply. A step the user did not ask for and the work does not need is not an action item, unless it is irreversible and waiting on the user: give it one line and no visual. -An action item is a decision or step only the user can take, such as a product choice, an approval, an irreversible action, or access you lack. Include any still waiting on the user, even one raised before their last message. Work you can still do yourself is not an action item. +One call can cover many items, such as each PR in a set awaiting merge. For each item, collect its link, what it changes for users, the evidence behind your pick, its risk, and your pick. Write "unknown" for a fact the call needs that you do not know. Leave out a link you do not have. ## Write the reply -Open with one sentence that says how many action items there are, or that there are none. If nothing happened and nothing waits on the user, say so and stop. +Spend words on the calls, not on the story. Keep what happened under 100 words, not counting pictures, and shorter when less happened. Leave out work that changed nothing for the user. Give each item about 30 words. The reply grows with the number of items: two items need well under 150 words in all, not counting pictures, and thirty need far more. Never cut items or shrink them to stay under a word count: no count in place of a list, and no link in place of an item. A link can add depth, but the reply alone must be enough to decide. + +Open with one short sentence that names what waits on the user, or says nothing does. The action items give the detail. If nothing happened and nothing waits on the user, say so and stop. + +Then show what happened, pictures first. Embed inline the screenshots and videos the work produced, then any diagram that explains a change. Then sum up in one short bullet per thing the user asked for: what came of it, and whether it is done, still running, failed, or not checked. Write each line the way you would say it out loud: "Search was slow from a missing index", not "Search review: done" or "Slow search: missing index". Name any bug found and fixed on the way. Each line must stay true when read alone. Never mark something done that still waits on the user, and leave out anything the action items say. Under the summary, give at most three short lines that change what the user knows or must do. Then name in one line any work left that you can do yourself. + +Then list the action items. Ask the call the user owns, such as merge, approve, or pick, and give your pick. A step you would take first goes in your pick. So does anything that must happen first, and anything that cannot be undone. Give the number or link of each PR, issue, or agent a call is about, also in a call between options. + +Fit the shape to the call. A call about one item is one bullet: the call, your pick, what it changes for users, the evidence behind your pick, and its risk, with the item named once. When items share a call, write the call and your pick once, then give each item its own bullet, riskiest first: its link and name, what it changes for users, the evidence behind your pick, and its risk. Name an item's pick only where it differs from the call's. Give each item's risk even when it is low, such as "Low risk: only the colors change." Say what waiting costs only when it is more than a delay, such as a deadline or lost data. + +For a call between options, give every option the work weighed, each with what it gets the user and what it costs, and the current state when there is one. Those costs are the call's risk, so add no separate risk line. When the screenshots above show the options, write each option's gain and cost under its screenshot. + +## Visuals -Then say what happened, most important first. Give each result and what it means for the user. Skip steps that changed nothing. Name in one line any work left that you can do yourself. +A visual is a picture: a screenshot, a video, a Mermaid diagram, a chart or diagram that the harness renders, or a generated image. A Markdown table is not a visual, even with before and after columns. -Then give each action item a short executive summary. Cover the decision to make, why it matters now, the options and what each costs, your recommendation, and what happens if the user does not answer. If you do not know a part, say so instead of guessing. Link to the detail when there is one, and write the summary so the user can decide without opening it. +When something is easier to grasp as a picture than as sentences, draw it: how a design or system works, a PR stack or dependency chain with each PR's state, a before and after of a fix, or options side by side. Never draw a status list, a list of action items, or boxes with nothing connecting them. Never give a fact twice, in text or across visuals. -Show each action item as a before and after, side by side. For a choice, put the current state beside each option, or the options alone when there is no current state. Use screenshots or a video when the change is visible. Otherwise use another artifact, such as a table of numbers, a diff, or a diagram. If no visual exists yet, make one. If you cannot, say plainly that none exists. Never invent a number, and never draw or generate a picture of a screen you did not capture. Link only artifacts that exist. +Keep each visual small enough to read at a glance: at most six boxes, with at most eight words in each. Write labels as plain words a person would say, not tags like "Tests: pass". A fact that needs a sentence belongs in the text, or nowhere. -Prefer visuals to dense prose. Use a table, a Mermaid diagram, a chart, or a generated image. +Embed screenshots and videos inline when the change is visible, not only as links. If a visible change has none, mark its summary line "No screenshot yet," and say there what changed on screen. Draw diagrams and charts from facts you have: a Mermaid diagram for how things connect and a rendered chart for numbers. Never invent a number, and never draw or generate a picture of a screen you did not capture. Link only artifacts that exist. diff --git a/plugins/pstack/skills/maintain-formal-verification-skill/SKILL.md b/plugins/pstack/skills/maintain-formal-verification-skill/SKILL.md new file mode 100644 index 00000000..5d2f7f22 --- /dev/null +++ b/plugins/pstack/skills/maintain-formal-verification-skill/SKILL.md @@ -0,0 +1,58 @@ +--- +name: maintain-formal-verification-skill +description: "Bring a project's formal verification back in line with the code after the code changes: find the commits that touched verified code, update the model, harness, or properties to describe the code as it is now, rerun, and say whether each failure is drift in the verification or a bug in the code. Use for /maintain-formal-verification-skill, \"is the model still true to the code\", or after merges and refactors that touched verified code without rechecking it." +disable-model-invocation: true +--- + +# Maintain a formal verification skill + +A formal check proves a property of what it describes. When the code changes and the verification does not, the check can keep passing while it describes code that no longer exists. That pass is the dangerous case, because it looks like a proof. This skill is the upkeep loop for a verify skill that `/create-formal-verification-skill` generated, or for any project-local skill that runs a model checker, prover, or code-level verifier against named code. The project-local skill handles a change that an agent makes while it uses that skill. This skill handles the changes that skipped it: merges, edits made without the skill, and drift that builds up across many commits. + +## Outcomes + +Give each check one outcome, and say which: + +- **in sync**: no change since the last sync alters what the check describes, and the check passes. +- **updated**: the code changed, the verification now describes the new code, and the check passes. +- **bug**: the verification describes the current code, and the code breaks the property. This is a finding about the code. It is not drift. +- **blocked**: the check could not be brought back in line or could not run. Say exactly why. + +A check can be both updated and bug: the update makes the verification describe the code again, and then the property fails on it. + +## Edit scope + +Edit only the verification: models, harnesses, property files, and the verify skill itself. Never edit product code during a run, unless the user asks you to. When a property fails on the current code, do not weaken the property or the model to make it pass. Report the failure. + +## Pass + +1. **Find the checks.** A verify skill has no fixed name, so find it by what it does: a project skill that runs a model checker, a prover, or another verifier against named code. Read each one you find, or the one the user named. For each check, collect the property, the code it covers, the bounds and assumptions, the command that runs it, and the commit where it was last confirmed to describe the code. If no verify skill exists, stop and point at `/create-formal-verification-skill`. If the skill does not record a sync commit, use the last commit that changed the verification files, and say that you did. + +2. **Run it as it is.** Run each check once before you change anything, and record the result. A pass here does not tell you that the code is correct. It tells you that the old description still holds. A failure here can be drift or a bug. Do not decide which until step 5. + +3. **Read every change to covered code.** Diff the covered files from the sync commit to the current commit, and read each commit message. Also look for code that moved: a covered function that is renamed, split, or moved to another file is still covered code. Also look for new code that touches the state that a check guards, such as a new function that writes the same fields. For each change, decide which of these it is: + - **No effect.** The change does not alter anything the verification describes, such as formatting, comments, or code paths that the verification does not cover. Check this against the model or harness, not against the commit message. + - **The verification no longer describes the code.** A step that was one indivisible action is now two, a branch was added or removed, a check moved, a new path reaches the guarded state, or a value now has a different range. The model or harness must change. + - **The promise changed on purpose.** The change alters what the code promises, and evidence in the change says that this is intended: the doc comment, the tests, the changelog, or the commit message. The property must change, and you must cite that evidence. + - **The promise may be broken.** The code changed, and no evidence says that the promise changed. Treat a failure that follows as a bug candidate. + +4. **Update from the code, not from the result.** Change the model or harness so that each changed part again points to code that a reviewer can open, and keep one indivisible step in the code as one step in the verification. Write the update from the code before you rerun, and do not shape it by what makes the check pass. Change a property only for a promise that changed on purpose. Write down the old wording, the new wording, and the evidence. Keep the bounds unless the change makes them wrong. If you must reduce a bound, say so in the report. + +5. **Rerun and judge each result.** + - **Pass:** the check can now pass on a description that has lost the ability to fail. Break the new code the way the property guards against, such as removing a new check or reordering a new step, and confirm that the check catches it. Then drop the break. + - **Fail:** replay the counterexample against the real code, step by step. If a step cannot happen in the code, the verification is wrong. Fix the verification, and say what was wrong. If every step can happen, it is a bug. Name the commit that introduced it, tell the counterexample as a sequence of steps in the code's own words, and state the harm in one sentence. If a cheap test or reproduction against the real code shows the bug, add it to the report as evidence. Do not commit it to the product code. + +6. **Update the verify skill.** Update the code that each check covers, the commands, the bounds, the assumptions, and the sync commit. Set the sync commit to the commit that you checked. Do not move the sync commit for a blocked check, and say so in the report. Then follow the verify skill once as the next agent would, so that you know its steps still run. + +## Report + +Tell the user, for each check: + +- the outcome, and the commits that you read; +- what changed in the verification and why, with a short reason for each change: the code changed shape, or the promise changed on purpose (with the evidence); +- the rerun result, and the break that you used to show the updated check can still fail; +- each bug, in plain words: the counterexample as steps in the code's words, the harm, and the commit that introduced it; +- what the verification still does not cover, including new code that it should cover and does not. + +Say outright which failures were drift in the verification and which were bugs in the code. For a check that passed before you updated it and fails after, say that the old pass was on a stale description. + +Put the verification updates in a change of their own, separate from any product fix. diff --git a/plugins/pstack/skills/maintain-formal-verification-skill/agents/openai.yaml b/plugins/pstack/skills/maintain-formal-verification-skill/agents/openai.yaml new file mode 100644 index 00000000..5b1f887a --- /dev/null +++ b/plugins/pstack/skills/maintain-formal-verification-skill/agents/openai.yaml @@ -0,0 +1,2 @@ +policy: + allow_implicit_invocation: false diff --git a/plugins/pstack/skills/no-comments/SKILL.md b/plugins/pstack/skills/no-comments/SKILL.md index 53b29723..3ef7a949 100644 --- a/plugins/pstack/skills/no-comments/SKILL.md +++ b/plugins/pstack/skills/no-comments/SKILL.md @@ -17,7 +17,7 @@ Use the caller's files or diff. Otherwise use the current diff against the base ## Steps -1. Spawn an `Agent` with `subagent_type: "comment-sicko"`. Pass the scope. Do not restate its rules. +1. Spawn an `Agent` with `subagent_type: "comment-sicko"`. Pass the scope. Pick each subagent's effort for the job, per the Subagents section of the **poteto-mode** skill. Do not restate its rules. 2. Inspect its report and diff. Reject application-code edits, scope escapes, exception-protected deletions, misstated `MUST KILL` reasons, and flags that treat kept intentional code as guilty. Reshape flags on our-code surprises stay actionable. Do not restore those comments. A keep survives only with proof it is about something we cannot change. Audit missed scoped lint and TypeScript suppressions. Correctness or safety suppressions stay actionable `MUST KILL`s. Restore deletions only with exact exceptions and scoped proof. Before accepting thin `IMPORTANT` or `do not remove` kills or keeps, run `/how` or `/why` on their symbol. If a kill is ambiguous, do not restore. If a keep is refuted or still ambiguous, delete it. Revert and rerun one rejected report with the failure named. Reject a second, report it open, and fail `/no-comments`. 3. Fix trivial accepted flags directly by deleting a dead path, dropping a parameter, or using the real API. If any fix needs a shape, run `/architect` once for the accepted set and surrounding code. Stop at the sketch. Architect shapes. Step 4 implements. 4. Implement the smallest root-cause fix in scope. Remove every named workaround. If the root cause is out of scope, land the smallest in-scope fix and report the rest open. The **principle-fix-root-causes** and **principle-redesign-from-first-principles** skills guide intent only. Neither authorizes widening the fence nor fixing instances outside it. Never bolt on symptom guards. diff --git a/plugins/pstack/skills/poteto-help/SKILL.md b/plugins/pstack/skills/poteto-help/SKILL.md index 45a0fc3d..594b8714 100644 --- a/plugins/pstack/skills/poteto-help/SKILL.md +++ b/plugins/pstack/skills/poteto-help/SKILL.md @@ -26,6 +26,7 @@ Check the state that changes the answer, and mention it only when it does: - No `pstack-models.md` in the active harness's config home means `/setup-pstack` hasn't run for this user, so every role uses its default model. Resolve the config home per [`codex-tools.md`](../poteto-mode/references/codex-tools.md#harness-config-homes). - No `verify-*` skill or other app harness in the project means agents have no scripted way to drive the app. Mention `/create-verification-skill` when the question is about proving a change works. +- No formal verification skill in the project means no property is checked past what tests reach. Mention `/create-formal-verification-skill` when the question is about a race, an invariant, or another case tests rarely hit. When one exists and the code it covers changed after the commit it records as last in sync, mention `/maintain-formal-verification-skill` if the question touches that code. When the model rule is missing and it matters, ask whether the user wants to pick a model for each role and a requested effort now. It matters when the user is new, the question is about setup or cost, or the answer depends on which models run. Ask at most once per chat. If the need is also unclear, ask both questions together. Offer two choices: @@ -35,7 +36,7 @@ When the model rule is missing and it matters, ask whether the user wants to pic ## Get set up 1. Install. In Claude Code, run `/plugin marketplace add ericlitman/open-pstack`, then `/plugin install pstack@open-pstack`, then `/reload-plugins`. In Codex, run `codex plugin marketplace add ericlitman/open-pstack --ref main`, then `codex plugin add pstack@open-pstack`, and turn on `multi_agent` per [`codex-tools.md`](../poteto-mode/references/codex-tools.md). -2. Run [`/setup-pstack`](../setup-pstack/SKILL.md). It maps a model to each role, asks one requested effort for each model family it assigns, probes those models, and writes a model sheet. The sheet applies to new sessions. +2. Run [`/setup-pstack`](../setup-pstack/SKILL.md). It maps a model to each role, asks one requested effort for each model family it assigns, probes those models, and writes a model sheet. The sheet applies to new sessions. Then it offers to generate a formal verification skill for the project, unless the project already has one. 3. Start a real task with `/poteto-mode`, a goal, and a check that can pass or fail. In Claude Code, a typed skill carries the plugin prefix, as in `/pstack:poteto-mode`. In Codex, ask for a skill by name, as in "use pstack:poteto-mode". The [README](https://github.com/ericlitman/open-pstack#readme) and [guide page 1](https://github.com/cursor/plugins/blob/main/pstack/docs/guide/01-setup.md) have the details. Offer to word their first prompt with them, per [`references/prompting.md`](references/prompting.md). @@ -63,11 +64,13 @@ The default answer is `/poteto-mode`, which runs most of the others when its ste | The user wants to | Skill | |---|---| | Do any non-trivial task with rigor | [`/poteto-mode`](../poteto-mode/SKILL.md) | +| Check that the agent understood their goal and the problem before it starts | [`/in-your-own-words`](../in-your-own-words/SKILL.md) | +| Aim a task higher before starting: more expert, more outcome-focused, not overspecified | [`/sharpen`](../sharpen/SKILL.md) | | Know how code works now, or where new code should live | [`/how`](../how/SKILL.md) | | Know why code is shaped this way, or where a number came from | [`/why`](../why/SKILL.md) | | Understand a change or subsystem, explained plainly | [`/teach`](../teach/SKILL.md) | | Catch up on their own recent work on a topic | [`/recall`](../recall/SKILL.md) | -| Catch up on everything in this chat since their last message, with enough context to decide each action item | [`/ketchup`](../ketchup/SKILL.md) | +| Catch up on what changed in this chat since their last message, in plain words and small visuals, with every item waiting on them listed and decidable from the reply alone | [`/ketchup`](../ketchup/SKILL.md) | | Know what a small diff could break outside itself | [`/blast-radius`](../blast-radius/SKILL.md) | | Settle types and module shape before code that crosses a function boundary | [`/architect`](../architect/SKILL.md) | | Get several attempts at one brief, merged into the best one | [`/arena`](../arena/SKILL.md) | @@ -81,6 +84,8 @@ The default answer is `/poteto-mode`, which runs most of the others when its ste | Hear the last reply again in plain words | [`/bro`](../bro/SKILL.md) | | Give agents a scripted way to drive the app and prove behavior | [`/create-verification-skill`](../create-verification-skill/SKILL.md) | | Bring a verification skill and its feature map back in line with the app | [`/maintain-verification-skill`](../maintain-verification-skill/SKILL.md) | +| Formally verify a race, invariant, or other property that tests rarely reach | [`/create-formal-verification-skill`](../create-formal-verification-skill/SKILL.md) | +| Bring a formal verification back in line after the code it covers changed, and learn whether a failure is drift or a bug | [`/maintain-formal-verification-skill`](../maintain-formal-verification-skill/SKILL.md) | | Vet a performance number before reporting or acting on it | [`/benchmark-checklist`](../benchmark-checklist/SKILL.md) | | Run a large or cross-cutting change, or one to review after stepping away | [`/figure-it-out`](../figure-it-out/SKILL.md) | | Keep a decision log during a run, and review it afterward | [`/show-me-your-work`](../show-me-your-work/SKILL.md) | diff --git a/plugins/pstack/skills/poteto-help/references/recipes.md b/plugins/pstack/skills/poteto-help/references/recipes.md index 73118e2a..c4f60466 100644 --- a/plugins/pstack/skills/poteto-help/references/recipes.md +++ b/plugins/pstack/skills/poteto-help/references/recipes.md @@ -5,6 +5,8 @@ Swap in the real paths, skills, and done checks. Informal wording works. ## Understand - `/poteto-mode read . restate the underlying issue in your own words, in plain english.` +- `/in-your-own-words ` +- `/sharpen ` - `/poteto-mode investigate why . give me what we know, what data you used, and your best hypotheses. don't change any code yet.` - `use /how to understand . then use /why to find out why it broke recently.` - `/recall my work on from last week, then read .` @@ -32,6 +34,7 @@ Swap in the real paths, skills, and done checks. Informal wording works. ## Review and ship - `/interrogate the whole branch, but skeptically. don't change anything yet. no nitpicks unless it's a real bug or regression.` Read the dismissals too. +- `/create-formal-verification-skill prove holds in , even when interleave or retry.` After merges or refactors in that code: `/maintain-formal-verification-skill`. - `/swarm check every package under against its check script. one worker per package. one report.` - `/poteto-mode open the pr. small ordered commits, evidence in the description.` - `/poteto-mode babysit this pr. get it green.` For status only: `/poteto-mode check on pr . anything outstanding?` @@ -41,6 +44,7 @@ Swap in the real paths, skills, and done checks. Informal wording works. - `/poteto-mode im going to bed. in a fresh worktree off . done means . keep a decision log. don't ask me before committing. /loop until done. if you're truly stuck after a few hours, stop and write up why.` - `/show-me-your-work catch me up on what you did last night.` Read its Attention section first. +- `/ketchup` catches you up on this chat since your last message and lists every call that waits on you. - `/poteto-mode full autopilot on this queue. each item is independent.` - `/poteto-mode autopilot these changes but stack them, don't ship. i'll land the stack.` - `/reflect capture what we learned so the next run doesn't repeat it.` Approve only edits that change a future decision. diff --git a/plugins/pstack/skills/poteto-mode/SKILL.md b/plugins/pstack/skills/poteto-mode/SKILL.md index 08750a51..dcab879a 100644 --- a/plugins/pstack/skills/poteto-mode/SKILL.md +++ b/plugins/pstack/skills/poteto-mode/SKILL.md @@ -5,6 +5,8 @@ description: poteto's agent style for concise, detailed responses, deliberate su # Poteto mode +Bundled skills live at `..//SKILL.md` from this file, so put this file's absolute path in every `poteto-agent` prompt. + ## Platform Adaptation These skills share one tree across Claude Code and Codex. Read [`references/provider-dispatch.md`](references/provider-dispatch.md) whenever a configured role launches. It defines the provider-qualified model descriptors, native/external route table, launcher, isolation, receipts, and dropout policy. Children never choose routes. When a skill names a Claude tool or built-in skill (`run`, `verify`, `plugin-dev:skill-development`), read [`references/codex-tools.md`](references/codex-tools.md) for the Codex equivalent. @@ -43,7 +45,7 @@ Read the leaf skill in full for any principle you apply. Each entry names when i - **Laziness Protocol.** Refactoring, sizing a diff, or tempted to add abstractions, layers, or signal threading. Bias to deletion and the smallest change that solves the problem. - **Foundational Thinking.** Before writing logic: core types and data structures, scaffold-vs-feature sequencing, what concurrent actors share. - **Redesign from First Principles.** Integrating a new requirement into an existing design. Redesign as if it had been foundational from day one. -- **Attack the Premise.** Two or more fixes sharing one premise fail the same gate. Question that premise. For imbalance problems, take a census per actor and test the asymmetry hypothesis. +- **Attack the Premise.** Two or more fixes that share one premise have failed the same gate. Write the premise down and test it directly before you write another fix that assumes it. - **Subtract Before You Add.** Sequencing an addition, refactor, or rewrite. Remove dead weight first, then build on the simpler base. - **Minimize Reader Load.** Reviewing or shaping code that's hard to trace. Count layers and hidden state, collapse one-caller wrappers, shrink mutable scope. - **Outcome-Oriented Execution.** Planned rewrites and migrations with explicit phase boundaries. Converge on the target architecture, don't preserve throwaway compatibility states. @@ -63,9 +65,9 @@ Read the leaf skill in full for any principle you apply. Each entry names when i **Verification** - **Prove It Works.** After a task, before declaring done. Verify against the real artifact, not a proxy or "it compiles". -- **Fix Root Causes.** Debugging. Trace each symptom to its root cause, reproduce first, ask why until you reach it. +- **Fix Root Causes.** Debugging. Trace each symptom back to the first place the system went wrong and fix it there. A guard is a fix only when the state it handles is legitimate. - **Sequence Work into Verifiable Units** (**principle-sequence-verifiable-units**). Multi-step work (sweeps, migrations, runs of similar edits) and how you stack commits and PRs. Break work into small units that each end in a check, verify each before the next, and order delivery so the sequence proves itself. -- **Test Behavior, Not Implementation.** Writing, changing, or keeping a test. Assert observable behavior or a contract and check that a relevant defect fails. Preserve useful negative-path and relational contract tests. +- **Test Behavior, Not Implementation.** Writing, changing, or keeping a test. Assert what the code's users can observe, so the test fails when that behavior breaks and keeps passing when only the internals change. - **Explain the Number.** Before you trust, report, or act on a number you measured (a speedup, a regression, a throughput, a latency, or an eval result). Find what limits it, and rule out that it measured something other than the work you think. **Delegation** @@ -89,10 +91,12 @@ Read the leaf skill in full for any principle you apply. Each entry names when i ## Subagents -For `inherit-parent`, `auto`, or an unconfigured native ad-hoc helper, prefer `poteto-agent`. The chat that receives `/poteto-mode` runs the playbook. Never spawn a `poteto-agent` to run the playbook itself. `/poteto-mode` and `poteto-agent` route through the same wrapper. A provider-qualified role instead follows provider dispatch: Claude's shipped frontier agent definitions select the model alias and requested effort, Codex passes both to `spawn_agent`, and external providers run through the deterministic launcher. Routed workflow skills set the task and access mode. Do not override their choices. +For `inherit-parent`, `auto`, or an unconfigured native ad-hoc helper, prefer `poteto-agent`. The chat that receives `/poteto-mode` runs the playbook. Never spawn a `poteto-agent` to run the playbook itself. A provider-qualified role instead follows provider dispatch: Claude's shipped frontier agent definitions select the model alias and requested effort, Codex passes both to `spawn_agent`, and external providers run through the deterministic launcher. Routed workflow skills set the task and access mode. Do not override their choices. **Defaults for every delegation.** Start independent lanes together, use file pointers rather than inlined dumps, preserve only the tools or MCPs the task needs, and assign every writer a worktree or unique output directory. `/setup-pstack` configures the descriptor per role. A role without a sheet line uses its [role default](references/model-catalog.md#role-defaults); the panel roles default to one model from each of three providers. Each code playbook's configured model comes from its line (`feature, refactoring`, `bug-fix`, `perf-issue`, or `hillclimb`), and the hardest changes read `hardest tasks`. Prose and judgment read `judgment and prose`. `inherit-parent` and `auto` use the parent model natively and reduce provider diversity when used in a panel. +**Match the model to the job.** Before each spawn, start from the role's line or default and pick the effort the job needs. That descriptor's effort is the starting level. Raise it one step, up to max, for the hardest or subtlest work, and leave a configured `ultra` as it is. Lower it one step for judgment and two or more steps for routine work. Verification is the exception and stays on the `swarm workers` descriptor. Change only the effort level, and keep the family the role names, so a diverse panel stays diverse. If the family's row in the [model matrix](references/model-catalog.md#model-matrix) does not list that effort, use its nearest listed effort. A role on `auto` or `inherit-parent` stays on the parent model, and an unregistered model keeps its configured effort. If the route rejects the adjusted effort, do not try another adjusted effort. Use the role's line or default and say so. Any other failure stays a dropout. + You own every subagent's work. Review the diff and write your own summary, don't pass through what it said. A second opinion is the same prompt against a different model. Agreement is high-signal. **Fresh subagents by default.** Give new work to a fresh subagent with consolidated scope, meaning the original brief, every later directive, and the prior agent's report and branch. This holds for a fix round, a follow-up, a retry, and the next queue item. Resume, message, or queue a follow-up on an existing subagent only when the new work strictly needs state that lives in that agent and is costly to move: its local checkout, its uncommitted changes, or a process it still runs, such as a dev server, a simulator, or a babysit watcher. A stop or hold order to a running agent is not reuse. A role such as a PR owner outlives its agent. Once that agent returns, a fresh agent takes the role's next round. Interrupt-chained resumes silently drop directives, so fire a fresh subagent with consolidated scope rather than trusting a "done" summary. diff --git a/plugins/pstack/skills/poteto-mode/playbooks/autonomous-run.md b/plugins/pstack/skills/poteto-mode/playbooks/autonomous-run.md index 196f525d..f45ac157 100644 --- a/plugins/pstack/skills/poteto-mode/playbooks/autonomous-run.md +++ b/plugins/pstack/skills/poteto-mode/playbooks/autonomous-run.md @@ -3,7 +3,7 @@ **You own the exit condition. Define done, then drive to it without stopping.** 1. State the exit condition as a checkable predicate before the first iteration (tests green, repro fixed, all N PRs merged, pixel-diff zero). -2. Pick the wake mechanism using Claude Code's `loop` skill (built-in). An event to watch (CI, a merge, a ref advancing) gets a watcher subagent that wakes you on the event, with a long time-based heartbeat as fallback. No event gets a fixed-interval heartbeat sized to when the result is worth re-checking. +2. Pick the wake mechanism using Claude Code's `loop` skill (built-in). An event to watch (CI, a merge, a ref advancing) gets a watcher subagent that wakes you on the event, with a long time-based heartbeat as fallback. No event gets a fixed-interval heartbeat sized to when the result is worth re-checking. Pick each subagent's effort for the job, per the Subagents section. 3. Each iteration makes the smallest change the evidence justifies, verifies it against the predicate, commits if it advanced, discards changes that didn't help. Belt-and-suspenders that "might help" gets reverted, not left to ride. Sequence the work via the **sequence-verifiable-units** principle skill, verifying each unit before the next instead of batching checks at the end. 4. Mid-run discoveries are yours. Address broken skills, related bugs, flaky verifiers, review noise, tooling failures, orphaned follow-ups, and fixable drift yourself via poteto-mode. Put out-of-band fixes in their own PR. Do not park reversible work for the human or use `AskUserQuestion`. Surface only irreversible actions, genuine product or preference calls no experiment can settle, or a real dead end. Keep the predicate as the main drive, and return to it after each side fix. diff --git a/plugins/pstack/skills/poteto-mode/playbooks/autopilot-full.md b/plugins/pstack/skills/poteto-mode/playbooks/autopilot-full.md index 1969bb97..3012d98d 100644 --- a/plugins/pstack/skills/poteto-mode/playbooks/autopilot-full.md +++ b/plugins/pstack/skills/poteto-mode/playbooks/autopilot-full.md @@ -1,13 +1,23 @@ ### Autopilot-full -**You own the verdicts, never the PRs. One owner runs each PR from build to merge, and nothing merges without your clean swarm verdict.** For "autopilot this queue", "full autopilot", and one-owner-per-PR programs. Orchestrate runs a standing program whose coordinator lands verified work itself and whose workers never merge. Here each PR's owner carries the whole lifecycle through the merge, and the root keeps only verification, countersigns, and audits. +**You own the verdicts, never the PRs. One owner carries each PR from build to merge, and nothing merges until your swarm, which never wrote the code, has proven the code that lands.** -1. **Mark the operator's items and honor state-then-wait.** Items the operator names stay with the operator. The operator reviews and clicks, and no owner merges one. When the operator asks for the protocol or the plan to be stated, deliver the statement and stop. Execution starts only on the operator's explicit go. -2. **Spawn one owner per PR with the full lifecycle and an early trail.** Resolve the forge once for the program. GitHub CLI (`gh`) is the default. If `command -v origin` succeeds and Origin can resolve the repository, use `origin pr ...` for PR create, edit, view, and watch operations; owner merges still follow Shipping step 5's expected-head rule. Otherwise stay on `gh` and record the fallback. Record the forge-reported base repository as `` and pass `--repo "$base_repo"` to every GitHub PR command. Resolve and validate ``, ``, and `` separately by Shipping step 1; never assume any of them is named `origin` or that the base and head repositories are the same. Never require Graphite (`gt`). One background subagent per PR, in its own worktree, owns build, the first push, an early PR opened per Opening a PR's readiness rule, self-proof on the real artifact (the **prove-it-works** principle skill), skeptical review-bot triage per `../references/bugbot-triage.md`, a slop-strip (the **deslop** skill, `/deslop`), and `/no-comments` (the **no-comments** skill). Before the code-ready report and babysit, whether or not trunk has drifted, a self-contained PR or private-stack root rebases onto current trunk fetched through ``. A private-stack child fetches its parent's exact tip through `` and rebases onto that tip instead. Record the selected exact commit as ``. Before either rebase or any other rewrite, the owner applies Shipping step 4's disarm-and-confirm rule to its PR and any private-stack descendants, stopping if the forge cannot confirm every request off. Record the branch's published SHA as `` with `git ls-remote -- "$head_url" "refs/heads/$branch"` and require it to equal the local pre-rebase tip. Record the branch's current patch base as ``, require `git merge-base --is-ancestor "$current_base_sha" "refs/heads/$branch"` to pass, and use `git rebase --onto "$target_tip" "$current_base_sha" -- "$branch"` when the target changed. After the rebase, publish with `git push --force-with-lease="refs/heads/$branch:$captured_sha" -- "$head_url" "HEAD:refs/heads/$branch"`. When the shipped code is final, after the slop-strip and `/no-comments`, it reports the code-ready head SHA, and the SHA of each later push that changes the patch. Self-proof, CI, and the babysit loop (`playbooks/babysit.md`) then run in parallel with the swarm. The owner reports merge-ready with the head SHA when they finish, and owns the merge itself. Before a push that starts a round, run the pre-review checks that the repo's instruction files (AGENTS.md, CLAUDE.md) name for the touched paths, on the committed head. A hook pass is not proof. Within about 15 minutes, every owner starts a `decisions.tsv` trail, pushes its first branch snapshot, and opens the PR before self-proof so the URL, decisions, and checks form a durable trail. After that, the owner pushes its branch again after every verifiable unit (hooks on, a WIP commit is fine). Follow Opening a PR's readiness rule. Repository instructions can keep the early PR draft until its required evidence is recorded. Keep `decisions.tsv` uncommitted and return it with the reports. As soon as a subagent starts, the owner adds its retained handle (Agent task, background Bash task, or Codex agent or session ID) and its state to a `children.tsv` kept the same way. In fix rounds, the owner keeps that merge base. It rebases again only at merge prep (step 5), on a `git merge-tree` conflict with trunk, or on a CI failure that comes from a change on trunk, each time through the disarm, captured-SHA, and lease steps above. Never force-push a shared branch. The merge is the one step an owner may not take alone. Step 4 gates it. -3. **Run owners in true parallel and never stack.** Many owners at once when PRs are self-contained: one writer per branch, disjoint files, cross-PR drift absorbed by rebase. Only genuinely overlapping work serializes. Self-contained PRs branch straight off main, and sequenced work is merge-then-branch. One exception: an owner that must split a genuinely dependent change may hold a short private stack. That stack follows Autopilot-stack's same-repository and fork base rules, and every child records its parent's exact tip as its patch base. -4. **Swarm-verify every round before its merge.** A round starts at the owner's code-ready head SHA and at each later push that changes the PR's patch. At that SHA, fan out parallel independent verifiers per the **swarm** skill and aggregate to one verdict. The merge needs a clean verdict from the round whose patch matches the merge-ready head. Audit the receipts in the merge-ready report before the verdict. The lanes: re-run the gates at that SHA. Prove the load-bearing behavior live on the real surface the change touches (the `run` skill for CLIs and TUIs, `verify` for UIs, or a named driver where neither fits). Audit the diff, distrusting the PR body. Run the audit as two or more review lanes with the full brief. Give each lane one main focus, such as consumer parity with trunk, lifetimes and races, or data and config safety. **Regression lane against trunk.** Run the same load-bearing scenario on current trunk. If trunk does not have the feature, record that fact and gate the behavior the diff adds plus the end state the user waits for instead of pretending trunk can produce it. The live lane is the floor, and a verdict without it is not clean. No merge without the root's clean verdict. When the lanes return, send every proven finding against the PR to the owner in one fix-forward. A defect that a lane filed as a note is a finding. For each behavior finding, ask for a red test that covers every site with the same defect. Where no test can show the defect, ask for a repro receipt instead. Add that defect to the next round's review brief. The new head gets a fresh swarm and a fresh verdict, except for lane results that stay valid under the patch-id rule in `playbooks/shipping.md`. -5. **On a clean verdict the owner merges, and a fresh owner takes the next item.** The owner merges only from a head freshly rebased onto trunk. Merge prep never comes before a round's lanes start, and it ends with a rebase onto current trunk right before the merge. After the merge-prep rebase, the owner reports the new head SHA. CI must pass on that head before the merge, and the patch-id rule decides whether the round's verdict still holds. A private-stack child reaches that state through Shipping step 4 after its parent merges. The merge-ready report records the verdict SHA and the current landing SHA. If trunk moves again before the merge, the patch ID rule in `playbooks/shipping.md` governs re-verification. A changed patch needs a new verdict. An unchanged patch keeps the verdict, but Shipping step 3 records the new `` after current CI and mergeability pass. The owner lands its own PR only through Shipping step 5's server-enforced expected-head flow bound to ``; never use a PR-number-only merge or unguarded auto-merge. The owner then returns. A fresh owner picks up the next self-contained item from the queue, per poteto-mode's Subagents section. The operator's full-autonomy grant plus the root's clean verdict is the merge authorization that babysitting alone never has. Operator-named items stop at merge-ready and wait for the operator's click. -6. **Run the root layer.** A genuinely new raise of a pinned gate or budget value (a limit CI only lets tighten) needs your fresh countersign, granted only after verifier proof. If the operator's grant or standing orders cover approvals, that countersign is the approval. The owner records it in the form that the tool's approval contract allows, with a pointer to the root's countersign. A lane checks the record against that countersign. The root never gives or bypasses an approval that the forge enforces. Absorbing values that already landed on main is drift, not a raise. Run an audit tick over all owners every hour. The tick is an observation cadence, never a lease, deadline, or cancellation threshold. On the operator's go, arm `/loop 1h` with a prompt that runs this tick. Never leave the cadence to memory or lossy completion notifications. At each tick, re-read this playbook from disk (`skills/poteto-mode/playbooks/autopilot-full.md` under the installed plugin) and audit the operation against it. Fix drift during that tick and treat it as urgent. Probe each owner through its retained handle with a generic liveness or status check, and collect the decision trails. Count commits, pushes, PR or check deltas, store reports, and a live retained process as evidence. Elapsed time or the absence of a new side effect alone never proves a lane is stuck; implementation runs can remain healthy for 90 minutes or much longer. Stand a lane down only on affirmative failure evidence such as a dead process, failed handle, explicit error, or a caller-supplied external deadline. The tick judges an owner by the pushed branch and the decision trail that step 2 requires, and it replaces an owner whose agent cannot start a turn. Each tick also runs this stuck test over the parent's background task list and every owner's `children.tsv`. Cancel a stuck lane through its retained handle. Whether or not the cancel succeeds, the root has the owner record it as stuck in `children.tsv` and, if its work is still needed, replace it. Each replacement that fails gets the same steps. The root takes both steps when the owner cannot. A stall never proves or drops the work. When merges batch, run a retro pass and a post-merge bot-comment sweep. End the tick only when no delegated work is left, even after the last merge. -7. **Stand down instantly on the operator's stop.** The operator's hold or stand-down reaches every owner as a zero-writes order immediately. Owners hold their briefs until the operator releases them. +A verdict covers one patch and green CI covers one head. Step 4 runs a full swarm once, and again only for a major change. The patch-id rule in `playbooks/shipping.md` can carry a lane's result, and every new head needs its own green CI. -**Reply:** the queue with each PR's owner, state, and head SHA. Each verdict and the swarm that produced it. What merged and what each fresh owner took next. Countersigns granted and why. Open operator gates. Where the collected decision trails live. +1. **The operator's gates come first.** The grant covers only the queue's items, so items the operator names as theirs and PRs outside the queue stop at merge-ready. A request to state the plan is not a go. +2. **Spawn one owner per PR, and keep its work where a replacement can find it.** Resolve the forge once for the program. GitHub CLI (`gh`) is the default. If `command -v origin` succeeds and Origin can resolve the repository, use `origin pr ...` for PR create, edit, view, and watch operations; owner merges still follow Shipping step 5's expected-head rule. Otherwise stay on `gh` and record the fallback. Record the forge-reported base repository as `` and pass `--repo "$base_repo"` to every GitHub PR command. Resolve and validate ``, ``, and `` separately by Shipping step 1; never assume any of them is named `origin` or that the base and head repositories are the same. Never require Graphite (`gt`). Each owner is a background subagent in its own worktree that builds, opens its PR per `playbooks/opening-a-pr.md`, proves the change on the real artifact (the **prove-it-works** principle skill), babysits to green per `playbooks/babysit.md`, and merges once steps 4 and 5 clear it. + + Open the PR early so CI and bots run alongside the work. The code is ready only after the slop-strip (the **deslop** skill, `/deslop`) and the `/no-comments` pass (the **no-comments** skill), so the full swarm runs on finished code. Then rebase onto current trunk fetched through `` and report the code-ready head before self-proof, so verification runs on the base the patch lands on. Report every later head, and merge-ready once self-proof, CI, and babysit are done. Fix rounds stay on that base unless trunk conflicts or breaks CI, because each rebase restarts CI. Run the pre-review checks that the repo's instruction files (AGENTS.md, CLAUDE.md) name for the touched paths on the committed head before the code-ready push and before a later push that needs a verifier. A hook pass is not proof. + + Assume the owner can die. Soon after starting, push and start an uncommitted `decisions.tsv` trail per the **show-me-your-work** skill, return it with each report, then push after every unit worth keeping. Whoever spawns a subagent records its retained handle (Agent task, background Bash task, or Codex agent or session ID) and its state in a `children.tsv` kept the same way, so the root can probe it if the parent dies. Force-push only your own branch, and on any push you did not make, stop and report to the root. Before any rebase or other rewrite, apply Shipping step 4's disarm-and-confirm rule to your PR, stopping if the forge cannot confirm every request off. Fetch current trunk through ``. Record the selected exact commit as ``. Record the branch's published SHA as `` with `git ls-remote -- "$head_url" "refs/heads/$branch"` and require it to equal the local pre-rebase tip. Record the branch's current patch base as ``, require `git merge-base --is-ancestor "$current_base_sha" "refs/heads/$branch"` to pass, and use `git rebase --onto "$target_tip" "$current_base_sha" -- "$branch"` when the target changed. Publish with `git push --force-with-lease="refs/heads/$branch:$captured_sha" -- "$head_url" "HEAD:refs/heads/$branch"`, pinned to the SHA you last pushed or took over. +3. **Run owners in parallel off trunk, never stacked,** so each PR lands on its own. +4. **Swarm-verify once.** A full swarm runs once, at the code-ready head. Another full swarm runs only for new behavior, a redesign, or a large new diff. A rebase, a merge-conflict resolution, or a minor fix gets a single verifier at the new head. Fan out independent verifiers at the code-ready SHA per the **swarm** skill. They re-run the gates, prove the load-bearing behavior live on the real surface through its driver skill (`run` for CLIs and TUIs, `verify` for UIs) or a named driver, and audit the diff. The verdict is not clean without the live lane, or before you audit the merge-ready report's receipts, because no lane trusts a claim it has not checked. + + Send every proven finding, whatever a lane labeled it, to the owner in one fix-forward, with a red test across every site with the same defect. A defect trunk shows without the PR is recorded against trunk and does not gate the PR, but one the PR copies or newly reaches is the PR's own. Never push a fix yourself, because each branch has one writer. Replace a stuck owner instead. +5. **The owner merges only what you verified, tested on the trunk it lands on.** Green CI alone never authorizes a merge. The owner merges only from a head freshly rebased onto trunk. Merge prep never comes before a round's lanes start, and it ends with a rebase onto current trunk right before the merge. After the merge-prep rebase, the owner reports the new head SHA. CI must pass on that head before the merge, and the patch-id rule decides whether the round's verdict still holds. The merge-ready report records the verdict SHA and the current landing SHA. If trunk moves again before the merge, the patch ID rule in `playbooks/shipping.md` governs re-verification. A changed patch needs a new verdict. An unchanged patch keeps the verdict, but Shipping step 3 records the new `` after current CI and mergeability pass. The owner lands its own PR only through Shipping step 5's server-enforced expected-head flow bound to ``; never use a PR-number-only merge or unguarded auto-merge. The owner then returns. A fresh owner picks up the next self-contained item from the queue, per poteto-mode's Subagents section. The operator's full-autonomy grant plus the root's clean verdict is the merge authorization that babysitting alone never has. +6. **Run the root layer.** Anything that loosens a gate, such as a raised budget or an allowlist entry, needs a verifier's proof and then your fresh countersign, because the owner is the one who wants green. Under a grant that covers approvals, that countersign is the approval, and the owner's record of it points at your countersign for a lane to check. A disabled or soft-failed check always comes out, because it stops guarding every case. Never give or bypass an approval the forge enforces, because no countersign covers it. + + On the operator's go, arm `/loop 1h` to audit until no delegated work is left, because long runs drift and agents stall silently. The tick is an observation cadence, never a lease, deadline, or cancellation threshold. Each tick re-reads this playbook from disk (`skills/poteto-mode/playbooks/autopilot-full.md` under the installed plugin) and fixes drift, such as an unreported head, an unrecorded agent, or an owner that has neither pushed nor written a trail row, by having its owner record it, because drift is a gap in the record, not a stall. It probes every agent through its retained handle, down to the leaf lanes in each `children.tsv`, and judges each by its side effects, because status text and summaries can hide a stuck child. An agent is stuck only on affirmative failure evidence: it errors, its process is dead, its handle fails, it cannot start turns, or a caller-supplied external deadline passed. Elapsed time or the absence of a new side effect alone never proves it stuck; implementation runs can stay healthy for 90 minutes or much longer. One waiting on you or the operator is not stuck, so answer it. Cancel a stuck agent through its retained handle and replace it at once, without waiting for its stop to land. A tick that changed something for the operator ends by catching them up per the **ketchup** skill, and one that changed nothing stays quiet. +7. **Stand down at once on the operator's stop.** A hold reaches every owner as a zero-writes order. Each reports where it stands and keeps its brief until the operator releases it. + +**Reply:** catch the operator up per the **ketchup** skill, covering each PR's state and verdict, what merged, countersigns and why, and open operator gates, with links to each PR and the trails. diff --git a/plugins/pstack/skills/poteto-mode/playbooks/autopilot-stack.md b/plugins/pstack/skills/poteto-mode/playbooks/autopilot-stack.md index e2a36e5e..6c2cab23 100644 --- a/plugins/pstack/skills/poteto-mode/playbooks/autopilot-stack.md +++ b/plugins/pstack/skills/poteto-mode/playbooks/autopilot-stack.md @@ -1,16 +1,14 @@ ### Autopilot-stack -**You own the stack, never the landing. Build and verify the queue with full autonomy, then hand the operator one linear stack to review and land.** The sibling of **Autopilot-full**. +**You own the stack, never the landing. Hand the operator one linear stack of verified PRs to review and land.** **Autopilot-full** holds except for how verified work leaves. Each tick re-reads both playbooks from disk (`skills/poteto-mode/playbooks/autopilot-stack.md` and `autopilot-full.md` under the installed plugin), and this one wins where they differ. -1. **Run the owner loop unchanged.** Resolve the forge once for the program. GitHub CLI (`gh`) is the default. If `command -v origin` succeeds and Origin can resolve the repository, use `origin pr ...` for PR create, edit, view, and watch operations; this playbook never merges. Otherwise stay on `gh` and record the fallback. Record the forge-reported base repository as `` and pass `--repo "$base_repo"` to every `gh pr` command. Resolve and validate ``, ``, and `` separately by Shipping step 1; never assume any of them is named `origin` or that the base and head repositories are the same. When the head repository is a fork, validate its identity and record its owner and repository name as `` and ``. Never require Graphite (`gt`). One background subagent per PR, in its own worktree, owns its change end to end: build, first push, an early PR opened per Opening a PR's readiness rule, self-proof (gates, CI, receipts), skeptical review-bot triage per `../references/bugbot-triage.md`, a slop-strip (the **deslop** skill, `/deslop`), `/no-comments` (the **no-comments** skill), and babysit to green per `playbooks/babysit.md`. Owners parallelize when the work is self-contained. Within about 15 minutes, every owner starts a `decisions.tsv` trail per the **show-me-your-work** skill, pushes its first branch snapshot, and opens the PR before self-proof. After that, the owner pushes its branch again after every verifiable unit (hooks on, a WIP commit is fine). Repository instructions can keep the early PR draft until its required evidence is recorded. Keep the trail uncommitted and return it in the report. Owners also keep the `children.tsv` of Autopilot-full step 2. -2. **Audit on a real loop.** The root runs an audit tick every hour. The tick is an observation cadence, never a lease, deadline, or cancellation threshold. On the operator's go, the root arms `/loop 1h` with a prompt that runs this tick, per Autopilot-full step 6. Never leave the cadence to memory or lossy completion notifications. At each tick, re-read this playbook from disk (`skills/poteto-mode/playbooks/autopilot-stack.md` under the installed plugin) and audit the operation against it. Fix drift during that tick and treat it as urgent. Probe each owner through its retained handle with a generic liveness or status check. Count commits, pushes, PR or check deltas, store reports, and a live retained process as evidence. Elapsed time or the absence of a new side effect alone never proves a lane is stuck; implementation runs can remain healthy for 90 minutes or much longer. Stand a lane down only on affirmative failure evidence such as a dead process, failed handle, explicit error, or a caller-supplied external deadline. Cancel through the retained handle before dispatching a replacement. Probe all subagents and end the tick per Autopilot-full step 6. -3. **Hold the operator gates.** State-then-wait, so a request to state the plan is not a go. On the operator's stop, every owner takes an immediate zero-writes hold. -4. **Verify each round.** The owner reports its code-ready head SHA once the shipped code is final, and STACK-READY with the exact head SHA when its loop is green. The root verifies each round per Autopilot-full step 4, with STACK-READY in place of merge-ready. Nothing enters the stack unverified. -5. **Append on a clean verdict, never ship.** No owner merges, arms auto-merge, or closes. A clean verdict appends the PR to one frozen bottom-to-top list, in verified order or an order the operator specified. For same-repository heads, the forge's base-branch chain also records that order. For fork heads, every PR targets trunk, so never infer stack order from their equal base branches. -6. **Single writer on topology, parallel writers on builds.** Owners push only their own branches and report the tip, current patch base, and intended parent. The root is the only topology writer. Before it rebases, force-pushes, or retargets an existing PR, the root applies Shipping step 4's disarm-and-confirm rule to that PR and every descendant, stopping if the forge cannot confirm every request off. Fetch trunk through ``. Fetch another stack branch directly through ``, because `` may have the base repository as its fetch URL and the fork as its push URL. Validate the chosen repository identity through the active forge, and record the fetched target as ``. Before rebasing, record the child branch's published SHA as `` with `git ls-remote -- "$head_url" "refs/heads/$branch"` and require it to equal the local pre-rebase tip. Require `git merge-base --is-ancestor "$current_base_sha" "refs/heads/$branch"` to pass. When the parent changed, move only the child's commits with `git rebase --onto "$parent_tip" "$current_base_sha" -- "$branch"`, then record `` as the child's new patch base. Publish with `git push --force-with-lease="refs/heads/$branch:$captured_sha" -- "$head_url" "HEAD:refs/heads/$branch"`. When the head and base repositories are the same, create or retarget the child PR with the parent branch as its base. Use `origin pr create --status open --base "$parent_branch"`, `gh pr create --base "$parent_branch" --repo "$base_repo"`, `origin pr edit "$pr" --base "$parent_branch"`, or `gh pr edit "$pr" --base "$parent_branch" --repo "$base_repo"` according to the resolved forge and operation. When the head repository is a fork, keep the local child branch rebased onto its parent's exact tip but create or retarget every PR against `` in the base repository. With GitHub, capture the approved PR title and body as `` and `<body>`, then create it with `gh api --method POST "repos/$base_repo/pulls" -f "title=$title" -f "body=$body" -f "head=$fork_owner:$branch" -f "head_repo=$head_name" -f "base=$trunk" --jq .html_url`; add `-F draft=true` only when Opening a PR's readiness rule requires a draft. Otherwise use the resolved Origin equivalent. Retarget it with `gh pr edit "$pr" --base "$trunk" --repo "$base_repo"` or the resolved Origin equivalent. A forge cannot use a fork-only parent branch as a PR base. Never submit or register the chain through `gt`. -7. **Absorb drift at the root, then re-verify what moved.** The root fetches current trunk through `<base-remote>` and rebases the chain from bottom to top with step 6's explicit old-base and new-parent flow. When a rebase surfaces conflicts in an owner's files, that owner fixes its own slice and the root pushes the result using step 6's captured-SHA lease flow. A rebase rewrites every SHA above it and voids verdicts at the old SHAs. Apply the patch-id rule in `playbooks/shipping.md` step 3 at each verdict SHA, using each PR's recorded patch-base-to-head diff. Anything no longer valid goes back through step 4 before delivery. Re-run mergeability and CI after every rewritten push even when the patch ID is unchanged. The countersign rule is unchanged from Autopilot-full. A genuinely new pin raises a stop for the root's fresh countersign. Absorbing drift of landed values is not a raise. -8. **Deliver the chain.** The deliverable is one frozen bottom-to-top list of verified PRs, reviewable in the resolved forge, every link carrying its verifier verdict in the PR body or a comment. Same-repository heads use a base-branch chain. Fork heads retain local parent ancestry while every PR targets trunk. The operator sends only the current bottom PR through Shipping, one at a time. A merge-when-ready request applies only to that bottom after Shipping has disarmed and confirmed every descendant; never offer or arm merge-when-ready for the chain as a whole or for a descendant. +1. **Run Autopilot-full's owners, minus the landing.** Resolve the forge, `<base-repo>`, `<head-remote>`, `<head-url>`, and `<base-remote>` as Autopilot-full step 2 does; this playbook never merges. When the head repository is a fork, validate its identity and record its owner and repository name as `<fork-owner>` and `<head-name>`. Owners report STACK-READY with the exact head SHA in place of merge-ready. They never rebase onto another branch, trunk included, and never retarget, merge, close a PR, or arm auto-merge, because the root owns the chain's topology and the operator owns the landing. When an item depends on another, its owner branches from that item's branch, not from trunk. Nothing enters the stack before a clean verdict from Autopilot-full step 4. +2. **Give the topology one writer.** Owners push only their own branches and report each tip, its current patch base, and its intended parent. Only you rebase or retarget a link, because two writers on one chain undo each other. Append each clean PR to one frozen bottom-to-top list in dependency order, or an order the operator specified. For same-repository heads, the forge's base-branch chain also records that order. For fork heads, every PR targets trunk, so never infer stack order from their equal base branches. Never require Graphite (`gt`). An owner takes your push as its new base and lease, and on a conflict resolves its own slice for you to push. + + Before you rebase, force-push, or retarget an existing PR, apply Shipping step 4's disarm-and-confirm rule to that PR and every descendant, stopping if the forge cannot confirm every request off. Fetch trunk through `<base-remote>`. Fetch another stack branch directly through `<head-url>`, because `<head-remote>` may have the base repository as its fetch URL and the fork as its push URL. Validate the chosen repository identity through the active forge, and record the fetched target as `<parent-tip>`. Before rebasing, record the child branch's published SHA as `<captured-sha>` with `git ls-remote -- "$head_url" "refs/heads/$branch"` and require it to equal the local pre-rebase tip. Require `git merge-base --is-ancestor "$current_base_sha" "refs/heads/$branch"` to pass. When the parent changed, move only the child's commits with `git rebase --onto "$parent_tip" "$current_base_sha" -- "$branch"`, then record `<parent-tip>` as the child's new patch base. Publish with `git push --force-with-lease="refs/heads/$branch:$captured_sha" -- "$head_url" "HEAD:refs/heads/$branch"`. When the head and base repositories are the same, create or retarget the child PR with the parent branch as its base. Use `origin pr create --status open --base "$parent_branch"`, `gh pr create --base "$parent_branch" --repo "$base_repo"`, `origin pr edit "$pr" --base "$parent_branch"`, or `gh pr edit "$pr" --base "$parent_branch" --repo "$base_repo"` according to the resolved forge and operation. When the head repository is a fork, keep the local child branch rebased onto its parent's exact tip but create or retarget every PR against `<trunk>` in the base repository. With GitHub, capture the approved PR title and body as `<title>` and `<body>`, then create it with `gh api --method POST "repos/$base_repo/pulls" -f "title=$title" -f "body=$body" -f "head=$fork_owner:$branch" -f "head_repo=$head_name" -f "base=$trunk" --jq .html_url`; add `-F draft=true` only when Opening a PR's readiness rule requires a draft. Otherwise use the resolved Origin equivalent. Retarget it with `gh pr edit "$pr" --base "$trunk" --repo "$base_repo"` or the resolved Origin equivalent. A forge cannot use a fork-only parent branch as a PR base. +3. **Absorb drift at the root.** Until landing starts, fetch current trunk through `<base-remote>` and rebase the chain bottom-up with step 2's explicit old-base and new-parent flow. When a rebase surfaces conflicts in an owner's files, that owner fixes its own slice and you push the result using step 2's captured-SHA lease flow. A rebase rewrites every SHA above it, so apply the patch-id rule in `playbooks/shipping.md` step 3 link by link, since a link's base is its parent, using each PR's recorded patch-base-to-head diff. Anything no longer valid goes back through Autopilot-full step 4 before delivery. Re-run mergeability and CI after every rewritten push even when the patch ID is unchanged. +4. **Deliver the chain, then keep your hands off it.** The deliverable is the frozen bottom-to-top list, reviewable in the resolved forge, and each link carries its verdict in its PR. Same-repository heads use a base-branch chain. Fork heads retain local parent ancestry while every PR targets trunk. The operator sends only the current bottom PR through Shipping, one at a time. A merge-when-ready request applies only to that bottom after Shipping has disarmed and confirmed every descendant; never offer or arm merge-when-ready for the chain as a whole or for a descendant. Once landing starts, nobody writes to the chain except the post-merge move that Shipping step 4 sanctions, because a second writer breaks a landing in flight. A mid-landing problem goes through you to the operator, and any fix waits for the operator's go. **Choosing between the autopilots.** Autopilot-full when the PRs are independent and landing authority is granted. Autopilot-stack when the operator wants review before landing, the work is sequenced or coupled, or merge authority is withheld. -**Reply:** links to the stack root and tip, a one-line verdict summary per link, and anything parked or excluded with the reason. +**Reply:** catch the operator up per the **ketchup** skill, with links to the stack root and tip, a one-line verdict per link, and anything parked or excluded with the reason. diff --git a/plugins/pstack/skills/poteto-mode/playbooks/bug-fix.md b/plugins/pstack/skills/poteto-mode/playbooks/bug-fix.md index 530e3c01..63844d6d 100644 --- a/plugins/pstack/skills/poteto-mode/playbooks/bug-fix.md +++ b/plugins/pstack/skills/poteto-mode/playbooks/bug-fix.md @@ -1,6 +1,6 @@ ### Bug fix -**You own this task. Plan, review, verify.** Delegate investigation and the fix to subagents, stay in the lead. +**You own this task. Plan, review, verify.** Delegate investigation and the fix to subagents, stay in the lead. Pick each subagent's effort for the job, per the Subagents section. Be scientific. Every shipped line traces to runtime evidence. Belt-and-suspenders that "might help" is a hypothesis, not a fix. It does not ship. When evidence refutes a hypothesis, revert what it motivated. The smallest change the evidence justifies ships, nothing more. diff --git a/plugins/pstack/skills/poteto-mode/playbooks/eval.md b/plugins/pstack/skills/poteto-mode/playbooks/eval.md index 3435bfab..b876ede1 100644 --- a/plugins/pstack/skills/poteto-mode/playbooks/eval.md +++ b/plugins/pstack/skills/poteto-mode/playbooks/eval.md @@ -17,7 +17,7 @@ 1. **Frame.** State what variant is under test and what behavior counts as success. Write the rubric (3-6 concrete criteria) for the judge only. Hold it back from candidates. 2. **Set up sanitized environments.** Per-candidate working dir with the variant in place. Plant any context an organic task would have: a project skeleton, the skills the candidate would naturally read. 3. **Author one organic prompt.** What a user would type. No leakage of what's being measured. -4. **Spawn N parallel candidates** on different models per the **arena** skill's Phase B. Each works in its own sanitized dir. Same prompt to each. +4. **Spawn N parallel candidates** on different models per the **arena** skill's Phase B. Each works in its own sanitized dir. Same prompt to each. Pick each subagent's effort for the job, per the Subagents section. 5. **Spawn one blinded judge** on a different model family per the **arena** skill's Phase C. Judge sees outputs by sanitized label and the rubric, never a model name. 6. **Verify the chain from transcripts, not self-report.** Read each candidate's local transcript under Claude Code's per-project transcripts directory at `~/.claude/projects/<encoded-cwd>/` (one `*.jsonl` per session for this workspace). Do not glob across `~/.claude/projects/`. That crosses workspace boundaries and reads private chats from unrelated projects. Look at which files each candidate actually opened. Grade chain-following from the files it really read plus the shape of the code, never from the candidate's own claims. 7. **Read every candidate output yourself** end to end. Compare to the judge's verdict. Disagreement means a model is biased or the rubric is ambiguous. Synthesize. diff --git a/plugins/pstack/skills/poteto-mode/playbooks/feature.md b/plugins/pstack/skills/poteto-mode/playbooks/feature.md index 86784215..ceb3e71b 100644 --- a/plugins/pstack/skills/poteto-mode/playbooks/feature.md +++ b/plugins/pstack/skills/poteto-mode/playbooks/feature.md @@ -9,7 +9,7 @@ - **Independent workstreams.** Disjoint files, services, or layers parallelize. Shared writes serialize. - **Shared mutable state.** Default to splitting the target (the **separate-before-serializing-shared-state** principle skill). Serialize only for real invariants. - **Smallest safe decomposition.** If one worker is best, name why. -4. Delegate code-writing through provider dispatch using your configured `feature, refactoring` descriptor (or its [role default](../references/model-catalog.md#role-defaults)) with `isolated-write`, a dedicated worktree, and a specific scope (file paths, named data shape and its organizing structure per **principle-model-the-domain**, a state machine over scattered booleans, a table/registry over branching, a typed model over repeated shape assumptions, chosen before the delegate writes logic, and success criteria). When the implementation admits multiple valid shapes (error handling, abstraction layer, test structure), delegate via the **arena** skill instead so the runners surface the alternatives and the cross-judge guards the pick. Mandatory: no skip-with-reason escape, and Laziness Protocol does not override it (the gain is review separation, not lines saved). The delegate owns the diff directly and never waits on or launches a nested agent. Comments per **Comments**. Surgical edits, re-ground against the source for upstream-derived files. Port shared-primitive improvements to all consumers and verify each. Commit liberally. +4. Delegate code-writing through provider dispatch using your configured `feature, refactoring` descriptor (or its [role default](../references/model-catalog.md#role-defaults)) with `isolated-write`, a dedicated worktree, and a specific scope (file paths, named data shape and its organizing structure per **principle-model-the-domain**, a state machine over scattered booleans, a table/registry over branching, a typed model over repeated shape assumptions, chosen before the delegate writes logic, and success criteria). Pick each subagent's effort for the job, per the Subagents section. When the implementation admits multiple valid shapes (error handling, abstraction layer, test structure), delegate via the **arena** skill instead so the runners surface the alternatives and the cross-judge guards the pick. Mandatory: no skip-with-reason escape, and Laziness Protocol does not override it (the gain is review separation, not lines saved). The delegate owns the diff directly and never waits on or launches a nested agent. Comments per **Comments**. Surgical edits, re-ground against the source for upstream-derived files. Port shared-primitive improvements to all consumers and verify each. Commit liberally. 5. Verify on the matching surface. "Inconclusive" or wrong-surface is not a pass. Flag it. 6. Rebase into small, ordered commits. Stack follow-ups. Use the **sequence-verifiable-units** principle skill, building, verifying, and committing each small unit before the next. diff --git a/plugins/pstack/skills/poteto-mode/playbooks/hillclimb.md b/plugins/pstack/skills/poteto-mode/playbooks/hillclimb.md index eb5e7239..a667d1f7 100644 --- a/plugins/pstack/skills/poteto-mode/playbooks/hillclimb.md +++ b/plugins/pstack/skills/poteto-mode/playbooks/hillclimb.md @@ -9,7 +9,7 @@ Core discipline: one change, one measurement, keep or revert. Never stack untest 3. Open the decision log via the **show-me-your-work** skill. A `decision.tsv`, one row per attempt: id, hypothesis, change, before, after, delta, tests, verdict (kept or reverted), note. Read it before each attempt. Keep it out of the tree (gitignored). 4. Ground each hypothesis in the architecture model from step 1, so it names a specific mechanism ("defer X off the boot path because it blocks first paint"), not "try memoizing something". For a perf metric, order hypotheses by the performance mantras in step 2 of the Perf issue playbook (`playbooks/perf-issue.md`). Borrow only their order, not that step's stop rule. 5. Loop, one hypothesis per iteration: - - Hand the change through provider dispatch using your configured `hillclimb` descriptor (or its [role default](../references/model-catalog.md#role-defaults)) with `isolated-write` and a tight worktree scope. Supervise and review the diff rather than typing it (the **guard-the-context-window** principle skill). When several independent hypotheses are live, fan them to parallel lanes, each in its own worktree (the **separate-before-serializing-shared-state** principle skill). + - Hand the change through provider dispatch using your configured `hillclimb` descriptor (or its [role default](../references/model-catalog.md#role-defaults)) with `isolated-write` and a tight worktree scope. Pick each subagent's effort for the job, per the Subagents section. Supervise and review the diff rather than typing it (the **guard-the-context-window** principle skill). When several independent hypotheses are live, fan them to parallel lanes, each in its own worktree (the **separate-before-serializing-shared-state** principle skill). - Measure before and after with the frozen harness, and run the regression gate. - Accept only when the metric moves past noise and the gate stays green. Otherwise revert the change in full. A tweak that "might help" is not kept. - One commit per accepted fix, staging only the files you changed (`git add <files>`, never `-A`). Log the row either way, kept or reverted. diff --git a/plugins/pstack/skills/poteto-mode/playbooks/multi-phase-plan.md b/plugins/pstack/skills/poteto-mode/playbooks/multi-phase-plan.md index a4b6989e..7eb10146 100644 --- a/plugins/pstack/skills/poteto-mode/playbooks/multi-phase-plan.md +++ b/plugins/pstack/skills/poteto-mode/playbooks/multi-phase-plan.md @@ -4,7 +4,7 @@ 1. When the change is one or two files with an obvious approach, skip the plan. Say so and stop. 2. Settle open questions by prototype before you write. Run `playbooks/prototype.md` for each. Keep the branch, the SHA, and the screenshots for Appendix A. Ask the operator only about a product or preference call that no run can settle. Give options (the **never-block-on-the-human** principle skill). -3. Explore in subagents. Resolve the configured `judgment and prose` role through [`../references/provider-dispatch.md`](../references/provider-dispatch.md). Use `poteto-agent` only for an unqualified `inherit-parent` / `auto` native helper. Never use Claude Code's built-in `Plan` agent. It ignores this skill (the **guard-the-context-window** principle skill). Each explorer returns file pointers, conventions, test commands, and entry points. No inlined dumps. Children do not detect the parent harness or choose a route. Preserve the selected effort. A dropout stays a dropout. Do not add a fallback or an implicit timeout. +3. Explore in subagents. Resolve the configured `judgment and prose` role through [`../references/provider-dispatch.md`](../references/provider-dispatch.md). Use `poteto-agent` only for an unqualified `inherit-parent` / `auto` native helper. Never use Claude Code's built-in `Plan` agent. It ignores this skill (the **guard-the-context-window** principle skill). Pick each subagent's effort for the job, per the Subagents section. Each explorer returns file pointers, conventions, test commands, and entry points. No inlined dumps. Children do not detect the parent harness or choose a route. Preserve the selected effort. A dropout stays a dropout. Do not add a fallback or an implicit timeout. 4. Copy the skeleton below into the plan file and fill every placeholder. Unless the operator names a path, write the file under the working repository's `docs/`. Keep every heading and every sub-block in the order shown. One section per PR. One PR is one change with its own evidence (the **sequence-verifiable-units** principle skill). Name the execution playbook in **How to read this**. Pick between `playbooks/autopilot-full.md` and `playbooks/autopilot-stack.md` per the rule at the end of `playbooks/autopilot-stack.md`. A standing program takes `playbooks/orchestrate.md`. 5. Write under `/technical-writing` in full, then `/unslop`. The body is one Diátaxis mode, how-to. Appendices hold explanation and reference. Each heading states the task or the finding. No long dashes. No mid-sentence colons. 6. Run `node skills/poteto-mode/scripts/check-plan.mjs <plan.md>` under the installed plugin and fix every line it prints (the **encode-lessons-in-structure** principle skill). It enforces the skeleton's shape, the verification rule in every verification block, and the punctuation rules. The playbook file is not checker input. Check the plan file created in step 4. @@ -64,8 +64,8 @@ Tests alone are not sufficient verification. A PR is verified only when its unit ### Verdict and merge, for every PR -- [ ] At the code-ready head SHA and at each later push that changes the patch, run the swarm per `skills/swarm/SKILL.md`. One gates lane. The ten live lanes from the PR's **Verify, live** block. The perf lane from its **Verify, perf** block. Two or more audit lanes, each with its own focus, that read the diff and the receipts and distrust the PR body. The root audits the receipts in the merge-ready report before the verdict. -- [ ] Clean only when every lane is `PASS`. Findings go back to the owner, including a defect that a lane filed as a note. A new head gets a fresh swarm and a fresh verdict, except for results that stay valid under the patch ID rule in `skills/poteto-mode/playbooks/shipping.md`. +- [ ] A full swarm runs once, at the code-ready head. Another full swarm runs only for new behavior, a redesign, or a large new diff. A rebase, a merge-conflict resolution, or a minor fix gets a single verifier at the new head. Run each full swarm per `skills/swarm/SKILL.md`. One gates lane. The ten live lanes from the PR's **Verify, live** block. The perf lane from its **Verify, perf** block. Two or more audit lanes, each with its own focus, that read the diff and the receipts and distrust the PR body. The root audits the receipts in the merge-ready report before the verdict. +- [ ] Clean only when every lane is `PASS`. Findings go back to the owner, including a defect that a lane filed as a note. A new head gets a fresh verdict, except for results that stay valid under the patch ID rule in `skills/poteto-mode/playbooks/shipping.md`. - [ ] <The merge or append rule from the execution playbook, with the verdict SHA, current landing SHA, recorded patch base, and patch ID rule from `skills/poteto-mode/playbooks/shipping.md`.> ### Boot recipe, for every live lane diff --git a/plugins/pstack/skills/poteto-mode/playbooks/opening-a-pr.md b/plugins/pstack/skills/poteto-mode/playbooks/opening-a-pr.md index b39571cd..04a24a22 100644 --- a/plugins/pstack/skills/poteto-mode/playbooks/opening-a-pr.md +++ b/plugins/pstack/skills/poteto-mode/playbooks/opening-a-pr.md @@ -23,7 +23,7 @@ After the sections, attach videos or screenshots when they prove a claim. Full S **Forge.** Resolve the forge before the first PR operation and keep that choice for create, edit, view, watch, and merge. GitHub CLI (`gh`) is the default. If `command -v origin` succeeds and Origin can resolve the repository, prefer `origin pr ...`. If Origin is absent or cannot resolve the repository, stay on `gh` and record the fallback. Record the intended PR base repository as canonical `<base-repo>` and validate it through the active forge. Do not infer it from the checkout's default remote. Capture it as a shell variable and pass `--repo "$base_repo"` to every `gh pr` command. When the head repository is a fork, validate its identity and record its owner and repository name as `<fork-owner>` and `<head-name>`. Do not require Graphite (`gt`). -**Size and stacks.** Prefer five narrow PRs to one large PR. Rebase each child branch onto its parent's exact tip and freeze the bottom-to-top order. When the head and base repositories are the same, make a base-branch chain. The root PR targets trunk and each child PR targets the parent branch. Create a same-repository child with `origin pr create --status open --base "$parent_branch"` or `gh pr create --base "$parent_branch" --repo "$base_repo"` according to the resolved forge. When the head repository is a fork, every PR targets trunk in the base repository while stacked local branches retain parent ancestry. Create every fork PR with the resolved Origin command. With GitHub, capture the approved PR title and body as `<title>` and `<body>`, then run `gh api --method POST "repos/$base_repo/pulls" -f "title=$title" -f "body=$body" -f "head=$fork_owner:$branch" -f "head_repo=$head_name" -f "base=$trunk" --jq .html_url`; add `-F draft=true` only when the readiness rule requires a draft. A fork-only parent branch cannot be a PR base. Before rebasing, force-pushing, or retargeting an existing child, apply Shipping step 4's disarm-and-confirm rule to that child and every descendant. Retarget a same-repository child with `origin pr edit "$pr" --base "$parent_branch"` or `gh pr edit "$pr" --base "$parent_branch" --repo "$base_repo"`. Retarget a fork child with the resolved Origin command or `gh pr edit "$pr" --base "$trunk" --repo "$base_repo"`. Branch from trunk only for independent work. Rebase on trunk before substantial stack work. +**Size and stacks.** Prefer five narrow PRs to one large PR. Rebase each child branch onto its parent's exact tip and freeze the bottom-to-top order. When the head and base repositories are the same, make a base-branch chain. The root PR targets trunk and each child PR targets the parent branch. Create a same-repository child with `origin pr create --status open --base "$parent_branch"` or `gh pr create --base "$parent_branch" --repo "$base_repo"` according to the resolved forge. When the head repository is a fork, every PR targets trunk in the base repository while stacked local branches retain parent ancestry. Create every fork PR with the resolved Origin command. With GitHub, capture the approved PR title and body as `<title>` and `<body>`, then run `gh api --method POST "repos/$base_repo/pulls" -f "title=$title" -f "body=$body" -f "head=$fork_owner:$branch" -f "head_repo=$head_name" -f "base=$trunk" --jq .html_url`; add `-F draft=true` only when the readiness rule requires a draft. A fork-only parent branch cannot be a PR base. As a parent merges, Shipping step 4 moves its child onto trunk. Before rebasing, force-pushing, or retargeting an existing child, apply Shipping step 4's disarm-and-confirm rule to that child and every descendant. Retarget a same-repository child with `origin pr edit "$pr" --base "$parent_branch"` or `gh pr edit "$pr" --base "$parent_branch" --repo "$base_repo"`. Retarget a fork child with the resolved Origin command or `gh pr edit "$pr" --base "$trunk" --repo "$base_repo"`. Branch from trunk only for independent work. Rebase on trunk before substantial stack work. **Readiness.** Open each PR ready by default. With Origin, pass `--status open`. With the GitHub CLI create command, omit `--draft` and pass `--repo "$base_repo"`. With the GitHub fork API above, omit the `draft` field. If repository instructions require a draft until named evidence exists, keep the early PR draft and mark it ready only after recording that evidence. On GitHub, pass `--draft` to the CLI create command or `-F draft=true` to the fork API, then later run `gh pr ready "$pr" --repo "$base_repo"`. Use Origin's documented draft and ready operations when Origin is active. If no draft rule applies and a tool still opens the PR as a draft, run `origin pr ready "$pr"` or `gh pr ready "$pr" --repo "$base_repo"` according to the resolved forge. Run `origin pr view "$pr"` or `gh pr view "$pr" --repo "$base_repo"` before you refer to PR status. diff --git a/plugins/pstack/skills/poteto-mode/playbooks/orchestrate.md b/plugins/pstack/skills/poteto-mode/playbooks/orchestrate.md index aaf02a02..42cfa0be 100644 --- a/plugins/pstack/skills/poteto-mode/playbooks/orchestrate.md +++ b/plugins/pstack/skills/poteto-mode/playbooks/orchestrate.md @@ -12,7 +12,7 @@ Three rules carry the rest. #### Roles and placement -- **Coordinator (this chat).** Frames, authors briefs, drains the inbox, owns the human report, makes judgment calls. It never authors or edits code. Conflicted merges, restacks, and code changes are always tasks. Mechanically landing a verified unit (fast-forward or clean cherry-pick of a worker's commit, then push) is bookkeeping the coordinator may do itself on repos where local git is cheap. Queueing finished work behind an idle stacker is how a deadline harvests nothing. The loop is agentic end to end. Agents are spawned, resumed, and drained only through the `Agent` tool. State reads and writes go through the `orch` CLI at drain points, one command in and one line out. The CLI never spawns, waits, or wakes anything. +- **Coordinator (this chat).** Frames, authors briefs, drains the inbox, owns the human report, makes judgment calls. It never authors or edits code. Conflicted merges, chain rebases, and code changes are always tasks. Mechanically landing a verified unit (fast-forward or clean cherry-pick of a worker's commit, then push) is bookkeeping the coordinator may do itself on repos where local git is cheap. Queueing finished work behind an idle stacker is how a deadline harvests nothing. The loop is agentic end to end. Agents are spawned, resumed, and drained only through the `Agent` tool. Pick each subagent's effort for the job, per the Subagents section. State reads and writes go through the `orch` CLI at drain points, one command in and one line out. The CLI never spawns, waits, or wakes anything. - **Sub-coordinator.** Durable, one per track, and only when the program exceeds what one coordinator's drains can manage. A track the coordinator can drain itself needs no middle layer. Each nested layer re-pays a full orientation preamble, and a blocking sub-coordinator hides its children while the parent idles. Owns its track's units and boards, authors its workers' briefs, spawns its own workers and verifiers where the runtime lets a subagent spawn one; where it does not, it owns its track's units directly with the same review separation. Rolls up aggregates at wave boundaries. Never forwards raw child reports. Cap in-flight children at what one drain can process, roughly ten, as a rolling window. Never as blocking batches, which cost the slowest child of every batch. - **Worker / verifier.** Background subagents (`run_in_background: true`). Claude Code has no remote worker environment, so isolation is a worktree or branch per writer, not a separate machine. Runtime verification goes through the `run` skill (CLIs and TUIs) or the `verify` skill (UIs). A subagent never sees this chat, so its brief inlines what it needs or points at repo and store paths. Prefer fewer, broader workers. One writer per worktree or branch (principle-separate-before-serializing-shared-state). Run a unit's verifier on a different model family from its worker. @@ -44,7 +44,8 @@ ACCEPTANCE checkable criteria, one per line VERIFY exact commands or the driver skill (`run` for CLIs and TUIs, `verify` for UIs), plus known gotchas TIMEBOX rough cap on runtime; on expiry, return partial findings and stop rather than run on -FORBIDDEN no gt, no rebase, no force-push, no fixes outside scope, plus unit-specific bans +FORBIDDEN no PR retargets or merges, no rebase, no force-push, no fixes outside scope, + plus unit-specific bans REPORT status, branch, head SHA, PRs, verdict, what you actually ran, deviations, suggested follow-ups STANDING <preferences.md pasted verbatim> @@ -59,7 +60,7 @@ A dependency is a context relay, not just ordering. Undeclared upstream context #### Steps 1. **Frame.** State the done predicate as something countable ("all 126 units merged, each ledger-verified `unit-test-verified` or better"). Quantify scope: units, rough effort, expected stacks, and the wall-clock budget. If one agent could finish inside that budget, stop here and run Autonomous run instead. Collapsing must not depend on another document being present. It means do the work directly in this session, plain workers where they help, verification inline, landing as you go, and none of the store, register, or pilot machinery below. Schedule landing against the budget. By roughly 70% of it, stop spawning and land what is verified. Name the tracks per project. A contested decomposition or one-way door goes through the arena skill before the pilot. Present the framing once. Reversible prep proceeds without waiting. -2. **Install the runtime.** Run `orch init`. Open the trail via the show-me-your-work skill, write the standing orders before any spawn, and seed `frontier.json` from existing PRs with `orch frontier set --repo <repo-dir>`. +2. **Install the runtime.** Run `orch init`. Open the trail via the show-me-your-work skill, write the standing orders before any spawn, and seed `frontier.json` from existing PRs with `orch frontier set --repo <repo-dir>`, adding `--forge origin` when the forge is Origin. 3. **Pilot.** Push one unit through the whole path: brief, worker, verification, stack entry, ledger row, merge. The pilot exists to falsify the brief template, the verify recipe, and the unit size while that costs one agent instead of fifty. Fix the contract from pilot evidence before any fan-out. Scale the pilot to the unit. On programs of near-identical cheap units, the first unit is the pilot, run as a normal unit with its verify command inline, and fan-out starts the moment it lands. The dedicated pilot pipeline (separate verifier agent, audit gate) is for expensive or novel unit shapes, not for clone-units where a serialized pilot has nothing to falsify. 4. **Scale.** Spawn a rolling window of workers up to the in-flight cap, refilling as children finish. Blocking batches pay the slowest child of every batch. Spawn track sub-coordinators only past the one-drain threshold in Roles. Recompute ready work after each drain. Relay upstream reports into downstream briefs. Keep sibling communication upward only. The sampled brief audit runs alongside the wave it samples and stops the next refill on failure, not the current one. 5. **Drain.** Run the queue discipline below at every drain point. @@ -77,9 +78,9 @@ A dependency is a context relay, not just ordering. Undeclared upstream context #### Stack safety -- The frontier is a computed object, never narrative. Recompute `frontier.json` from `gt` after every merge and stack mutation because GitHub base refs drift mid-restack while gt tracking is authoritative: ordered PR list, branch names, head SHAs, a generation number, the lowest unmerged PR. Resolve it where gt knows the stack, normally the stacker's clone. A checkout whose gt metadata never saw the submits reports no PRs and the command errors rather than guessing. -- Exactly one stacker per stack may run `gt`, serialized within its stack. Record the holder in the standing orders. A restack at this scale is slow and blocks whoever runs it, so give it its own unit and keep the coordinator out of it. -- Workers never rebase and never run `gt`. Babysitters follow `playbooks/babysit.md`, one per stack, scoped to one immutable frontier generation. They report conflicts to the stacker rather than restacking. +- The frontier is a computed object, never narrative. Recompute `frontier.json` after every merge and stack mutation: ordered PR list, branch names, head SHAs, a generation number, the lowest unmerged PR. The command walks the PR base chain through the checkout's current branch, so chain order and PR states come from the forge. It uses `gh` by default. Pass `--forge origin` when the program resolved the forge to Origin. Head SHAs come from local branch refs, so resolve it from the stacker's clone. A branch with no PR errors rather than guessing. +- Exactly one stacker per stack may mutate topology (chain rebases, force-pushes, base retargets, merges), serialized within its stack. Record the holder in the standing orders. A chain rebase at this scale is slow and blocks whoever runs it, so give it its own unit and keep the coordinator out of it. +- Workers never rebase and never retarget or merge PRs. Babysitters follow `playbooks/babysit.md`, one per stack, scoped to one immutable frontier generation. They report conflicts to the stacker rather than rebasing. - PR closes and retargets go through the stacker only. Closing a base PR orphans every chain above it. Merges and stack surgery are units with briefs like any other. - One retro watcher follows merged PRs for reverts, post-merge CI breaks, and orphaned follow-ups. @@ -105,7 +106,7 @@ A unit is not done until its output is externalized the moment it lands, never b Reaches the human, batched into the status page rather than per item: irreversible actions (force-push to shared branches, deploys, deletions, closing someone else's PR), genuine product or preference calls no experiment settles, a standing order that contradicts observed reality, a program-level dead end that survived a replan. Park each as a `gates.md` entry before asking, and route work around it. -Never reaches the human: frontier nudges, restack mechanics, retries, CI flake triage, review-thread triage, format fixes, scope the brief already forbids (refuse and continue), and "should I keep going". When in doubt, act and log. +Never reaches the human: frontier nudges, rebase mechanics, retries, CI flake triage, review-thread triage, format fixes, scope the brief already forbids (refuse and continue), and "should I keep going". When in doubt, act and log. Mid-run discoveries fix only what blocks the frontier. Everything else parks in follow-ups. At this fan-out a small scope leak multiplies into PRs nobody asked for. diff --git a/plugins/pstack/skills/poteto-mode/playbooks/perf-issue.md b/plugins/pstack/skills/poteto-mode/playbooks/perf-issue.md index b44aa2be..618f32e9 100644 --- a/plugins/pstack/skills/poteto-mode/playbooks/perf-issue.md +++ b/plugins/pstack/skills/poteto-mode/playbooks/perf-issue.md @@ -14,7 +14,7 @@ 7. Do it cheaper. When an earlier mantra meets the target, stop. -3. Plan the fix from the trace. If it crosses a function boundary, `architect` first. Delegate implementation through provider dispatch using your configured `perf-issue` descriptor (or its [role default](../references/model-catalog.md#role-defaults)) with `isolated-write` in a dedicated worktree. Review the diff. Capture a post-fix trace. +3. Plan the fix from the trace. If it crosses a function boundary, `architect` first. Delegate implementation through provider dispatch using your configured `perf-issue` descriptor (or its [role default](../references/model-catalog.md#role-defaults)) with `isolated-write` in a dedicated worktree. Pick each subagent's effort for the job, per the Subagents section. Review the diff. Capture a post-fix trace. Apply the **sequence-verifiable-units** principle skill, verifying each attempt before trying the next. 4. Parse and compare the artifacts (JSON to sqlite, diff). "Inconclusive" or wrong-surface is not a pass. Flag it. 5. Cite the measurement in the PR. diff --git a/plugins/pstack/skills/poteto-mode/playbooks/refactoring.md b/plugins/pstack/skills/poteto-mode/playbooks/refactoring.md index 3bfc5a9e..efb8d6ab 100644 --- a/plugins/pstack/skills/poteto-mode/playbooks/refactoring.md +++ b/plugins/pstack/skills/poteto-mode/playbooks/refactoring.md @@ -8,7 +8,7 @@ If the cleanup reveals a missing feature or a real bug, split it out and ship th 2. Name the structure the code is missing per **principle-model-the-domain**. Boring code stays when the shape is already clear and local. The reshape must delete branches or invalid states, not add indirection. 3. Name the target shape. State what the module layout, types, and call graph should be if built today (**principle-foundational-thinking**, **principle-redesign-from-first-principles**). If the target crosses a function boundary, run the **architect** skill for parallel design exploration of the shape before the move. 4. Subtract before you add. Delete dead code, collapse one-caller wrappers, drop redundant validators, and remove orphan references before introducing the new shape (**principle-subtract-before-you-add**). The smallest change that reaches the target shape ships (**principle-laziness-protocol**). A speculative cleanup that "might help" gets reverted. -5. Move in small behavior-preserving steps, each keeping the pin green. For API reshapes, migrate every caller and delete the old API in the same wave (**principle-migrate-callers-then-delete-legacy-apis**). No compatibility shims, no parallel old-and-new paths. Spot-check every rename against the actual files. Renames silently miss usages in strings, prose, and back-references. Delegate the mechanical edits through provider dispatch using your configured `feature, refactoring` descriptor (or its [role default](../references/model-catalog.md#role-defaults)) with `isolated-write`, a dedicated worktree, and a specific scope (file paths, the names being moved, the behavior to hold). +5. Move in small behavior-preserving steps, each keeping the pin green. For API reshapes, migrate every caller and delete the old API in the same wave (**principle-migrate-callers-then-delete-legacy-apis**). No compatibility shims, no parallel old-and-new paths. Spot-check every rename against the actual files. Renames silently miss usages in strings, prose, and back-references. Delegate the mechanical edits through provider dispatch using your configured `feature, refactoring` descriptor (or its [role default](../references/model-catalog.md#role-defaults)) with `isolated-write`, a dedicated worktree, and a specific scope (file paths, the names being moved, the behavior to hold). Pick each subagent's effort for the job, per the Subagents section. 6. Prove behavior is unchanged on the real artifact, not "it compiles" (**principle-prove-it-works**). For larger reshapes, run an equivalence check: a script that diffs old-vs-new outputs, a recorded baseline replayed against the new code, or a smoke run on the matching surface via the driver skill (`run` for CLIs/TUIs, `verify` for UIs). 7. Confirm the change is worth keeping. The success measure is reduced reader load (**principle-minimize-reader-load**). If the diff does not lower reader load somewhere, revert it. 8. Rebase into small ordered commits. A subtraction commit, then the reshape, then any follow-on cleanup. Shape them with the **sequence-verifiable-units** principle skill, so each behavior-preserving slice stays green before the next. Run **Opening a PR**. diff --git a/plugins/pstack/skills/poteto-mode/playbooks/runtime-forensics.md b/plugins/pstack/skills/poteto-mode/playbooks/runtime-forensics.md index cb5e6418..0ded5c2a 100644 --- a/plugins/pstack/skills/poteto-mode/playbooks/runtime-forensics.md +++ b/plugins/pstack/skills/poteto-mode/playbooks/runtime-forensics.md @@ -3,7 +3,7 @@ **You own the diagnosis. Instrument the live process, don't theorize from source.** The deliverable is a cited diagnosis, not a fix. 1. Capture the live signal on the matching surface via the driver skill (`run` for CLIs/TUIs, `verify` for UIs): a CPU profile for a spinning process, a heap snapshot for a leak, a CDP trace for a visual glitch. A real artifact, not a guess. -2. Reduce the artifact to the smoking gun: the function on the hot path, the retainer chain from the leaked object to a GC root, the loop firing without input. Parse large artifacts in a subagent (the **guard-the-context-window** principle skill), keep the reduced finding in the main thread. +2. Reduce the artifact to the smoking gun: the function on the hot path, the retainer chain from the leaked object to a GC root, the loop firing without input. Parse large artifacts in a subagent (the **guard-the-context-window** principle skill), keep the reduced finding in the main thread. Pick each subagent's effort for the job, per the Subagents section. 3. Prove the mechanism before believing it. Inject instrumentation via CDP eval on the running process, or hotfix the live code without reloading, to confirm the hypothesis cheaply. 4. Map the finding back to source: file, symbol, the line that allocates or schedules. 5. Throughput checkpoint stays one line: `throughput checkpoint: n/a, read-only forensics`. diff --git a/plugins/pstack/skills/poteto-mode/playbooks/session-pickup.md b/plugins/pstack/skills/poteto-mode/playbooks/session-pickup.md index 2fb49be0..01b8cea4 100644 --- a/plugins/pstack/skills/poteto-mode/playbooks/session-pickup.md +++ b/plugins/pstack/skills/poteto-mode/playbooks/session-pickup.md @@ -2,7 +2,7 @@ **You own the resume point. Read the prior trail, don't redo it.** -1. Locate the prior trail. A local transcript under Claude Code's per-project transcripts directory at `~/.claude/projects/<encoded-cwd>/*.jsonl` (where `<encoded-cwd>` is the workspace cwd with `/` → `-`. Do not glob across other directories under `~/.claude/projects/`, that crosses workspace boundaries and reads private chats from unrelated projects), a cloud-agent URL, or a pushed branch. Read the metadata overview and last messages first, then scan back for the decision points. Parse a long transcript in a subagent and keep the reduced timeline in the main thread (the **principle-guard-the-context-window** skill). +1. Locate the prior trail. A local transcript under Claude Code's per-project transcripts directory at `~/.claude/projects/<encoded-cwd>/*.jsonl` (where `<encoded-cwd>` is the workspace cwd with `/` → `-`. Do not glob across other directories under `~/.claude/projects/`, that crosses workspace boundaries and reads private chats from unrelated projects), a cloud-agent URL, or a pushed branch. Read the metadata overview and last messages first, then scan back for the decision points. Parse a long transcript in a subagent and keep the reduced timeline in the main thread (the **principle-guard-the-context-window** skill). Pick each subagent's effort for the job, per the Subagents section. 2. Reconstruct operational state. The branch and worktree, what already landed (`git log`, `git diff` against the base), the open todos, the decisions made. The prior trail is authoritative input. Resist the bias to re-derive it. 3. Diff done vs pending. Compare what shipped against what was planned, name the resume point, do not re-run the prior repro or redo completed work. A "let me verify from scratch" pass means you're treating the trail as untrustworthy when it's authoritative. 4. Route the remaining work to the matching playbook and pick the verdict: continue the execution, ship a finished recommendation, ratify or override a prior conclusion, or postmortem a failed run. The pickup playbook ends here. The routed playbook owns the rest. diff --git a/plugins/pstack/skills/poteto-mode/playbooks/shipping.md b/plugins/pstack/skills/poteto-mode/playbooks/shipping.md index 068c3530..45c9e657 100644 --- a/plugins/pstack/skills/poteto-mode/playbooks/shipping.md +++ b/plugins/pstack/skills/poteto-mode/playbooks/shipping.md @@ -1,17 +1,15 @@ ### Shipping -**You own what lands. Verify each PR independently, land only the verified run from the root, then keep your hands off the queue.** - -This is the half after `playbooks/babysit.md`. +**You own what lands. Verify each PR independently, land only the verified run from the root, then keep your hands off the stack.** 1. **Resolve the forge, repository identity, and both Git remotes.** GitHub CLI (`gh`) is the default. If `command -v origin` succeeds and Origin can resolve the repository, use `origin pr ...` for PR view, watch, and edit operations; use it for merge only when step 5's expected-head requirement is available. Otherwise stay on `gh` and record the fallback. Record the forge-reported base repository as canonical `<base-repo>`. On GitHub, split it into `<base-owner>` and `<base-name>`, pass `--repo "$base_repo"` to every `gh pr` command, and pass the two components to the GitHub watcher. Resolve `<head-remote>` from the branch's configured push remote, `remote.pushDefault`, branch remote, or sole unambiguous remote, in that order. Confirm its push URL names the PR head repository, and record that exact URL as `<head-url>` so the captured remote state and the guarded push address the same repository. Resolve `<base-remote>` independently by matching a fetch URL to `<base-repo>`, and confirm that URL before using it for trunk. The same remote name may fill both roles when its push URL matches the head repository and its fetch URL matches the base repository. Compare each URL by role instead of assuming the remote name identifies one repository. Capture every resolved or forge-reported value directly into a shell variable. Command examples use quoted lower-case variables such as `"$branch"` and `"$head_url"`. Never paste those values into shell source. Never guess or treat the forge name as a Git remote, and never require Graphite (`gt`). -2. **Freeze and disarm the queue before verification.** Freeze an explicit bottom-to-top PR list. Confirm a same-repository stack against its base-branch chain. For fork heads, take the order from the verified local parent ancestry because every PR targets trunk and the forge bases do not encode the stack. Before launching any verifier, inspect every PR in the frozen list through the active forge, disarm every pre-existing merge-when-ready or auto-merge request, and confirm each request is off. On GitHub, query each PR through GraphQL for `id`, `headRefOid`, `baseRefName`, `autoMergeRequest`, and `mergeQueueEntry`. When `autoMergeRequest` is non-null, run `gh pr merge "$pr" --disable-auto --repo "$base_repo"`. When `mergeQueueEntry` is non-null, invoke the `dequeuePullRequest` mutation with `gh api graphql -F "id=$pr_node_id" -f query='mutation($id:ID!){dequeuePullRequest(input:{id:$id}){mergeQueueEntry{id}}}'`. The mutation takes the pull request node ID. Re-query and require that both `autoMergeRequest` and `mergeQueueEntry` are null. A null `autoMergeRequest` alone does not prove that the pull request is unarmed. On Origin, use its reported cancel operation and inspect every separately reported queue state. Stop if the active forge cannot confirm the whole list is unarmed. One subagent per PR, not batched, each in its own worktree, exercises the real surface against that PR's parent versus head. The bottom PR's patch base is trunk. Each child's patch base is the preceding PR's exact head, including when a fork child targets trunk at the forge. Each subagent returns `PASS`, `PASS+NOTES` or `FAIL` and posts that verdict on its own PR. Safe means a verdict from an agent that did not write the code. CI green is not a verdict, and an approving bot review is not a verdict. Walk up from the bottom and stop at the first PR without a passing verdict, where both `PASS` and `PASS+NOTES` pass. Report that ceiling and what breaks the chain. -3. **Re-check that each verdict still describes the patch.** At the passing verdict, record `<verdict-sha>`, `<verdict-base-sha>`, and the stable `<verdict-patch-id>` of `<verdict-base-sha>..<verdict-sha>`. Initialize `<landing-sha>` and `<landing-base-sha>` to those head and base values. Do not replace a fork child's patch base with its forge base, because that forge base is trunk and includes its unmerged parents. Before landing, read the current published head through the active forge as `<current-head>` and require the local branch tip to match it. Calculate the stable patch ID of `<landing-base-sha>..<current-head>` and compare it with `<verdict-patch-id>`. When the two patches differ only in tests, docs, or lint config, build what each lane ran. Build it twice at `<verdict-sha>` and once at `<current-head>`. A difference is noise if the two builds at `<verdict-sha>` also show it, or if it is an embedded commit SHA. Judge each difference, not each file, and report each kind of noise with its files. If only noise differs, that lane's result stays valid. Set `<landing-sha>` to `<current-head>`, and run checks and a review of the change fresh at that head. Do not reuse a lane result from a dev server or from anything else with no build output. Rerun that lane. Re-verify anything else when the patch changed. When it did not, keep the code verdict, set `<landing-sha>` to `<current-head>`, and re-run mergeability and CI at that head. A patch-equivalent rebase can make `<landing-sha>` differ from `<verdict-sha>`. Keep both values. Never use matching commit messages or a green check from an older SHA as a substitute. +2. **Freeze and disarm the queue before verification.** Freeze an explicit bottom-to-top PR list. Confirm a same-repository stack against its base-branch chain. For fork heads, take the order from the verified local parent ancestry because every PR targets trunk and the forge bases do not encode the stack. Before launching any verifier, inspect every PR in the frozen list through the active forge, disarm every pre-existing merge-when-ready or auto-merge request, and confirm each request is off. On GitHub, query each PR through GraphQL for `id`, `headRefOid`, `baseRefName`, `autoMergeRequest`, and `mergeQueueEntry`. When `autoMergeRequest` is non-null, run `gh pr merge "$pr" --disable-auto --repo "$base_repo"`. When `mergeQueueEntry` is non-null, invoke the `dequeuePullRequest` mutation with `gh api graphql -F "id=$pr_node_id" -f query='mutation($id:ID!){dequeuePullRequest(input:{id:$id}){mergeQueueEntry{id}}}'`. The mutation takes the pull request node ID. Re-query and require that both `autoMergeRequest` and `mergeQueueEntry` are null. A null `autoMergeRequest` alone does not prove that the pull request is unarmed. On Origin, use its reported cancel operation and inspect every separately reported queue state. Stop if the active forge cannot confirm the whole list is unarmed. One subagent per PR, not batched, each in its own worktree, runs the change the way a user would through the driver skill (`run` for CLIs/TUIs, `verify` for UIs), parent against head. The bottom PR's patch base is trunk. Each child's patch base is the preceding PR's exact head, including when a fork child targets trunk at the forge. Each subagent posts `PASS`, `PASS+NOTES`, or `FAIL` on its own PR. Only a verdict from an agent that did not write the code counts, because an author checks what it meant to build. Green CI and an approving bot review are not verdicts, because neither runs the change as a user would. Walk up from the bottom and stop at the first PR without `PASS` or `PASS+NOTES`. The PRs below it are the run, and the top one is the ceiling. A verified PR above an unverified one waits, because it cannot land without the code below it. One whose branch neither holds nor needs that code can move onto trunk and land on its own. It needs a fresh verdict first, because its old one ran with that code beneath it and trunk does not have it. Report the ceiling and what breaks the chain. +3. **Re-check that each verdict still describes the patch.** A verdict covers a patch, and a chain rebase moves every PR above it to a new SHA without failing any check. At the passing verdict, record `<verdict-sha>`, `<verdict-base-sha>`, and the stable `<verdict-patch-id>` of `<verdict-base-sha>..<verdict-sha>`. Initialize `<landing-sha>` and `<landing-base-sha>` to those head and base values. Do not replace a fork child's patch base with its forge base, because that forge base is trunk and includes its unmerged parents. Before landing, read the current published head through the active forge as `<current-head>` and require the local branch tip to match it. Calculate the stable patch ID of `<landing-base-sha>..<current-head>` and compare it with `<verdict-patch-id>`. When the two patches differ only in tests, docs, or lint config, build what each lane ran. Build it twice at `<verdict-sha>` and once at `<current-head>`. A difference is noise if the two builds at `<verdict-sha>` also show it, or if it is an embedded commit SHA. Judge each difference, not each file, and report each kind of noise with its files, so a reader can check the call. If only noise differs, that lane's result stays valid. Set `<landing-sha>` to `<current-head>`, and run checks and a review of the change fresh at that head, because the old verdict never saw it. A lane with no build output, such as a dev server, cannot show this, so rerun it. Re-verify anything else when the patch changed. When it did not, keep the code verdict, set `<landing-sha>` to `<current-head>`, and re-run mergeability and CI at that head. A patch-equivalent rebase can make `<landing-sha>` differ from `<verdict-sha>`. Keep both values. Never use matching commit messages or a green check from an older SHA as a substitute. 4. **Prepare only the bottom PR.** Before touching any branch or base, inspect the current bottom and every descendant through the active forge. On GitHub, repeat step 2's GraphQL state query. Disable each non-null `autoMergeRequest`, dequeue each non-null `mergeQueueEntry`, and confirm that both fields are null for every pull request. On Origin, use its reported cancel operation and inspect every separately reported queue state. Repeat this disarm check in case another actor rearmed a pull request after step 2. Stop before any rebase, force-push, retarget, arm, or merge if the active forge cannot disarm and confirm every PR in that set. Fetch current trunk through `<base-remote>` and record its exact tip as `<trunk-tip>`. Before rebasing the lowest verified branch, record its published SHA with `git ls-remote -- "$head_url" "refs/heads/$branch"` and require that `<captured-sha>` to equal the local pre-rebase tip. Require `git merge-base --is-ancestor "$landing_base_sha" "refs/heads/$branch"` to pass. When `<landing-base-sha>` differs from `<trunk-tip>`, move only this PR's commits with `git rebase --onto "$trunk_tip" "$landing_base_sha" -- "$branch"`, then set `<landing-base-sha>` to `<trunk-tip>`. Publish rewritten history with `git push --force-with-lease="refs/heads/$branch:$captured_sha" -- "$head_url" "HEAD:refs/heads/$branch"`, never a bare lease or plain force. Retarget only that PR to trunk with `origin pr edit "$pr" --base "$trunk"` or `gh pr edit "$pr" --base "$trunk" --repo "$base_repo"`. Re-read `baseRefName` after any retarget and require it to equal `<trunk>`. Record `<trunk>` as `<landing-base-ref>`, separate from the commit-valued `<landing-base-sha>`. After a push or retarget, repeat step 3's patch comparison and current-head checks. Do not retarget, arm, or merge descendants yet. -5. **Land one PR at a time, checked against the current landing SHA and base.** Immediately before any merge or auto-merge request, read the current `headRefOid` and `baseRefName`. Require that `headRefOid` equals `<landing-sha>` and `baseRefName` equals `<trunk>`, the recorded `<landing-base-ref>`. This read is a preflight check, not a lock. Every merge operation needs a server-enforced expected-head precondition. Run the GitHub merge immediately after the matching preflight with `gh pr merge "$pr" --squash --match-head-commit "$landing_sha" --repo "$base_repo"`. GitHub has no server-enforced expected-base precondition, so keep watching `baseRefName` and verify the merged base in step 8 instead of inventing a guard or disabling the GitHub flow. Origin has no documented expected-head option. Use an Origin merge only when the active server reports that guard. Otherwise fall back to a resolvable forge that supplies it or stop before merging. If requirements are still running and the user asked for merge-when-ready, use server-side auto-merge only when the repository has a required, SHA-scoped verification check for the independent verdict that becomes unsatisfied on every head update. On GitHub, run `gh pr merge "$pr" --squash --auto --match-head-commit "$landing_sha" --repo "$base_repo"` immediately after the matching preflight, then disarm it on any observed head or base change. When GitHub adds the pull request to a required merge queue instead of merging it, treat a non-null `mergeQueueEntry` as an armed request and keep watching both the head and the base. Without the required verification check, keep the dynamic watch active and run the guarded immediate merge when the pull request becomes ready instead of arming server-side auto-merge. Wait for that PR to merge before preparing the next one. +5. **Land one PR at a time, checked against the current landing SHA and base.** Immediately before any merge or auto-merge request, read the current `headRefOid` and `baseRefName`. Require that `headRefOid` equals `<landing-sha>` and `baseRefName` equals `<trunk>`, the recorded `<landing-base-ref>`, because a PR whose base is still its parent's branch merges into that branch, not trunk. This read is a preflight check, not a lock. Every merge operation needs a server-enforced expected-head precondition. Run the GitHub merge immediately after the matching preflight with `gh pr merge "$pr" --squash --match-head-commit "$landing_sha" --repo "$base_repo"`. GitHub has no server-enforced expected-base precondition, so keep watching `baseRefName` and verify the merged base in step 8 instead of inventing a guard or disabling the GitHub flow. Origin has no documented expected-head option. Use an Origin merge only when the active server reports that guard. Otherwise fall back to a resolvable forge that supplies it or stop before merging. If requirements are still running and the user asked for merge-when-ready, remember that auto-merge lands whatever head is ready. Use server-side auto-merge only when the repository has a required, SHA-scoped verification check for the independent verdict that becomes unsatisfied on every head update. On GitHub, run `gh pr merge "$pr" --squash --auto --match-head-commit "$landing_sha" --repo "$base_repo"` immediately after the matching preflight, then disarm it on any observed head or base change. When GitHub adds the pull request to a required merge queue instead of merging it, treat a non-null `mergeQueueEntry` as an armed request and keep watching both the head and the base. Without the required verification check, keep the dynamic watch active and run the guarded immediate merge when the pull request becomes ready instead of arming server-side auto-merge. Wait for that PR to merge before preparing the next one. 6. **Do not read GitHub auto-merge or merge-queue state as stack readiness.** `autoMergeRequest` says that GitHub auto-merge was requested for one pull request. `mergeQueueEntry` says that one pull request entered GitHub's native merge queue. One field can be null while the other is non-null. Neither proves that Origin merge-when-ready is armed, that a descendant is queued, that a patch verdict is current, or that the contiguous stack is safe. Confirm both fields and the active forge's state for the current bottom PR. Say that the state is unknown if the active forge cannot report it. -7. **Recompute after every merge.** Fetch trunk through `<base-remote>`, confirm the merged SHA is present, drop the merged PR from the frozen bottom-to-top list, and inspect the new bottom PR's head, `<landing-base-sha>`, checks, and patch ID. The new bottom still records its former parent's exact tip as `<landing-base-sha>`. Do not replace that value with the forge base before step 4 drops the old parent commits. A host may retarget a child automatically, but do not assume it did. Repeat steps 3 through 6 for that one PR. Independent work stays outside this chain and ships on its own. -8. **Watch the current frontier until it merges or fails. Do not mutate the queue around it.** With Origin, use `origin pr view "$pr" --checks --comments` and `origin pr checks "$pr" --watch`, then re-read the current version's head SHA, base branch, auto-merge state, merge-queue state, and the pull request until it reports merged or blocked. With GitHub, run `skills/poteto-mode/scripts/watch-pr/watch-pr --owner "$base_owner" --repo "$base_name" --queued-stack --stack-prs "$bottom"` under the installed plugin only as an event wake. After each wake, poll `gh pr view "$pr" --json headRefOid,baseRefName,state,mergedAt,mergeStateStatus,statusCheckRollup,autoMergeRequest --repo "$base_repo"` and run step 2's GraphQL state query to read `mergeQueueEntry`. Before accepting checks, a queue entry, or a merge, require that `headRefOid` equals `<landing-sha>` and `baseRefName` equals `<trunk>`, the recorded `<landing-base-ref>`. On any head or base change, disable a non-null `autoMergeRequest`, dequeue a non-null `mergeQueueEntry`, stop the current watch, and return through step 4 to step 3. Re-verify a changed patch and re-run CI and mergeability even when the stable patch ID is unchanged. If a PR merged at another head or into another base despite those checks, report a safety failure and do not continue the stack. If the PR becomes mergeable while both `autoMergeRequest` and `mergeQueueEntry` are null, return to step 5 and merge it. Do not run step 7 until `mergedAt` is non-null or `state` is `MERGED`. Hard-fail when `state` is `CLOSED` with no `mergedAt`, or when a required check reaches any terminal, non-passing conclusion, regardless of whether auto-merge or a merge-queue entry is pending. Disable or dequeue every pending request before diagnosis. `UNSTABLE` is not a failure by itself. Keep watching it while checks are pending, and judge completed checks by whether they are required. When `mergeStateStatus` is `DIRTY`, or when `BEHIND` blocks landing, disable or dequeue every pending request and return to step 4's guarded rebase and step 3's re-verification. Stop for conflicts you cannot resolve. `BLOCKED` while checks are still pending, auto-merge is armed, or a merge-queue entry exists is not failure. Do not use Babysit's queued `WAITING` or `merge-queue` stop condition here. Hold the watch under `/loop` in dynamic mode. Report each merge and the new ceiling. If the queue stalls, diagnose before mutating. -9. **Stop at the ceiling.** When the verified run is merged, report what landed, what the next unverified PR is, and what verifying it would take. Extending the run is a new pass through step 1. +7. **Recompute after every merge.** Fetch trunk through `<base-remote>`, confirm the merged SHA is present, drop the merged PR from the frozen bottom-to-top list, and inspect the new bottom PR's head, `<landing-base-sha>`, checks, and patch ID. The new bottom still records its former parent's exact tip as `<landing-base-sha>`. Do not replace that value with the forge base before step 4 drops the old parent commits. A host may retarget a child automatically, but do not assume it did. A child base that moves to trunk when its parent merges is expected, not damage to repair. The verdict covers the patch, not its commits, so step 3's patch-ID rule decides whether it survives the move. Repeat steps 3 through 6 for that one PR. Independent work stays outside this chain and ships on its own. +8. **Watch the current frontier until it merges or fails. Do not mutate the stack around it,** because a merge lands whatever its branch and base hold at that moment, verdict or not. With Origin, use `origin pr view "$pr" --checks --comments` and `origin pr checks "$pr" --watch`, then re-read the current version's head SHA, base branch, auto-merge state, merge-queue state, and the pull request until it reports merged or blocked. With GitHub, run `skills/poteto-mode/scripts/watch-pr/watch-pr --owner "$base_owner" --repo "$base_name" --queued-stack --stack-prs "$bottom"` under the installed plugin only as an event wake. After each wake, poll `gh pr view "$pr" --json headRefOid,baseRefName,state,mergedAt,mergeStateStatus,statusCheckRollup,autoMergeRequest --repo "$base_repo"` and run step 2's GraphQL state query to read `mergeQueueEntry`. Before accepting checks, a queue entry, or a merge, require that `headRefOid` equals `<landing-sha>` and `baseRefName` equals `<trunk>`, the recorded `<landing-base-ref>`. On any head or base change, disable a non-null `autoMergeRequest`, dequeue a non-null `mergeQueueEntry`, stop the current watch, and return through step 4 to step 3. Re-verify a changed patch and re-run CI and mergeability even when the stable patch ID is unchanged. If a PR merged at another head or into another base despite those checks, report a safety failure and do not continue the stack. If the PR becomes mergeable while both `autoMergeRequest` and `mergeQueueEntry` are null, return to step 5 and merge it. A merge command can report success when a merge queue only queued the PR, and a requested auto-merge is not a merge. Do not run step 7 until `mergedAt` is non-null or `state` is `MERGED`. Hard-fail when `state` is `CLOSED` with no `mergedAt`, or when a required check reaches any terminal, non-passing conclusion, regardless of whether auto-merge or a merge-queue entry is pending. Disable or dequeue every pending request before diagnosis. `UNSTABLE` is not a failure by itself. Keep watching it while checks are pending, and judge completed checks by whether they are required. When `mergeStateStatus` is `DIRTY`, or when `BEHIND` blocks landing, disable or dequeue every pending request and return to step 4's guarded rebase and step 3's re-verification. Stop for conflicts you cannot resolve. `BLOCKED` while checks are still pending, auto-merge is armed, or a merge-queue entry exists is not failure. Do not use Babysit's queued `WAITING` or `merge-queue` stop condition here. Hold the watch under `/loop` in dynamic mode. Report each merge and the new ceiling. If the queue stalls, diagnose before mutating. +9. **Stop at the ceiling.** When the verified run is merged, report what landed, what the next unverified PR is, and what verifying it would take. Extending the run takes new verdicts, so it is a new pass through step 1. -**Reply:** the verified run and its ceiling, each PR's verdict and who produced it, what you armed and how you confirmed it, what landed, and what the next gap needs. +**Reply:** the verified run and its ceiling, each PR's verdict and who produced it, what you armed, what landed and how you confirmed it, and what the next gap needs. diff --git a/plugins/pstack/skills/poteto-mode/playbooks/trace-forensics.md b/plugins/pstack/skills/poteto-mode/playbooks/trace-forensics.md index ee54a239..f77dc489 100644 --- a/plugins/pstack/skills/poteto-mode/playbooks/trace-forensics.md +++ b/plugins/pstack/skills/poteto-mode/playbooks/trace-forensics.md @@ -4,7 +4,7 @@ Distinct from **Runtime forensics**, which instruments the live process. Here the capture already exists. The artifact is a fixed dataset, read it, don't re-run it. Keep tooling generic: a DevTools or trace parser for cpuprofile and `.json.gz`, a text editor for a spindump, your heap tooling for a heapsnapshot. -1. Identify the format and load it with the right tool. Parse large artifacts in a subagent (the **principle-guard-the-context-window** skill) and keep the reduced finding in the main thread. +1. Identify the format and load it with the right tool. Parse large artifacts in a subagent (the **principle-guard-the-context-window** skill) and keep the reduced finding in the main thread. Pick each subagent's effort for the job, per the Subagents section. 2. Transform the raw artifact into a form you can query. Dump the trace or heap snapshot into sqlite, one row per sample, frame, or node. Reach the queryable shape before you read. 3. Narrow to the cause. Query for the frames that hold the most time and walk the call tree to the hot path. For a leak, follow the retainer chain from the leaked object to a GC root. For a spindump, find the thread stuck on-CPU or blocked and its wait reason. 4. Attribute to source. Map the hot frame to file, symbol, and line via the artifact's own symbols. A frame with no source mapping is not yet a diagnosis. Resolve the symbols, or say plainly the artifact does not carry them. diff --git a/plugins/pstack/skills/poteto-mode/playbooks/worktree-cleanup.md b/plugins/pstack/skills/poteto-mode/playbooks/worktree-cleanup.md index f4ff433d..65f5dc55 100644 --- a/plugins/pstack/skills/poteto-mode/playbooks/worktree-cleanup.md +++ b/plugins/pstack/skills/poteto-mode/playbooks/worktree-cleanup.md @@ -2,7 +2,7 @@ 1. Snapshot and audit. Record `df -h /`, then run `skills/poteto-mode/scripts/worktree-audit.sh [-b <pr-base-ref>] [repo-path]` under the installed plugin (principle-build-the-lever). Without `-b` it checks `MERGED` against `origin/HEAD`, so pass `-b origin/<branch>` when PRs target another branch. It reads paths from `git worktree list`, never hand-typed, since a hand-typed `myrepo-worktrees/x` misses one that lives at `.worktrees/myrepo/x` (principle-encode-lessons-in-structure). It classifies each worktree by size, age, merge state, uncommitted work, PR state, and the newest chat that touched it, then suggests a bucket. The transcript scan is slow, so background it. 2. The bucket is advice, not permission. The pinned and active chats are the real artifact (principle-prove-it-works). Get that set from the user or your own session list and cross-check every candidate. The lever has marked `safe` a worktree the user had pinned, so the pinned set wins. -3. Verify usage before deleting. For every `verify-recent-chat` row, or anything you doubt, fan subagents out to read the transcripts and report whether the chat is pinned or ongoing and which worktrees it touches (principle-guard-the-context-window, transcripts are bulk). A pinned chat spawns arena and repro trees into sibling worktrees via background subagents, and those are in use even when their names never hit the session list. +3. Verify usage before deleting. For every `verify-recent-chat` row, or anything you doubt, fan subagents out to read the transcripts and report whether the chat is pinned or ongoing and which worktrees it touches (principle-guard-the-context-window, transcripts are bulk). A pinned chat spawns arena and repro trees into sibling worktrees via background subagents, and those are in use even when their names never hit the session list. Pick each subagent's effort for the job, per the Subagents section. 4. Pause on irreversible loss. `wip:N` is N tracked uncommitted edits. Show the diff and get a decision first, since removing a clean worktree is recoverable from its branch but uncommitted work is gone. `scratch:N` is untracked throwaway, safe to drop, but name the files. Per Autonomy, clean and merged and not-in-use proceeds. `wip` and in-use pause. 5. Prune the confirmed set. Per path, `git worktree remove --force <path>`. If the dir survives on ignored build artifacts, `rm -rf` it, then `git worktree prune`. Branch refs survive, so no commits are lost. Confirm with `df -h /` and re-list. 6. Simulators and other reclaimers. Simulators are usually the next-biggest win. `xcrun simctl --set testing delete all` (XCTestDevices clones), `xcrun simctl delete unavailable`, and `xcrun simctl runtime list` then `runtime delete <id>` for old runtimes. More when needed: Xcode `DerivedData` and `iOS DeviceSupport`, your editor's application-support caches, `~/.claude/shell-snapshots/` and old `~/.claude/projects/` transcripts, keeping the recent ones the **recall** skill and Session pickup read, package caches (pnpm, uv, brew, yarn). Clear only caches the user has not said to keep. diff --git a/plugins/pstack/skills/poteto-mode/references/codex-tools.md b/plugins/pstack/skills/poteto-mode/references/codex-tools.md index 33bf092c..c12a130e 100644 --- a/plugins/pstack/skills/poteto-mode/references/codex-tools.md +++ b/plugins/pstack/skills/poteto-mode/references/codex-tools.md @@ -92,7 +92,7 @@ Some triggers name skills that ship with Claude Code, not pstack. They do not ex ## Vendored scripts -`skills/poteto-mode/scripts/` ships the `watch-pr` PR watcher, the `orch` store CLI, `worktree-audit.sh`, and `runner/pstack-runner`. They are plain bun and bash, so they run the same on Codex; invoke them through `shell`. The external runner additionally needs the assigned `claude`, `codex`, or `grok` executable already authenticated. It rejects a Codex provider when Codex is the parent because that lane belongs on native `spawn_agent`. The other scripts need `bun`, `gh`, (for stack work) `gt`, and (for `worktree-audit.sh`) `jq` and `rg`. `worktree-audit.sh` reads Claude Code transcripts under `~/.claude/projects/`; point it at your runtime's transcript directory instead when you run it elsewhere. +`skills/poteto-mode/scripts/` ships the `watch-pr` PR watcher, the `orch` store CLI, `worktree-audit.sh`, and `runner/pstack-runner`. They are plain bun and bash, so they run the same on Codex; invoke them through `shell`. The external runner additionally needs the assigned `claude`, `codex`, or `grok` executable already authenticated. It rejects a Codex provider when Codex is the parent because that lane belongs on native `spawn_agent`. The other scripts need `bun`, `gh` (or Origin's `origin` CLI where the playbook resolves it), and (for `worktree-audit.sh`) `jq` and `rg`. `worktree-audit.sh` reads Claude Code transcripts under `~/.claude/projects/`; point it at your runtime's transcript directory instead when you run it elsewhere. ## Instructions file diff --git a/plugins/pstack/skills/poteto-mode/references/model-catalog.md b/plugins/pstack/skills/poteto-mode/references/model-catalog.md index 9fa3f3fd..da327135 100644 --- a/plugins/pstack/skills/poteto-mode/references/model-catalog.md +++ b/plugins/pstack/skills/poteto-mode/references/model-catalog.md @@ -12,9 +12,38 @@ Model and provider facts for [`provider-dispatch.md`](provider-dispatch.md), whi | sol | GPT-6.1 Sol | codex | gpt-6.1-sol | max | low medium high xhigh max ultra | - | default | gpt-5.6-sol-max | | grok | Grok 4.7 | grok | grok-4.7 | xhigh | low medium high xhigh max | - | default | grok-4.7-xhigh-fast | | opus | Opus | claude | opus | max | low medium high xhigh max | `pstack-opus-<effort>` | default | opus | +| luna | GPT-6 Luna | codex | gpt-6-luna | max | low medium high xhigh max | - | optional | - | +| sonnet | Sonnet | claude | sonnet | max | low medium high xhigh max | `pstack-sonnet-<effort>` | optional | - | +| opus-5-5 | Opus 5.5 | claude | claude-opus-5-5 | max | low medium high xhigh max | `pstack-opus-5-5-<effort>` | optional | claude-opus-5-5-xhigh | +| swe-2 | SWE-2 | devin | swe-2 | max | medium high max | - | optional | - | +| swe-1.6 | SWE-1.6 | devin | swe-1.6 | default | default | - | optional | - | +| deepseek-v4.1-flash-free | DeepSeek V4.1 Flash (OpenCode Zen) | opencode | opencode/deepseek-v4.1-flash-free | max | low high max | - | optional | - | +| deepseek-flash | DeepSeek Flash (dsh) | dsh | deepseek-flash | max | low high max | - | optional | - | +| deepseek-v4-pro | DeepSeek V4 Pro (dsh) | dsh | deepseek-v4-pro | max | low high max | - | optional | - | +| amp-deepseek-v4.1-flash | DeepSeek V4.1 Flash (Amp) | amp | deepseek-v4.1-flash | default | default | - | optional | - | +| gemini-3.8-flash | Gemini 3.8 Flash (Antigravity) | agy | gemini-3.8-flash | high | low medium high | - | optional | - | +| composer-2.5 | Composer 2.5 (Cursor) | cursor | composer-2.5 | default | default | - | optional | - | +| gemini-3.7-flash | Gemini 3.7 Flash (Cursor) | cursor | gemini-3.7-flash | high | low medium high | - | optional | - | A family is `default` when a role default names it and `optional` otherwise. A Claude-native agent exists for each selectable effort of a Claude family. +Effort in the model ID: for these families the effort selects the CLI model, and argv carries no effort flag. + +| Family | Effort | CLI model | +| --- | --- | --- | +| swe-2 | medium | `swe-2-medium` | +| swe-2 | high | `swe-2-high` | +| swe-2 | max | `swe-2-max` | +| swe-1.6 | default | `swe-1-6` | +| amp-deepseek-v4.1-flash | default | `deepseek-v4.1-flash` | +| gemini-3.8-flash | low | `gemini-3.8-flash-low` | +| gemini-3.8-flash | medium | `gemini-3.8-flash-medium` | +| gemini-3.8-flash | high | `gemini-3.8-flash-high` | +| composer-2.5 | default | `composer-2.5` | +| gemini-3.7-flash | low | `gemini-3.7-flash-low` | +| gemini-3.7-flash | medium | `gemini-3.7-flash-medium` | +| gemini-3.7-flash | high | `gemini-3.7-flash-high` | + ## Kept predecessors and normalization | Provider | Kept model | Family | Current model | Selectable efforts | @@ -28,14 +57,23 @@ Rolling aliases: | --- | --- | --- | | fable | `fable` | `claude-fable-<revision>`, where the revision contains only digits and hyphens | | opus | `opus` | `claude-opus-<revision>`, where the revision contains only digits and hyphens | +| sonnet | `sonnet` | `claude-sonnet-<revision>`, where the revision contains only digits and hyphens | ## Routes -| Provider | Claude Code parent | Codex parent | Prompt | Model proof | Child environment | Host prerequisites | -| --- | --- | --- | --- | --- | --- | --- | -| claude | native `Agent` | external runner | stdin | provider report | read-only: inherited; isolated-write: inherited without `GIT_*` | - | -| codex | external runner | native `spawn_agent` | stdin | pinned argv | inherited | - | -| grok | external runner | external runner | file | provider report | allowlist | Linux: Landlock and `bwrap` | +| Provider | Claude Code parent | Codex parent | Access modes | Prompt | Model proof | Child environment | Host prerequisites | +| --- | --- | --- | --- | --- | --- | --- | --- | +| claude | native `Agent` | external runner | `read-only`, `isolated-write` | stdin | provider report | read-only: inherited; isolated-write: inherited without `GIT_*` | - | +| codex | external runner | native `spawn_agent` | `read-only`, `isolated-write`, `isolated-write-full-access` | stdin | pinned argv | inherited | - | +| grok | external runner | external runner | `read-only`, `isolated-write` | file | provider report | allowlist | Linux: Landlock and `bwrap` | +| devin | external runner | external runner | `read-only` | file | provider report | allowlist | - | +| opencode | external runner | external runner | `read-only` | stdin | pinned argv | allowlist | - | +| dsh | external runner | external runner | `read-only` | stdin | pinned argv | allowlist | dsh 0.2.0-rc.2 exactly | +| amp | external runner | external runner | `read-only` | stdin | pinned argv | allowlist | - | +| agy | external runner | external runner | `read-only` | stdin | pinned argv | allowlist | - | +| cursor | external runner | external runner | `read-only` | stdin | pinned argv | allowlist | - | + +The runner refuses an access mode outside a provider's Access modes cell before launch. `isolated-write-full-access` is a writer capability, not a `--mode` value: the runner takes it as `--mode isolated-write --writer-sandbox full-access`, which a parent passes only when the model sheet opts in, as [`provider-dispatch.md`](provider-dispatch.md#codex-writer-sandbox) says. ## Provider notes @@ -63,6 +101,112 @@ Rolling aliases: - The preflight passes when `grok models` reports a login (`logged in`, `You are using XAI_API_KEY.`, or `You are authenticated via deployment key.`) and lists the lane's model on a line under `Available models:`. A model named only above that header, such as on the `Default model:` line, or output without the header, is `unavailable-model`/69. - The authentication preflight has one bounded retry. If the first `grok models` result would be classified as unauthenticated, the runner waits five seconds and tries the same preflight once more. A second failure is terminal. The delay and second attempt share the runner's absolute deadline and cancellation latch, and the receipt keeps evidence from both attempts. Model execution is never retried. +### devin: Devin CLI + +- Executable: `devin`. +- Host prerequisites: none. +- State paths: `~/.local/share/devin/` (the login, which the CLI reads) and the config, telemetry and log files it writes under the home. +- Child environment: only `PATH` `HOME` `USER` `LOGNAME` `SHELL` `TMPDIR` `LANG` `TERM` `COLORTERM` `TZ` `HTTP_PROXY` `HTTPS_PROXY` `NO_PROXY` `ALL_PROXY` `http_proxy` `https_proxy` `no_proxy` `all_proxy` `SSL_CERT_FILE` `SSL_CERT_DIR` `NODE_EXTRA_CA_CERTS`, and variables starting with `LC_`, `XDG_`. +- Effort is part of the model ID. The runner passes the CLI model ID that the effort table gives for the requested effort. An unregistered Devin model must be a CLI model ID with the `default` effort. +- Read-only only. A lane runs `--permission-mode auto` without `--sandbox`, so Devin's approval policy, not an OS sandbox, rejects edits and writing shell commands, as Claude's `plan` mode does. With `--sandbox`, approved shell commands write in the cwd while the edit tools are rejected, so no Devin mode is a bounded writer. The runner refuses an `isolated-write` lane before launch. +- A Devin lane runs in any directory, and the configuration it finds there applies as it is. The lane passes `--respect-workspace-trust false`, so Devin loads the `.devin/config.json` hooks and permissions and the `.devin/mcp_config.json` MCP servers of a `.devin/` directory in the cwd or any parent up to the git root, or up to `/` outside a repository. So a lane whose cwd is under the home directory and outside any git repository also loads the home directory's own `~/.devin/`. No key turns them off. +- Each lane gets a run-owned `--config` file in its private `<receipt>.provider/` directory (0700). It turns off subagents, every host rule source (such as `CLAUDE.md`) and MCP tools, and denies web search and the fetch tools. Devin still loads its own skills. Network isolation is not established: shell network access and the fetch denials were not measured. +- The preflight passes when `devin auth status` exits 0 and starts with `Logged in (via Devin).`. It also exits 0 when logged out, so output starting with `Not logged in.` is `unauthenticated`/77, and any other output is `child-failed`/70. +- The reply, session, token usage and model proof come from the `--export` ATIF file at `<receipt>.provider/export.json`, never from stdout. The proof is the last step's `extra.generation_model`, which must equal the CLI model ID. An empty final message, as after a rejected tool call, is `child-failed`/70. An interrupted response, or exit 143 after SIGTERM, is `cancelled`/130. `Error: Unknown model:` is `unavailable-model`/69. + +### opencode: OpenCode + +- Executable: `opencode`. +- Host prerequisites: none. +- State paths: `~/.local/share/opencode/` (`auth.json`, the stored logins, which the CLI reads; the session database, logs and tool output it writes); `~/.local/state/opencode/`, `$XDG_CACHE_HOME/opencode/` and `$TMPDIR/opencode/`, which it writes. +- Child environment: only `PATH` `HOME` `USER` `LOGNAME` `SHELL` `TMPDIR` `LANG` `TERM` `COLORTERM` `TZ` `HTTP_PROXY` `HTTPS_PROXY` `NO_PROXY` `ALL_PROXY` `http_proxy` `https_proxy` `no_proxy` `all_proxy` `SSL_CERT_FILE` `SSL_CERT_DIR` `NODE_EXTRA_CA_CERTS`, and variables starting with `LC_`, `XDG_`; sets `XDG_CONFIG_HOME=<receipt>.provider`, `OPENCODE_TEST_HOME=<receipt>.provider`, `OPENCODE_CONFIG=<receipt>.provider/opencode.json`, `OPENCODE_DISABLE_PROJECT_CONFIG=1`, `OPENCODE_DISABLE_CLAUDE_CODE=1`, `OPENCODE_DISABLE_EXTERNAL_SKILLS=1`, `OPENCODE_DISABLE_AUTOUPDATE=1`. +- The model is `provider/model`, such as `opencode/deepseek-v4.1-flash-free`. The effort is `--variant <effort>`, omitted for `default`. OpenCode accepts an unknown variant silently, so only a registered family's efforts are checked. +- Read-only only, measured on OpenCode 1.19.1. Each lane runs the `build` agent (`--agent build`, because a config's `default_agent` would otherwise pick another agent) with a run-owned config whose `"*": "deny"` permission removes every tool except `read`, `glob`, `grep`, `list` and `todowrite` from the model's tool list. OpenCode's permission policy enforces this, not an OS sandbox. With `external_directory` denied, a shell write still escaped the cwd, so no OpenCode mode is a bounded writer, and the runner refuses an `isolated-write` lane before launch. +- The runner writes that config to `<receipt>.provider/opencode.json` (0600, in a 0700 directory), points `OPENCODE_CONFIG` at it, and sets `XDG_CONFIG_HOME` and `OPENCODE_TEST_HOME` to the directory, so OpenCode reads the run-owned directory in place of the host's `~/.config/opencode/` and `~/.opencode/`. OpenCode 1.19.1 merges `~/.opencode/opencode.json` after `OPENCODE_CONFIG` and imports `~/.opencode/plugins/*.js` even with `--pure`, and it finds that home `.opencode/` through `OPENCODE_TEST_HOME` when set. Measured with a fresh `HOME` whose `.opencode/opencode.json` allowed `bash`, `edit` and `write` and whose `.opencode/plugins/` held a canary plugin: without `OPENCODE_TEST_HOME`, `opencode debug agent build` listed those tools and the canary ran, and neither `OPENCODE_CONFIG_CONTENT` nor `OPENCODE_CONFIG_DIR` changed that; with it, `debug agent build` and `debug config` showed only the lockdown, the canary did not run, and a lane asked to write and edit files changed nothing. Stored logins still come from the real home. `OPENCODE_DISABLE_PROJECT_CONFIG=1` stops the repository's `opencode.json`, `opencode.jsonc`, `.opencode/` agents and commands, and `AGENTS.md` from loading. `OPENCODE_DISABLE_CLAUDE_CODE=1` stops `~/.claude/CLAUDE.md` and `~/.claude/skills`, and `OPENCODE_DISABLE_EXTERNAL_SKILLS=1` stops `~/.agents/skills`. `OPENCODE_DISABLE_AUTOUPDATE=1` and `--pure` turn off updates and external plugins. The built-in `customize-opencode` skill remains, but the `skill` tool is denied. +- A managed config (`/Library/Application Support/opencode/` on macOS, `/etc/opencode/` on Linux) merges after the run-owned config and overrides it. Measured on OpenCode 1.19.1 with layers planted through the binary's `OPENCODE_TEST_MANAGED_CONFIG_DIR` redirect: a managed `mcp` server started in a real lane, a managed `lsp` command ran when the model called `read`, a managed `provider` `baseURL` rerouted the lane, and a managed `agent.build` prompt replaced the build agent's prompt. `--agent build` and `-m` took precedence over a managed `default_agent` and `agent.build.model`, and `--pure` kept a managed `plugin` from loading. On macOS, `/Library/Managed Preferences/<user>/ai.opencode.managed.plist` and `/Library/Managed Preferences/ai.opencode.managed.plist` merge last. The plist layer is from OpenCode's source and was not measured. +- A stored `wellknown` login (from `opencode providers login <url>`) or an active console organization (from `opencode console login`) makes every OpenCode process, including each lane, fetch remote config and merge it into the lane's configuration. Measured with a fake well-known login and a local server, whose config added an MCP server and a `default_agent`. The console organization fetch is from OpenCode's source and was not measured. +- OpenCode 1.19.1 imports a repository's `.opencode/plugins/*.js` from the cwd or a parent up to the git root, even with `--pure` and `OPENCODE_DISABLE_PROJECT_CONFIG=1`. The import runs each module's top-level code; only the plugin's initialization is skipped. +- The child environment drops every other `OPENCODE_*` variable, because `OPENCODE_PERMISSION` turns `bash` and `edit` back on and others replace the config or the model catalog. It also drops provider keys such as `OPENROUTER_API_KEY`. A lane authenticates through OpenCode's stored logins (`opencode auth login`, kept in `~/.local/share/opencode/auth.json`), or uses a free OpenCode Zen model, which needs no login. +- The preflight runs `opencode models <provider>`, where `<provider>` is the model's part before the first `/`. It passes when it exits 0 and one line equals the model. `Error: Provider not found:` with a nonzero exit is `unauthenticated`/77. Exit 0 without the model line is `unavailable-model`/69, and so is a model with no `/`. Any other output is `child-failed`/70. +- The lane passes `--title pstack-lane`, because without a title OpenCode calls a second model to name the session. The prompt goes on stdin. +- The reply, session ID, token usage and cost come from the `--format json` event stream on stdout. A lane is complete only when the last `step_finish` has reason `stop` and a `text` event follows the last `step_start`; the reply is those texts in order. Usage and cost are summed over every `step_finish`. An `error` event is `child-failed`/70. For an unknown model that event says only `UnknownError`, and `--print-logs --log-level ERROR` puts its `ProviderModelNotFoundError` cause on stderr, which the receipt keeps. A final `step_finish` with reason `tool-calls` is a permission prompt that headless OpenCode auto-rejected at exit 0, and is `child-failed`/70. Exit 130 or 143 after a signal is `cancelled`/130. +- No event names the served model, so the model proof is the pinned argv (`pinned-argv`). +- Sessions stay in the local session database. The config turns sharing off, and OpenCode 1.19.1 keeps no server-side session record. + +### dsh: DeepSeek Harness + +- Executable: `dsh`. +- Host prerequisites: dsh 0.2.0-rc.2 exactly. +- State paths: `<receipt>.provider/`, the lane's own `DSH_HOME`: the sessions (`sessions/<cwd-slug>/<session-id>/`), projection cache (`storages/session_projcache/`), `profiles/headless/` and `.anonymous-user-id` that dsh writes on every run; The stored key `<DSH_HOME, else ~/.dsh>/.credentials.yaml`, which the lane reads in place and does not copy. +- Child environment: only `PATH` `HOME` `USER` `LOGNAME` `SHELL` `TMPDIR` `LANG` `TERM` `COLORTERM` `TZ` `HTTP_PROXY` `HTTPS_PROXY` `NO_PROXY` `ALL_PROXY` `http_proxy` `https_proxy` `no_proxy` `all_proxy` `SSL_CERT_FILE` `SSL_CERT_DIR` `NODE_EXTRA_CA_CERTS` `DEEPSEEK_API_KEY`, and variables starting with `LC_`, `XDG_`; sets `DSH_HOME=<receipt>.provider`. +- Measured with dsh 0.2.0-rc.2, a developer preview. The runner checks `dsh --version` before the preflight and refuses any other version as `unavailable-cli`/69. Recheck on an upgrade: the config row IDs the patch names (dsh silently ignores a patch row it does not have, so a renamed row would drop that part of the lockdown), the SIGTERM exit 0 with a partial `final`, the hardcoded `/tmp` writable root, the session-log upload being on by default, that dsh reads only the cwd's `.env` and skips a directory there, the transport retry row, and the `credentials` row's `path` config. +- Model and effort are config, not flags. Each lane gets a run-owned `--patch` file at `<receipt>.provider/patch.yml` (directory 0700, file 0600) that pins `agent-default-model` to provider `deepseek-official`, the lane's model and, unless the effort is `default`, its `reasoningEffort`. dsh offers efforts `off`, `low`, `high` and `max`, and fails `medium` locally. +- Each lane runs with `DSH_HOME` set to its `<receipt>.provider/` directory, and the allowlist drops every inherited `DSH_*` name. dsh creates the `headless` profile there on first use with no network or `pnpm` install (about 70 ms), so no user layer (`profiles/headless/cordis.patch.yml`) and no home `.env` from the operator's dsh home loads. Measured with `--dump-config`: a planted user layer that redirected `llm-deepseek.config.baseURL`, moved `fs-sandbox.config.cwd`, picked another `agent-default-model` provider and disabled `bash-sandbox` changed nothing, and the composed tree was the default rows plus the lane's patch. The patch's `credentials` row points the credential store at `<the parent's DSH_HOME, else ~/.dsh>/.credentials.yaml`, read in place. A key kept only in the operator's `<DSH_HOME>/.env` no longer authenticates; store it or export `DEEPSEEK_API_KEY`. +- Read-only only. The patch pins `sandbox-policy` to `read-only` with the lane's cwd as the workspace root and `approval` to `ask`, which overrides `DSH_PERMISSION_MODE`. The other sandbox rows keep dsh's defaults, including the bash tool's own timeout; the runner adds no deadline. Seatbelt enforces it on macOS for bash and the file tools. Headless has no approval channel, so an escalation request fails closed. Bash keeps network access, and the `read` tool is not confined to the cwd. The runner refuses an `isolated-write` lane before launch. +- The patch turns off host and project rules (`AGENTS.md`, `CLAUDE.md` and their local variants), every skill root, the web, subagent and workflow tools, the session-title model call, the plugin inventory and the `dsh_session_log` upload. The goal, job, todo, plan-mode and `read_image` tools still load. dsh loads no project patch, hook or MCP file. +- A dsh lane runs in any cwd, and a `.env` file there applies as it is. dsh loads the cwd's `.env` at boot, before any config row applies, and no setting turns it off. dsh rejects only bootstrap names there (`DSH_*`, `DEEPSEEK_BASE_URL`, `PATH`, `NODE_OPTIONS`, `XDG_*`, `DYLD_*`, proxies), and every other value reaches the bash tool: a repository `.env` that sets `GIT_CONFIG_PARAMETERS` with `core.fsmonitor` makes a plain `git status` run the repository's script. dsh reads only the cwd's `.env`, not a parent's, and warns and skips a directory named `.env`, such as a Python virtual environment. +- dsh has no status command. The preflight is a turn on the lane's `DSH_HOME` with the lane's patch and then the shipped `providers/dsh-preflight.yml` patch, which points inference at a closed loopback port and turns off retries. The lane's patch already turns off the title call and the session-log upload. `TRANSPORT` passes with the fixed evidence `credential present; not validated`: it proves only that a key is stored or exported, not that the key works. `MISSING_CREDENTIAL` is `unauthenticated`/77, and any other output is `child-failed`/70. +- The reply is the `--json` stream's `final` text, accepted only after a `status` `turn_end` with `reason.kind` `completed`. SIGTERM exits 0 with a partial `final` and no `turn_end`, so that is `malformed-output`/65, never complete. Exit 130 after SIGINT is `cancelled`/130. A `turn_end` error is `child-failed`/70 with its code and message, except that DeepSeek's `The supported API model names are ...` refusal is `unavailable-model`/69. Usage sums the `step_end` usage; the session ID is the `session` event's. No cost is reported. +- No served model is reported, so the model proof is the pinned patch (`pinned-argv`). +- Server side: requests still carry the `x-deepseek-harness-user-id` and `x-deepseek-harness-session-id` headers. Whether DeepSeek keeps request content is not measured. + +### amp: Amp CLI + +- Executable: `amp`. +- Host prerequisites: none. +- State paths: `~/.local/share/amp/` (the login in `secrets.json`, accounts, session and recent threads) and `~/.cache/amp/` (logs, downloaded plugins and skills). +- Child environment: only `PATH` `HOME` `USER` `LOGNAME` `SHELL` `TMPDIR` `LANG` `TERM` `COLORTERM` `TZ` `HTTP_PROXY` `HTTPS_PROXY` `NO_PROXY` `ALL_PROXY` `http_proxy` `https_proxy` `no_proxy` `all_proxy` `SSL_CERT_FILE` `SSL_CERT_DIR` `NODE_EXTRA_CA_CERTS` `AMP_API_KEY`, and variables starting with `LC_`, `XDG_`; sets `AMP_IGNORE_GUIDANCE_FILES=*`. +- Effort is part of the model ID. The model is an official Amp agent mode, such as `deepseek-v4.1-flash`, from Amp's own `official-modes` plugin, which Amp serves to every user. Where a mode has effort variants, the variant is part of its name (for example `gpt-6-astra-low` to `gpt-6-astra-max`). An unregistered Amp model must be a mode name with the `default` effort. An unknown mode exits 1 with a generic error and ends `child-failed`/70. +- Read-only only. Amp has no OS sandbox and auto-approves its `ask` rules in execute mode, and a shell write escaped the cwd whenever the shell was allowed, so no Amp mode is a bounded writer. The runner refuses an `isolated-write` lane before launch. +- A read-only lane is a run-owned `--settings-file` in its private `<receipt>.provider/` directory (0700). It enables only the `Read` tool and rejects every other tool, which turns off the shell, edits, subagents, web tools, code execution, thread tools and `manage_amp`. It also turns off MCP servers, Claude Code and `~/.agents` skills, updates and notifications. The lane sets `AMP_IGNORE_GUIDANCE_FILES=*`, so `AGENTS.md` in the cwd and its parents does not load. `Read` can read any path on the host. Measured with Amp CLI 0.0.1792094445-g3475f5 and 0.0.1792123242-g03edc7. +- Amp's own `official-modes` plugin supplies the pinned modes and runs its code in every lane. Amp serves and updates it with no version pin: its cache hash changed from `a1f1ef61c4dd011d` to `cb61cf1a7f4be96c` between 2026-10-15 and 2026-10-16. +- System plugins in `$XDG_CONFIG_HOME/amp/plugins` or `~/.config/amp/plugins` load in every lane, and the lane keeps an inherited `XDG_CONFIG_HOME`. A plugin is Bun code that runs as the user, and Amp has no production switch that turns plugins off (`AMP_DISABLE_PLUGINS` is ignored outside development builds). Measured with Amp CLI 0.0.1792123242-g03edc7: a system plugin ran at import and in the `session.start`, `agent.start` and `agent.end` hooks. One that called `configuration.update` at startup rewrote the lane's settings file to enable every tool with an allow-all rule, after which the model had 29 tools and ran a shell write in the cwd. One that registered the key `deepseek-v4.1-flash` replaced the official mode's prompt and tools, while the argv still named that mode. +- Personal and workspace plugins come from ampcode.com, load in every lane and use the same plugin API as system plugins. The measured account had none. +- A machine `managed-settings.json` (`/Library/Application Support/ampcode/` on macOS, `/etc/ampcode/` on Linux) and server-managed workspace settings apply above the user settings that `--settings-file` replaces, per Amp's settings docs, so they can change the enabled tools and permissions. Neither was measured: the Mac had no managed file, and the account has no workspace. +- A repository's `.amp/settings.json` in the cwd overrides the run-owned settings, and its `.amp/plugins/*.ts` runs at startup with no trust check. Measured with Amp CLI 0.0.1792094445-g3475f5: a cwd `.amp/settings.json` turned `create_file` and `shell_command` back on, and a `.amp/plugins/` plugin ran. A workspace MCP server in `.amp/` waited for approval and did not start. +- The environment is an allowlist that adds only `AMP_API_KEY`. No other `AMP_` name passes: `AMP_SETTINGS_FILE`, `AMP_EXECUTOR`, `AMP_INITIAL_AGENT_MODE_KEY` and `AMP_URL` would change the settings, where the turn runs, the mode or the service. +- The preflight is `amp usage`, run without a terminal. It passes when it exits 0 and starts with `Signed in as `, and records the fixed evidence `authenticated`, because the output holds the account email and credit balance. Exit 1 with `Invalid or missing API key` is `unauthenticated`/77, and any other output is `child-failed`/70. A logged-out Amp command other than `amp usage` can open a browser login on its own, which is why the runner never starts a turn without a passing preflight. +- The reply, session and token usage come from the stream JSON on stdout: the reply is the `result` event's text, the session is its `session_id` (`T-<uuid>`, also the ampcode.com thread ID), and the usage sums each assistant message's `usage`. The stream has no cost and never names the model, so the proof is the pinned mode in the argv (`pinned-argv`). +- Failures and cancellations can exit 0, so the `result` event decides. `is_error: true` with `User cancelled (SIGINT/SIGTERM)` is `cancelled`/130, and any other `is_error` result is `child-failed`/70 with its error text, for example a Claude mode on a subscription-only account. A rejected tool comes back to the model as an ordinary tool result, and the turn can still complete. +- Each run is an ampcode.com thread holding the prompt, tool inputs and outputs, the hostname and the working directory. The lane passes `--visibility private` and the label `pstack-runner`; every captured thread was private and archived. Turns spend Amp credits, or a linked ChatGPT subscription for GPT modes. + +### agy: Antigravity CLI (`agy`) + +- Executable: `agy`. +- Host prerequisites: none. +- State paths: `~/.gemini/antigravity-cli/`, where the CLI writes conversations, `brain/`, logs, caches and `conversation_summaries.db`; the login is in the macOS keychain. +- Child environment: only `PATH` `HOME` `USER` `LOGNAME` `SHELL` `TMPDIR` `LANG` `TERM` `COLORTERM` `TZ` `HTTP_PROXY` `HTTPS_PROXY` `NO_PROXY` `ALL_PROXY` `http_proxy` `https_proxy` `no_proxy` `all_proxy` `SSL_CERT_FILE` `SSL_CERT_DIR` `NODE_EXTRA_CA_CERTS`, and variables starting with `LC_`, `XDG_`. +- Effort is part of the model ID. The runner passes the CLI model ID that the effort table gives for the requested effort and never passes `--effort`, because a variant ID with a conflicting `--effort` exits 1. A bare `gemini-3.8-flash` runs as `gemini-3.8-flash-high`. Issue #37 asked for Gemini 3.7 Flash, but `agy models` did not list it for the measured account. The runner registers Gemini 3.8 Flash and never maps a 3.7 request to it. +- Read-only only. A lane runs agy's default mode, whose approval policy, not an OS sandbox, denies the write tool and every shell command. `--mode accept-edits` writes only through the edit tool and denies shell, and `--sandbox` blocks shell writes even in the cwd, so no agy mode is a bounded writer. The runner refuses an `isolated-write` lane before launch. +- Headless agy cannot ask for approval. The first tool that needs it is denied, and the turn ends with exit 0, `status: "SUCCESS"` and a `denied_actions` list. Any `response` text is a preamble written before the denied tool, so the runner records a non-empty `denied_actions` as `child-failed`/70 whatever the response says. +- Each lane writes a run-owned agent file, `.agents/agents/pstack-read-only.md`, in its private `<receipt>.provider/` directory (0700, the file 0600), and runs `--add-dir <that directory> --agent pstack-read-only`. The agent declares only `view_file`, no MCP servers and no skills. The lane keeps `view_file` and `manage_task` and loses web search, `read_url_content`, subagents, `schedule`, `send_message` and the browser. If the file is missing, agy exits 1 with `unknown agent`. +- An agy lane runs in any directory, and the configuration it finds there applies as it is. agy runs the hooks and MCP servers in `.agents/`, `.agent/`, `_agents/` and `_agent/` directories in the cwd and each parent up to the git root, or up to `/` outside a repository, with no trust check, and loads their rules and skills. So a lane whose cwd is under the home directory and outside any git repository, or in a repository rooted at the home directory, also loads the home directory's own `~/.agents/` and its siblings. No flag stops them. A repository agent named `pstack-read-only` replaces the run-owned one. +- No flag turns off the operator's own configuration. `~/.gemini/GEMINI.md` loads as a global rule, and host skills and agy plugin hooks under `~/.gemini/` still load in every lane. The run-owned agent limits only the lane's tools. +- The lane passes the base allowlist and nothing else. The login is in the macOS keychain, found through `HOME`, so a run-owned `HOME` would be logged out. `GEMINI_API_KEY` did not authenticate and is not passed. +- The preflight runs `agy models` and passes when it exits 0 and a line's first tab-separated field is the lane's CLI model ID. Exit 0 without that line is `unavailable-model`/69. Exit 1 with `Please sign in` is `unauthenticated`/77. Any other output is `child-failed`/70. +- The prompt goes to stdin as one stream-json line, `{"event":"user","message":{"role":"user","content":<prompt>}}`, with `--input-format stream-json --output-format stream-json`. The reply is the `result` event's `response` without its trailing newline. The session is `conversation_id`, and usage is `usage`, with `thinking_tokens` as reasoning and `cache_read_tokens` as cached input. agy reports no cost. +- agy never reports the served model. `init.model` repeats `--model`, and only an internal log names the resolved model, so the proof is the pinned argv. An unknown model exits 1 with `invalid model selection`, which is `unavailable-model`/69. A `result` error of `interrupted`, after SIGINT or SIGTERM, is `cancelled`/130. + +### cursor: Cursor CLI (`cursor-agent`) + +- Executable: `cursor-agent`. +- Host prerequisites: none. +- State paths: `~/.cursor/`, where the CLI reads `mcp.json`, `hooks.json`, plugins and skills and writes `chats/`, `projects/<slug>/`, `statsig-cache.json` and `agent-cli-state.json`; the stored login is in the macOS keychain; `<receipt>.provider/`, the run-owned `CURSOR_CONFIG_DIR`, where the CLI writes `chats/` and `statsig-cache.json` and rewrites `cli-config.json` at 0644 with an identity block, keeping the run-owned permissions; `~/Library/Caches/cursor-compile-cache/`, `$TMPDIR/cursor-agent-logs-<uid>/` and `~/.npm/_npx/`, which it writes. +- Child environment: only `PATH` `HOME` `USER` `LOGNAME` `SHELL` `TMPDIR` `LANG` `TERM` `COLORTERM` `TZ` `HTTP_PROXY` `HTTPS_PROXY` `NO_PROXY` `ALL_PROXY` `http_proxy` `https_proxy` `no_proxy` `all_proxy` `SSL_CERT_FILE` `SSL_CERT_DIR` `NODE_EXTRA_CA_CERTS` `CURSOR_API_KEY` `CURSOR_AUTH_TOKEN`, and variables starting with `LC_`, `XDG_`; sets `CURSOR_CONFIG_DIR=<receipt>.provider`. +- The executable is `cursor-agent`. Cursor also installs `agent`, and on a host where another CLI's `agent` comes first on PATH (Grok's, for example), `agent status --format json` exits 2 with empty stdout, which looked like an empty Cursor status in #94. +- Effort is part of the model ID. The runner passes the CLI model ID that the effort table gives for the requested effort, such as `gemini-3.7-flash-high`. An unregistered Cursor model must be a CLI model ID with the `default` effort. An unknown model exits 1 with `Cannot use this model: <id>`, which is `unavailable-model`/69. +- Read-only only. A lane runs `-p --output-format stream-json --mode ask --sandbox enabled --trust` with the prompt on stdin. Only `--mode ask --sandbox enabled` refused every edit and shell request, including `ls`; the CLI's permission check enforces it, and the Read tool still works. In agent mode the edit tool wrote outside the cwd under every flag combination, so no Cursor mode is a bounded writer. The runner refuses an `isolated-write` lane before launch. `--trust` is required in a new directory. The lane never passes `--force`, `--yolo` or `--approve-mcps`. +- Each lane sets `CURSOR_CONFIG_DIR` to its private `<receipt>.provider/` directory (0700) and writes `cli-config.json` there (0600) before the preflight. The config denies `Write(**)`, `Shell(*)` and `Mcp(*:*)`, sets `approvalMode: "allowlist"` with no allow rules, enables the sandbox, and turns off web-search auto-approval and the web-fetch domain allowlist. MCP and plugin tools (such as the GitHub, Gmail and Calendar plugins) are still listed to the model, but every call is blocked. Web search and web fetch are refused, because nothing approves them. A Task subagent runs with the same limits. Without the run-owned directory, every `-p` run rewrites `~/.cursor/cli-config.json` with its default model and an identity block. +- A Cursor lane runs in any repository, and the repository's configuration applies as it is. `.cursor/cli.json`, which the CLI reads in every directory from the git root down to the cwd, replaces the run-owned deny list. A measured one that allowed `Shell(*)` let the lane run shell commands, and an invalid one stops the run with exit 1. `AGENTS.md`, `CLAUDE.md`, `.cursor/rules/*.mdc` and `.cursorrules` load as always-applied rules. +- Hooks run unsandboxed: the CLI's command executor runs them with sandbox policy `insecure_none` whatever `--sandbox` is set to (batch-158 containment captures, 2026-10-16). No flag stops them, and no command lists them without running them. Enterprise hooks load from a fixed path that follows neither `CURSOR_CONFIG_DIR` nor `HOME`: `/Library/Application Support/Cursor/hooks.json` on macOS and `/etc/cursor/hooks.json` on Linux. For a team account, a turn fetches the team's hooks from Cursor's server, writes them to `~/.cursor/managed/team_<id>/hooks/` and runs them. User hooks load from `~/.cursor/hooks.json` and `~/.claude/settings.json`. Repository hooks load from `.cursor/hooks.json`, `.claude/settings.json` and `.claude/settings.local.json`. +- No flag turns off the operator's own configuration. Host skills (from `~/.cursor`, `~/.claude`, `~/.codex` and `~/.agents`), account plugins and `~/.cursor/mcp.json` still load. The run-owned deny list limits only what their tools can do. +- The environment is an allowlist that adds only `CURSOR_API_KEY` and `CURSOR_AUTH_TOKEN`, the two logins the CLI's own error names, and sets `CURSOR_CONFIG_DIR`. No other `CURSOR_` name passes: `CURSOR_API_ENDPOINT`, `CURSOR_API_BASE_URL`, `CURSOR_SANDBOX_POLICY_DIR`, `CURSOR_FORCED_SHELL_EGRESS` and the rest change the service, the sandbox or the egress. On the measured macOS host, a `cursor-agent login` run while `CURSOR_API_KEY` was set left a stored login that `status` reported as authenticated but that failed every model call once the variable was gone; signing out and in again with `env -u CURSOR_API_KEY cursor-agent login` fixed it. +- The preflight runs `cursor-agent models`, not `status`: `status` reports a stored login as authenticated while every model call fails with `Authentication required`, and it ignores `CURSOR_API_KEY`. The preflight passes when `models` exits 0, its stdout starts with `Available models`, and a line starts with `<CLI model ID> - `, and records the fixed evidence `model <id> listed`. Exit 0 without that line is `unavailable-model`/69. Exit 1 with `Error: Authentication required` is `unauthenticated`/77. Any other output, including empty output, is `child-failed`/70. +- The `result` event decides. A `success` result with `is_error: false` is complete, and the reply is the last `assistant` event's text, because the `result` field joins every assistant text segment of the turn. `is_error: true` is `child-failed`/70 with its text. A refused tool comes back to the model as a tool result, and the turn can still complete. SIGINT exits 130 and SIGTERM 143 with no `result` event. The session is `session_id`, and usage is `result.usage`, with `cacheReadTokens` as cached input and `cacheWriteTokens` as cache creation. Cursor reports no cost. +- Cursor never reports the served model ID. The init event's `model` is a display label such as `Claude Opus 5.5 300K High`, so the proof is the pinned argv (`pinned-argv`). +- Every turn starts a `worker-server` process in the workspace with the lane's environment, in an empty repository too and whether or not the model reads a file. The worker runs `npx -y typescript-language-server --stdio`, and npx runs a repository's own copy when it has one. In the batch-158 containment captures (Cursor CLI 2026.10.14-0f9edde, 2026-10-16), each of these ran repository code: a `node_modules/.bin/typescript-language-server`, a root `package.json` named `typescript-language-server` with a `bin`, an `.npmrc` `script-shell`, and an `.npmrc` `node-options`. The language server inherits the lane's environment plus `AGENT_CLI_*` and `npm_*` variables. A `package.json` `bin` run leaves a `file:` link to the repository in `~/.npm/_npx/`. +- The worker and the language server stay running in the CLI's process group after the turn and exit after about five idle minutes. An unknown model exits before the worker starts. + ## Role defaults The complete first-run sheet. A sheet that lacks a role uses that role's line here. diff --git a/plugins/pstack/skills/poteto-mode/references/provider-dispatch.md b/plugins/pstack/skills/poteto-mode/references/provider-dispatch.md index 8789ea7d..f7b3f56e 100644 --- a/plugins/pstack/skills/poteto-mode/references/provider-dispatch.md +++ b/plugins/pstack/skills/poteto-mode/references/provider-dispatch.md @@ -10,7 +10,7 @@ pstack model choices are provider-qualified descriptors: The generated [model catalog](model-catalog.md) holds the model facts: the model matrix, kept predecessors and rolling aliases, the route table, provider notes, and role defaults. Match a descriptor's `(provider, model)` there. -The allowed effort universe is exactly `low`, `medium`, `high`, `xhigh`, `max`, `ultra`. A row offers `ultra` only when its CLI's model entry lists that effort; for Codex, that is the model's entry in `codex`'s model list. First-run requested efforts are the Default effort cell of each row. +The allowed effort universe is exactly `low`, `medium`, `high`, `xhigh`, `max`, `ultra`, and `default`, which passes no effort flag. A row offers `default` only when its Selectable efforts cell lists it. A row offers `ultra` only when its CLI's model entry lists that effort; for Codex, that is the model's entry in `codex`'s model list. First-run requested efforts are the Default effort cell of each row. Per spawn, the parent may move a role's effort to another effort that the row lists, under poteto-mode's Match the model to the job rule. A child never changes its assigned effort. A rolling alias, such as `opus`, names a family whose CLI resolves it to the latest available family revision. A runner receipt keeps the requested alias in `model` and the concrete provider-reported revision in `reportedModel`; verification accepts only a revision of the matching family, as the catalog's rolling-alias table gives it. @@ -18,11 +18,11 @@ A descriptor whose `(provider, model)` matches no matrix row, kept predecessor, ## Read-time normalization -Normalize configured descriptors before matching them to the matrix or choosing a route. If a provider-qualified model is a versioned predecessor of a rolling alias in the catalog, replace that model component in memory with the alias. Preserve provider, effort, role, and lane order. Use only the normalized descriptor for native dispatch or runner argv. Never pass the versioned predecessor to Claude. +Match each configured descriptor before choosing a route, in this order: an exact model-matrix row, then a kept predecessor, then a rolling-alias revision. A model that is its own matrix row, such as a registered exact pin, is never normalized. If a provider-qualified model matches no matrix row or kept predecessor and is a versioned predecessor of a rolling alias in the catalog, replace that model component in memory with the alias. Preserve provider, effort, role, and lane order. Use only the normalized descriptor for native dispatch or runner argv. Never pass the versioned predecessor to Claude. This read-time rule makes an older installed sheet use the latest family revision immediately without writing user files. Once per parent run, report that the persisted sheet is stale and that `/setup-pstack` will rewrite it after its normal probes and confirmation. Unknown versioned Claude models remain invalid. The external runner rejects a missed rolling-alias version pin instead of silently executing it. -A kept predecessor in the catalog matches its family. Dispatch it unchanged, with the sheet's model and effort, until `/setup-pstack` replaces it. Do not rewrite it in memory. +A kept predecessor in the catalog matches its family. Dispatch it with the sheet's model until `/setup-pstack` replaces it, and with the sheet's effort or a per-spawn effort that its row lists. Do not rewrite its model in memory. ## The parent owns the route @@ -36,7 +36,7 @@ The catalog's [route table](model-catalog.md#routes) gives each provider's route Native dispatch avoids a second CLI startup and its base context. -- Claude Code: match the descriptor's `(provider, model)` to one model-matrix row, then dispatch it through `pstack-<family>-<effort>` using that row's family and the descriptor's effort. Those definitions select the rolling model alias, requested effort, and `background: true`. `pstack-fable-max` and `pstack-opus-xhigh` remain in that set. Pass the complete task, grounding paths, access mode, and unique output location in the `Agent` prompt. Retain the task handle and drain it only after fan-out. +- Claude Code: match the descriptor's `(provider, model)` to one model-matrix row, then dispatch it through `pstack-<family>-<effort>` using that row's family and the descriptor's effort. Those definitions select the row's model, requested effort, and `background: true`. `pstack-fable-max` and `pstack-opus-xhigh` remain in that set. Pass the complete task, grounding paths, access mode, and unique output location in the `Agent` prompt. Retain the task handle and drain it only after fan-out. - Codex: call `spawn_agent` with the descriptor's model and `reasoning_effort`, the complete task, grounding paths, access mode, and unique output location. Use an isolated worktree for a writer. Codex subagents already run concurrently. Do not send a same-provider descriptor to the external runner. It rejects that call because the native route is cheaper and already available. @@ -50,8 +50,9 @@ pstack-runner \ --parent <claude|codex> \ --provider <provider in the route table> \ --model <real CLI model> \ - --effort <low|medium|high|xhigh|max|ultra> \ + --effort <low|medium|high|xhigh|max|ultra|default> \ --mode <read-only|isolated-write> \ + [--writer-sandbox full-access] \ --prompt <unique prompt file> \ --cwd <repository or dedicated worktree> \ --output <unique final-response file> \ @@ -63,6 +64,10 @@ Pass arguments as an argv array or quote every path. Never interpolate prompt te Provider-specific preflight details, state paths, and host prerequisites are in the provider notes. +### Codex writer sandbox + +A model sheet may hold one setting line that is not a role: `codex writer sandbox: full-access` or `codex writer sandbox: bounded`. Read it once, from the same sheet as the roles. Append `--writer-sandbox full-access` only when that line says `full-access` and the lane is an external Codex `isolated-write` lane. Without the line, or with `bounded`, pass no `--writer-sandbox`, and the writer stays on Codex `workspace-write`. Never pass it for a read-only lane or another provider; the runner refuses both before launch. A native lane is unchanged: a Codex writer on a Codex parent stays on `spawn_agent` with the parent's sandbox, and the setting never moves a native lane to the runner. Never add the flag, or retry with it, after a sandbox failure; only the sheet opts in. Full access removes Codex's sandbox. A dedicated worktree keeps the writer's files apart, but it confines nothing, and a linked worktree shares the main repository's Git state. + ### Host and parent prerequisites - The parent tool sandbox still governs whether a subscribed child CLI can reach its credentials and network. Run setup's live probe from the actual parent profile. A blocked external CLI is a loud dropout, not a reason to elevate permissions or substitute a model silently. @@ -80,9 +85,9 @@ Start the background process, continue launching the other lanes, then drain the The runner and its preflight have no implicit timeout. Do not invent a duration from role, mode, or a convenient round number; real implementation lanes can run for 90 minutes or much longer. Pass `--timeout` only when the user, an external service deadline, or a measured task contract supplies a real bound. That value starts at wrapper entry, before module loading and argument parsing, and remains one absolute deadline across setup, preflight, model execution, and output capture. It is never a fresh allowance per child, and long waits are armed in runtime-safe chunks without shortening the supplied deadline. Otherwise supervise liveness through the retained background task/session handle and cancel manually only on evidence that the run is dead. Cancel through that retained handle so the runner receives SIGINT or SIGTERM, sends it to an active child when one remains, stops waiting on inherited output pipes, removes the empty output reservation, and writes a `cancelled` receipt. Preserve that receipt; a retry is a new attempt with new unique output and receipt paths. Unchanged running state is not a dropout, and Claude's ten-minute foreground ceiling is never a reason to terminate a healthy lane. -Read-only mode maps to Claude plan mode with project-only settings and an explicit tool list, Codex's read-only sandbox, and Grok auto mode plus its `read-only` sandbox and read-oriented tool list. Headless Grok cancels the whole turn on a permission prompt, which its `plan` mode raises for any shell command outside its built-in read-only list; auto mode reports a blocked call to the model instead. Grok's built-in read-only profile deliberately keeps its own state and system temporary directories writable, so point a read-only Grok lane at the actual checkout rather than a worktree under `/tmp`, `/var/tmp`, or the host's temporary directory. `isolated-write` maps to Claude `acceptEdits` with project-only settings plus the Claude Code Bash sandbox, Codex `workspace-write`, and Grok auto mode plus its `workspace` sandbox and write-capable tool list. The Claude writer's Bash runs without prompts in that sandbox, and in a linked worktree the runner denies writes to the main repository's shared `.git`. The runner refuses the writer before launch when it cannot resolve that shared `.git` exactly or express its path literally. Grok uses auto mode in both access modes because `acceptEdits` raises the same turn-cancelling prompt for a shell command outside Grok's built-in list. Give every writer only a dedicated worktree or output directory. Never route a writer into the primary checkout. +Read-only mode maps to Claude plan mode with project-only settings and an explicit tool list, Codex's read-only sandbox, and Grok auto mode plus its `read-only` sandbox and read-oriented tool list. Headless Grok cancels the whole turn on a permission prompt, which its `plan` mode raises for any shell command outside its built-in read-only list; auto mode reports a blocked call to the model instead. Grok's built-in read-only profile deliberately keeps its own state and system temporary directories writable, so point a read-only Grok lane at the actual checkout rather than a worktree under `/tmp`, `/var/tmp`, or the host's temporary directory. `isolated-write` maps to Claude `acceptEdits` with project-only settings plus the Claude Code Bash sandbox, Codex `workspace-write` (or `danger-full-access` for a writer the sheet opts in, as [Codex writer sandbox](#codex-writer-sandbox) says), and Grok auto mode plus its `workspace` sandbox and write-capable tool list. The Claude writer's Bash runs without prompts in that sandbox, and in a linked worktree the runner denies writes to the main repository's shared `.git`. The runner refuses the writer before launch when it cannot resolve that shared `.git` exactly or express its path literally. Grok uses auto mode in both access modes because `acceptEdits` raises the same turn-cancelling prompt for a shell command outside Grok's built-in list. Every other provider's mapping is in its [provider notes](model-catalog.md#provider-notes). The route table's Access modes cell lists each provider's modes. Give every writer only a dedicated worktree or output directory. Never route a writer into the primary checkout. -Every concurrent external lane needs distinct prompt, output, and receipt paths. The launcher exclusively reserves the output, receipt, and `<receipt>.stdout` / `<receipt>.stderr` sidecars with private (`0600`) permissions and refuses to overwrite existing files. All these paths must be distinct from each other and the prompt; a failed reservation rolls back only files created by that attempt. Schema-version-1 receipts add nullable `stdoutPath` and `stderrPath` fields identifying reserved artifacts. Sidecars retain the model process's raw stdout/stderr bytes, including on dropouts; they do not include authentication-preflight output. Cancellation and timeout retain every byte captured before the drain stops. Keep these private local files with the receipt; quote only operator-selected excerpts in public evidence. +Every concurrent external lane needs distinct prompt, output, and receipt paths. The launcher exclusively reserves the output, receipt, and `<receipt>.stdout` / `<receipt>.stderr` sidecars with private (`0600`) permissions and refuses to overwrite existing files. For a provider whose notes name one, it also creates the private `<receipt>.provider/` directory (`0700`) that holds the provider's run-owned config and the file the CLI exports. All these paths must be distinct from each other and the prompt; a failed reservation rolls back only files created by that attempt. Schema-version-1 receipts add nullable `stdoutPath` and `stderrPath` fields identifying reserved artifacts. Sidecars retain the model process's raw stdout/stderr bytes, including on dropouts; they do not include authentication-preflight output. Cancellation and timeout retain every byte captured before the drain stops. Keep these private local files with the receipt; quote only operator-selected excerpts in public evidence. ## Completion and dropouts @@ -90,11 +95,11 @@ Success requires all of these: 1. Exit status `0`. 2. Receipt status `complete`. -3. Either `modelVerified: true` with `modelEvidence: "provider-report"`, or a Codex receipt with `reportedModel: null`, `modelVerified: false`, and `modelEvidence: "pinned-argv"`. For Claude's `fable` and `opus` aliases, the concrete provider report must belong to the requested family. Codex 0.149.0 accepts the exact `--model` argument but does not report the served model in its JSONL stream. +3. Either `modelVerified: true` with `modelEvidence: "provider-report"`, or, for a provider whose Model proof cell in the [route table](model-catalog.md#routes) reads `pinned argv`, such as Codex, a receipt with `reportedModel: null`, `modelVerified: false`, and `modelEvidence: "pinned-argv"`. For Claude's `fable` and `opus` aliases, the concrete provider report must belong to the requested family. Codex 0.149.0 accepts the exact `--model` argument but does not report the served model in its JSONL stream. 4. A non-empty output file. The receipt also carries elapsed time, token usage when the CLI exposes it, and cost when available. Keep it with the arena or review artifacts so parent-harness comparisons are evidence-based. -Any missing CLI, failed login, unavailable model, explicit timeout, cancellation, catchable post-reservation launcher failure, non-zero child exit, malformed result, or model mismatch is a receipt-bearing dropout. Record it and apply the calling skill's existing dropout policy. A `cancelled` receipt can represent either a launcher signal or a well-formed Grok terminal cancellation. Provider cancellations return wrapper exit 130; other valid Grok terminal failures return `child-failed`/70. After a zero exit, invalid or incomplete Grok terminal data is `malformed-output`/65. After a nonzero exit, it takes the ordinary failure classification: normally `child-failed`/70, or `unavailable-model`/69 when the model-error text matches. Only the authentication preflight classifies a lane as `unauthenticated`/77. A nonzero child exit after the model call starts, without a provider-reported reason, is never `unauthenticated`, even when its output mentions authentication. If a login expires between the preflight and the model call, the receipt is `child-failed`/70 and keeps the CLI's error text in `error.evidence`. These provider failures preserve the exact reason in `error.message`, put it first in bounded evidence, and retain reported model/session/usage/cost and the actual child exit code, even after an ordinary nonzero child exit. Missing provider metadata stays null. Launcher cancellation and timeout take precedence. The `signal` field is non-null only when the runner sent that signal to a still-active direct CLI child, and remains null for provider-only cancellation or when launcher cancellation only stopped a post-exit pipe drain. The provider CLI owns any processes it starts beneath that direct child; the receipt does not claim a process-tree kill. Do not delete or overwrite the receipt. Never substitute the parent model, retry another provider, or reinterpret an external descriptor as a native model slug. +Any missing CLI, failed login, unavailable model, explicit timeout, cancellation, catchable post-reservation launcher failure, non-zero child exit, malformed result, or model mismatch is a receipt-bearing dropout. Record it and apply the calling skill's existing dropout policy. A `cancelled` receipt can represent either a launcher signal or a provider-reported cancellation: a well-formed Grok terminal cancellation, or a Devin cancellation that its provider notes describe. Provider cancellations return wrapper exit 130; other valid Grok terminal failures return `child-failed`/70. After a zero exit, invalid or incomplete Grok terminal data is `malformed-output`/65. After a nonzero exit, it takes the ordinary failure classification: normally `child-failed`/70, or `unavailable-model`/69 when the model-error text matches. Only the authentication preflight classifies a lane as `unauthenticated`/77. A nonzero child exit after the model call starts, without a provider-reported reason, is never `unauthenticated`, even when its output mentions authentication. If a login expires between the preflight and the model call, the receipt is `child-failed`/70 and keeps the CLI's error text in `error.evidence`. These provider failures preserve the exact reason in `error.message`, put it first in bounded evidence, and retain reported model/session/usage/cost and the actual child exit code, even after an ordinary nonzero child exit. Missing provider metadata stays null. Launcher cancellation and timeout take precedence. The `signal` field is non-null only when the runner sent that signal to a still-active direct CLI child, and remains null for provider-only cancellation or when launcher cancellation only stopped a post-exit pipe drain. The provider CLI owns any processes it starts beneath that direct child; the receipt does not claim a process-tree kill. Do not delete or overwrite the receipt. Never substitute the parent model, retry another provider, or reinterpret an external descriptor as a native model slug. Start native and external lanes in the same fan-out phase, then wait for all of them before judging. A judge must not read candidate paths while their owners are still writing. diff --git a/plugins/pstack/skills/poteto-mode/scripts/orch/orch.test.ts b/plugins/pstack/skills/poteto-mode/scripts/orch/orch.test.ts index 9c234de7..6bf25017 100644 --- a/plugins/pstack/skills/poteto-mode/scripts/orch/orch.test.ts +++ b/plugins/pstack/skills/poteto-mode/scripts/orch/orch.test.ts @@ -16,6 +16,7 @@ import { UserError, openStore, parseVerdict, + type Forge, type OpenStoreOptions, type Store, } from "./store.ts"; @@ -76,6 +77,7 @@ async function makeGitStack(directory: string): Promise<{ readonly mergedSha: string; readonly closedSha: string; readonly openSha: string; + readonly draftSha: string; }> { const repo = join(directory, "repo"); await mkdir(repo); @@ -86,70 +88,154 @@ async function makeGitStack(directory: string): Promise<{ git({ repo, args: ["add", "."] }); git({ repo, args: ["commit", "-m", "main"] }); - const branches = ["stack/merged", "stack/closed", "stack/open"]; + const branches = [ + "stack/merged", + "stack/closed", + "stack/open", + "stack/draft", + ]; for (const [index, branch] of branches.entries()) { git({ repo, args: ["checkout", "-b", branch] }); await writeFile(join(repo, `stack-${index}.txt`), `${branch}\n`); git({ repo, args: ["add", "."] }); git({ repo, args: ["commit", "-m", branch] }); } + git({ repo, args: ["checkout", "stack/open"] }); return { repo, mergedSha: git({ repo, args: ["rev-parse", "stack/merged"] }), closedSha: git({ repo, args: ["rev-parse", "stack/closed"] }), openSha: git({ repo, args: ["rev-parse", "stack/open"] }), + draftSha: git({ repo, args: ["rev-parse", "stack/draft"] }), }; } -async function withFakeGt<T>({ +type FixtureState = "open" | "draft" | "merged" | "closed"; + +const FORGE_FIXTURES: Readonly< + Record< + Forge, + { + readonly fields: string; + readonly row: (pr: { + number: number; + state: FixtureState; + head: string; + base: string; + }) => Record<string, unknown>; + } + > +> = { + gh: { + fields: "number,state,headRefName,baseRefName", + row: ({ number, state, head, base }) => ({ + number, + state: state === "draft" ? "OPEN" : state.toUpperCase(), + headRefName: head, + baseRefName: base, + }), + }, + origin: { + fields: "number,status,headRef,baseRef", + row: ({ number, state, head, base }) => ({ + number, + status: state, + headRef: head, + baseRef: base, + }), + }, +}; + +function forgeRows( + forge: Forge, + prs: readonly { + number: number; + state: FixtureState; + head: string; + base: string; + }[] +): string { + return JSON.stringify(prs.map((pr) => FORGE_FIXTURES[forge].row(pr))); +} + +async function withFakeForge<T>({ directory, + forge, operation, - output, + currentRows, }: { directory: string; - operation: (outputPath: string) => Promise<T>; - output: string; + forge: Forge; + operation: () => Promise<T>; + currentRows?: string; }): Promise<T> { const bin = join(directory, "bin"); - const outputPath = join(directory, "gt-output.txt"); - await mkdir(bin); - await writeFile(outputPath, output); - const gt = join(bin, "gt"); + const outputPath = join(directory, "forge-output.txt"); + await mkdir(bin, { recursive: true }); + await writeFile( + outputPath, + `${ + currentRows ?? + forgeRows(forge, [ + { + number: 11, + state: "open", + head: "stack/open", + base: "stack/closed", + }, + ]) + }\n` + ); + const { fields } = FORGE_FIXTURES[forge]; + const list = (filter: string, branch: string): string => + `pr list ${filter} ${branch} --state all --json ${fields}`; + const fake = join(bin, forge); await writeFile( - gt, + fake, `#!/usr/bin/env bash set -euo pipefail if [ "$(pwd -P)" != "${realpathSync(join(directory, "repo"))}" ]; then - printf 'gt ran outside the fixture repo: %s\\n' "$(pwd -P)" >&2 + printf '${forge} ran outside the fixture repo: %s\\n' "$(pwd -P)" >&2 exit 2 fi case "$*" in - "--no-interactive log short --stack --reverse") + "${list("--head", "stack/open")}") cat "${outputPath}" ;; - "--no-interactive info stack/merged") - printf 'stack/merged\\nPR #10 (Merged) merged change\\n' + "${list("--head", "stack/closed")}") + printf '%s\\n' '${forgeRows(forge, [{ number: 13, state: "closed", head: "stack/closed", base: "stack/merged" }])}' + ;; + "${list("--head", "stack/merged")}") + printf '%s\\n' '${forgeRows(forge, [{ number: 10, state: "merged", head: "stack/merged", base: "main" }])}' ;; - "--no-interactive info stack/closed") - printf 'stack/closed\\nPR #13 (Closed) closed change\\n' + "${list("--base", "stack/open")}") + printf '%s\\n' '${forgeRows(forge, [ + { number: 14, state: "draft", head: "stack/draft", base: "stack/open" }, + { + number: 15, + state: "closed", + head: "stack/abandoned", + base: "stack/open", + }, + ])}' ;; - "--no-interactive info stack/open") - printf 'stack/open\\nPR #11 (Needs approvals) open change\\n' + "${list("--base", "stack/draft")}") + printf '[]\\n' ;; *) - printf 'unexpected gt arguments: %s\\n' "$*" >&2 + printf 'unexpected ${forge} arguments: %s\\n' "$*" >&2 exit 2 ;; esac ` ); - await chmod(gt, 0o755); + await chmod(fake, 0o755); const originalPath = process.env.PATH; process.env.PATH = `${bin}:${originalPath ?? ""}`; try { - return await operation(outputPath); + return await operation(); } finally { if (originalPath === undefined) { delete process.env.PATH; @@ -406,90 +492,117 @@ describe("Store", () => { ]); }); - it("resolves the ordered Graphite frontier and validates an optional pin", async () => { - const { directory, store } = await initializedStore(); - const stack = await makeGitStack(directory); - const output = `◯ main -◯ stack/merged -◯ stack/closed -◉ stack/open (current) -`; - - await withFakeGt({ - directory, - output, - operation: async () => { - expect(await store.frontier.set({ repo: stack.repo })).toEqual({ - generation: 1, - prs: [ - { - pr: 10, - branches: "stack/merged", - sha: stack.mergedSha, - state: "MERGED", - }, - { - pr: 13, - branches: "stack/closed", - sha: stack.closedSha, - state: "CLOSED", - }, - { - pr: 11, - branches: "stack/open", - sha: stack.openSha, - state: "OPEN", - }, - ], - lowestUnmerged: 11, - }); - expect( - ( - await store.frontier.set({ + for (const forge of ["gh", "origin"] as const) { + it(`resolves the ordered frontier from the ${forge} base chain and validates an optional pin`, async () => { + const { directory, store } = await initializedStore(); + const stack = await makeGitStack(directory); + const forgeParams = forge === "gh" ? {} : { forge }; + + await withFakeForge({ + directory, + forge, + operation: async () => { + expect( + await store.frontier.set({ repo: stack.repo, ...forgeParams }) + ).toEqual({ + generation: 1, + prs: [ + { + pr: 10, + branches: "stack/merged", + sha: stack.mergedSha, + state: "MERGED", + }, + { + pr: 13, + branches: "stack/closed", + sha: stack.closedSha, + state: "CLOSED", + }, + { + pr: 11, + branches: "stack/open", + sha: stack.openSha, + state: "OPEN", + }, + { + pr: 14, + branches: "stack/draft", + sha: stack.draftSha, + state: "OPEN", + }, + ], + lowestUnmerged: 11, + }); + expect( + ( + await store.frontier.set({ + repo: stack.repo, + ...forgeParams, + prs: [10, 13, 11, 14], + }) + ).generation + ).toBe(2); + expect((await store.frontier.show()).generation).toBe(2); + await expect( + store.frontier.set({ repo: stack.repo, - prs: [10, 13, 11], + ...forgeParams, + prs: [10, 11, 12], }) - ).generation - ).toBe(2); - expect((await store.frontier.show()).generation).toBe(2); - await expect( - store.frontier.set({ - repo: stack.repo, - prs: [10, 11, 12], - }) - ).rejects.toThrow( - "frontier pin mismatch: missing from gt: 12; extra in gt: 13" - ); - await expect( - store.frontier.set({ - repo: stack.repo, - prs: [13, 10, 11], - }) - ).rejects.toThrow( - "frontier pin mismatch: order differs: expected 13,10,11; gt 10,13,11" - ); - await expect( - store.frontier.set({ - repo: stack.repo, - prs: [10, 10], - }) - ).rejects.toThrow("--prs must not contain duplicates"); - }, + ).rejects.toThrow( + "frontier pin mismatch: missing from resolved chain: 12; extra in resolved chain: 13,14" + ); + await expect( + store.frontier.set({ + repo: stack.repo, + ...forgeParams, + prs: [13, 10, 11, 14], + }) + ).rejects.toThrow( + "frontier pin mismatch: order differs: expected 13,10,11,14; resolved 10,13,11,14" + ); + await expect( + store.frontier.set({ + repo: stack.repo, + ...forgeParams, + prs: [10, 10], + }) + ).rejects.toThrow("--prs must not contain duplicates"); + }, + }); }); - }); - it("rejects unparseable Graphite output loudly", async () => { + it(`rejects unparseable ${forge} output loudly`, async () => { + const { directory, store } = await initializedStore(); + const stack = await makeGitStack(directory); + + await withFakeForge({ + directory, + forge, + currentRows: "this line is not forge output", + operation: async () => { + await expect( + store.frontier.set({ repo: stack.repo, forge }) + ).rejects.toThrow( + `${forge} pr list --head stack/open output is not valid JSON` + ); + }, + }); + }); + } + + it("names the missing pull request when a chain branch has none", async () => { const { directory, store } = await initializedStore(); const stack = await makeGitStack(directory); - await withFakeGt({ + await withFakeForge({ directory, - output: "◯ main\nthis line is not Graphite output\n", + forge: "gh", + currentRows: "[]", operation: async () => { - await expect( - store.frontier.set({ repo: stack.repo }) - ).rejects.toThrow( - 'gt log short output has an unparseable line 2: "this line is not Graphite output"' + await expect(store.frontier.set({ repo: stack.repo })).rejects.toThrow( + "branch stack/open has no pull request; open one with gh pr create" ); }, }); @@ -548,12 +661,26 @@ describe("orch CLI", () => { const frontierHelp = runCli(["frontier", "set", "--help"]); expect(frontierHelp.code).toBe(0); expect(frontierHelp.stdout).toContain("--repo <dir>"); + expect(frontierHelp.stdout).toContain("--forge <forge>"); expect(frontierHelp.stdout).toContain("--prs <n,...>"); const directory = await makeDirectory(); const invalid = runCli(["--store", directory, "unit", "add", "u1"]); expect(invalid.code).toBe(1); expect(invalid.stderr).toContain("required option '--track <track>'"); + + const badForge = runCli([ + "--store", + directory, + "frontier", + "set", + "--repo", + directory, + "--forge", + "gt", + ]); + expect(badForge.code).toBe(1); + expect(badForge.stderr).toContain("Allowed choices are gh, origin"); }); it("accepts ORCH_STORE and emits complete JSON", async () => { diff --git a/plugins/pstack/skills/poteto-mode/scripts/orch/orch.ts b/plugins/pstack/skills/poteto-mode/scripts/orch/orch.ts index 219ac17b..14cb5139 100755 --- a/plugins/pstack/skills/poteto-mode/scripts/orch/orch.ts +++ b/plugins/pstack/skills/poteto-mode/scripts/orch/orch.ts @@ -7,6 +7,7 @@ import { openStore, parseVerdict, type Counts, + type Forge, type Frontier, type InboxPointer, type OpenGate, @@ -81,6 +82,7 @@ interface GateResolveOptions { interface FrontierSetOptions { readonly repo?: string; + readonly forge: Forge; readonly prs?: readonly number[]; } @@ -472,15 +474,24 @@ function createProgram(io: Io): Command { const frontier = program .command("frontier") - .description("manage the Graphite stack frontier") + .description("manage the stack frontier") .action(() => requireSubcommand(program)); - leaf(frontier, "set", "discover the Graphite stack and set the frontier") + leaf( + frontier, + "set", + "resolve the pull request base chain and set the frontier" + ) .addOption( new Option( "--repo <dir>", "repository directory (or ORCH_REPO)" ).env("ORCH_REPO") ) + .addOption( + new Option("--forge <forge>", "pull request forge") + .choices(["gh", "origin"]) + .default("gh") + ) .option( "--prs <n,...>", "optional expected pull request order pin", @@ -493,6 +504,7 @@ function createProgram(io: Io): Command { (store) => store.frontier.set({ repo: frontierRepo(options), + forge: options.forge, prs: options.prs, }), frontierLine diff --git a/plugins/pstack/skills/poteto-mode/scripts/orch/store.ts b/plugins/pstack/skills/poteto-mode/scripts/orch/store.ts index 5e6c602f..41736891 100644 --- a/plugins/pstack/skills/poteto-mode/scripts/orch/store.ts +++ b/plugins/pstack/skills/poteto-mode/scripts/orch/store.ts @@ -167,6 +167,7 @@ export interface ResolveGateParams { export interface SetFrontierParams { readonly repo: string; + readonly forge?: Forge; readonly prs?: readonly number[]; } @@ -967,168 +968,187 @@ function countLine(value: Counts): string { : entries.map(([name, count]) => `${name}=${count}`).join(", "); } -const OPEN_GT_PR_STATUSES = new Set([ - "Trunk branch locked", - "Changes requested", - "Waiting on PRs in this stack to merge", - "Waiting on downstack merge state", - "Draft", - "Required checks failed", - "Undergoing failure detection", - "Merge queue failed on current head commit", - "Handed off to merge queue...", - "Waiting on downstack", - "Merge conflicts", - "Needs reviewers", - "Needs approvals", - "Needs restack", - "Queued to merge...", - "Ready to merge", - "Ready to merge as stack", - "Rebasing...", - "Waiting on CI...", - "Stale, needs rebase onto trunk", - "Unresolved comments", - "Waiting on required CI", - "Waiting to merge...", -]); - -interface GtPullRequest { - readonly pr: number; - readonly state: FrontierPrState; +export type Forge = "gh" | "origin"; + +interface ForgePullRequestQuery { + readonly stateField: string; + readonly headField: string; + readonly baseField: string; + readonly states: ReadonlyMap<string, FrontierPrState>; +} + +const FORGE_PULL_REQUEST_QUERIES: Readonly< + Record<Forge, ForgePullRequestQuery> +> = { + gh: { + stateField: "state", + headField: "headRefName", + baseField: "baseRefName", + states: new Map<string, FrontierPrState>([ + ["OPEN", "OPEN"], + ["MERGED", "MERGED"], + ["CLOSED", "CLOSED"], + ]), + }, + origin: { + stateField: "status", + headField: "headRef", + baseField: "baseRef", + states: new Map<string, FrontierPrState>([ + ["open", "OPEN"], + ["draft", "OPEN"], + ["merged", "MERGED"], + ["closed", "CLOSED"], + ]), + }, +}; + +export function parseForge(value: string): Forge { + if (value === "gh" || value === "origin") { + return value; + } + throw new UserError("forge must be gh or origin"); } -interface GtFrontierEntry extends GtPullRequest { - readonly branches: string; +interface ForgePullRequest { + readonly pr: number; + readonly state: FrontierPrState; + readonly branch: string; + readonly base: string; } -function parseGtPullRequest({ - branch, - detail, +function runTool({ + command, + args, + repo, }: { - branch: string; - detail: string; -}): GtPullRequest { - const match = - /^(?:\[origin\] )?PR #([1-9]\d*)(?: \(([^)\r\n]+)\))?(?: .+)?$/.exec( - detail - ); - const pr = Number(match?.[1] ?? 0); - if (match === null || !Number.isSafeInteger(pr)) { + command: string; + args: readonly string[]; + repo: string; +}): string { + try { + return execFileSync(command, args, { + cwd: repo, + encoding: "utf8", + env: process.env, + stdio: ["ignore", "pipe", "pipe"], + }); + } catch (error) { throw new UserError( - `gt info output has an invalid PR row for branch ${branch}: ${detail}` + `${command} ${args.join(" ")} failed: ${errorMessage(error)}` ); } - const status = match[2]; - if (status === "Merged") { - return { pr, state: "MERGED" }; - } - if (status === "Closed") { - return { pr, state: "CLOSED" }; - } - if (status === undefined || OPEN_GT_PR_STATUSES.has(status)) { - return { pr, state: "OPEN" }; - } - throw new UserError( - `gt info output has an unknown PR state for branch ${branch}: ${status}` - ); } -function parseGtBranches(raw: string): readonly string[] { - const branches: string[] = []; - const lines = raw.replace(/\r/g, "").split("\n"); - for (const [index, line] of lines.entries()) { - if (line.length === 0) { - continue; - } - const branchMatch = - /^(?:│ )*[◯◉] +([^\s]+)((?: \([^()\r\n]*\))*)$/.exec(line); - if (branchMatch === null) { - throw new UserError( - `gt log short output has an unparseable line ${index + 1}: ${JSON.stringify(line)}` - ); - } - const branch = branchMatch[1] ?? ""; - if (branches.includes(branch)) { - throw new UserError( - `gt log short output contains duplicate branch ${branch}` - ); - } - branches.push(branch); +function listPullRequests({ + filter, + branch, + forge, + repo, +}: { + filter: "--head" | "--base"; + branch: string; + forge: Forge; + repo: string; +}): readonly ForgePullRequest[] { + const query = FORGE_PULL_REQUEST_QUERIES[forge]; + const command = `${forge} pr list ${filter} ${branch}`; + // Every state is fetched because a merged or closed PR still anchors the + // chain, and a draft counts as OPEN. + const raw = runTool({ + command: forge, + args: [ + "pr", + "list", + filter, + branch, + "--state", + "all", + "--json", + ["number", query.stateField, query.headField, query.baseField].join( + "," + ), + ], + repo, + }); + let value: unknown; + try { + value = JSON.parse(raw); + } catch { + throw new UserError(`${command} output is not valid JSON`); } - const trunk = branches[0]; - if (trunk === undefined) { - throw new UserError("gt log short output did not contain a stack"); + if (!isUnknownArray(value)) { + throw new UserError(`${command} output is not a JSON array`); } - return branches.slice(1); + return value.map((row) => { + const status = isRecord(row) ? row[query.stateField] : undefined; + const state = + typeof status === "string" ? (query.states.get(status) ?? null) : null; + const head = isRecord(row) ? row[query.headField] : undefined; + const base = isRecord(row) ? row[query.baseField] : undefined; + if ( + !isRecord(row) || + typeof row.number !== "number" || + !Number.isSafeInteger(row.number) || + row.number < 1 || + state === null || + typeof head !== "string" || + head.length === 0 || + typeof base !== "string" || + base.length === 0 + ) { + throw new UserError(`${command} output has an invalid PR row`); + } + return { pr: row.number, state, branch: head, base }; + }); } -function graphitePullRequest({ +function branchPullRequest({ branch, + forge, repo, }: { branch: string; + forge: Forge; repo: string; -}): GtPullRequest { - let raw: string; - try { - raw = execFileSync("gt", ["--no-interactive", "info", branch], { - cwd: repo, - encoding: "utf8", - env: { ...process.env, NO_COLOR: "1" }, - stdio: ["ignore", "pipe", "pipe"], - }); - } catch (error) { - throw new UserError( - `gt info ${branch} failed: ${errorMessage(error)}` - ); - } - const rows = raw - .replace(/\r/g, "") - .split("\n") - .filter( - (line) => - line.startsWith("PR #") || line.startsWith("[origin] PR #") - ); - if (rows.length === 0) { +}): ForgePullRequest { + const rows = listPullRequests({ filter: "--head", branch, forge, repo }); + const row = rows[0]; + if (row === undefined) { throw new UserError( - `gt info output branch ${branch} has no pull request; this clone's gt metadata may predate the submit, so resolve the frontier from the stacker's clone or after gt sync` + `branch ${branch} has no pull request; open one with ${forge} pr create, or resolve the frontier from a branch whose chain is fully submitted` ); } if (rows.length > 1) { throw new UserError( - `gt info output contains multiple PRs for branch ${branch}` + `branch ${branch} has multiple pull requests (${rows.map((item) => `#${item.pr}`).join(", ")}); close the stale ones first` ); } - return parseGtPullRequest({ branch, detail: rows[0] ?? "" }); + return row; } -function graphiteFrontier(repo: string): readonly GtFrontierEntry[] { - let raw: string; - try { - raw = execFileSync( - "gt", - ["--no-interactive", "log", "short", "--stack", "--reverse"], - { - cwd: repo, - encoding: "utf8", - env: { ...process.env, NO_COLOR: "1" }, - stdio: ["ignore", "pipe", "pipe"], - } - ); - } catch (error) { - throw new UserError( - `gt log short --stack --reverse failed: ${errorMessage(error)}` - ); - } - const result = parseGtBranches(raw).map((branch) => ({ - branches: branch, - ...graphitePullRequest({ branch, repo }), - })); - if (new Set(result.map((row) => row.pr)).size !== result.length) { - throw new UserError("gt info output contains duplicate pull requests"); - } - return result; +function openChildren({ + branch, + forge, + repo, +}: { + branch: string; + forge: Forge; + repo: string; +}): readonly ForgePullRequest[] { + return listPullRequests({ filter: "--base", branch, forge, repo }).filter( + (row) => row.state === "OPEN" + ); +} + +function runGit({ + args, + repo, +}: { + args: readonly string[]; + repo: string; +}): string { + return runTool({ command: "git", args, repo }).trim(); } function branchSha({ @@ -1138,30 +1158,79 @@ function branchSha({ branch: string; repo: string; }): string { - let raw: string; - try { - raw = execFileSync("git", ["rev-parse", branch], { - cwd: repo, - encoding: "utf8", - env: process.env, - stdio: ["ignore", "pipe", "pipe"], - }); - } catch (error) { - throw new UserError( - `git rev-parse ${branch} failed: ${errorMessage(error)}` - ); - } - const sha = raw.trim(); + const sha = runGit({ args: ["rev-parse", branch], repo }); if (!/^[0-9a-f]{40,64}$/i.test(sha)) { throw new UserError(`git rev-parse ${branch} returned an invalid SHA`); } return sha; } -function resolveFrontier(repo: string): readonly FrontierPr[] { - return graphiteFrontier(repo).map((row) => ({ - ...row, - sha: branchSha({ branch: row.branches, repo }), +function currentBranch(repo: string): string { + return runGit({ args: ["symbolic-ref", "--short", "HEAD"], repo }); +} + +function trunkBranch(repo: string): string { + try { + return runGit({ + args: ["symbolic-ref", "--short", "refs/remotes/origin/HEAD"], + repo, + }).replace(/^origin\//, ""); + } catch { + // Init-plus-fetch checkouts and local fixtures lack origin/HEAD. + return "main"; + } +} + +function resolveFrontier(repo: string, forge: Forge): readonly FrontierPr[] { + const trunk = trunkBranch(repo); + const current = currentBranch(repo); + if (current === trunk) { + return []; + } + const seen = new Set([current]); + let bottom = branchPullRequest({ branch: current, forge, repo }); + const chain = [bottom]; + while (bottom.base !== trunk) { + if (seen.has(bottom.base)) { + throw new UserError( + `pull request base chain loops at branch ${bottom.base}` + ); + } + seen.add(bottom.base); + bottom = branchPullRequest({ branch: bottom.base, forge, repo }); + chain.unshift(bottom); + } + let tip = current; + for (;;) { + const children = openChildren({ branch: tip, forge, repo }); + const child = children[0]; + if (child === undefined) { + break; + } + if (children.length > 1) { + throw new UserError( + `stack is not linear: branches ${children.map((row) => row.branch).join(", ")} all target ${tip}` + ); + } + if (seen.has(child.branch)) { + throw new UserError( + `pull request base chain loops at branch ${child.branch}` + ); + } + seen.add(child.branch); + chain.push(child); + tip = child.branch; + } + if (new Set(chain.map((row) => row.pr)).size !== chain.length) { + throw new UserError( + "pull request base chain contains duplicate pull requests" + ); + } + return chain.map((row) => ({ + pr: row.pr, + branches: row.branch, + sha: branchSha({ branch: row.branch, repo }), + state: row.state, })); } @@ -1184,14 +1253,14 @@ function validateFrontierPin({ const extra = actual.filter((pr) => !expectedSet.has(pr)); const drift: string[] = []; if (missing.length > 0) { - drift.push(`missing from gt: ${missing.join(",")}`); + drift.push(`missing from resolved chain: ${missing.join(",")}`); } if (extra.length > 0) { - drift.push(`extra in gt: ${extra.join(",")}`); + drift.push(`extra in resolved chain: ${extra.join(",")}`); } if (missing.length === 0 && extra.length === 0) { drift.push( - `order differs: expected ${expected.join(",")}; gt ${actual.join(",")}` + `order differs: expected ${expected.join(",")}; resolved ${actual.join(",")}` ); } throw new UserError(`frontier pin mismatch: ${drift.join("; ")}`); @@ -1484,6 +1553,7 @@ export function openStore( set: async (params) => { await beginWrite(); const repo = resolve(requiredLine(params.repo, "repo directory")); + const forge = parseForge(params.forge ?? "gh"); const pin = params.prs === undefined ? undefined @@ -1492,7 +1562,7 @@ export function openStore( throw new UserError("--prs must not contain duplicates"); } const old = await readFrontier(store); - const prs = resolveFrontier(repo); + const prs = resolveFrontier(repo, forge); if (pin !== undefined) { validateFrontierPin({ actual: prs.map((row) => row.pr), diff --git a/plugins/pstack/skills/poteto-mode/scripts/runner/cli.test.ts b/plugins/pstack/skills/poteto-mode/scripts/runner/cli.test.ts index 504c9662..214699cc 100644 --- a/plugins/pstack/skills/poteto-mode/scripts/runner/cli.test.ts +++ b/plugins/pstack/skills/poteto-mode/scripts/runner/cli.test.ts @@ -41,6 +41,22 @@ describe("runner CLI parsing", () => { ); }); + it("defaults a writer to the bounded sandbox and passes an explicit full-access opt-in", () => { + const writer = argv().map((value) => (value === "read-only" ? "isolated-write" : value)); + expect(parseArgs(writer)?.access).toEqual({ mode: "isolated-write", sandbox: "bounded" }); + expect(parseArgs([...writer, "--writer-sandbox", "full-access"])?.access) + .toEqual({ mode: "isolated-write", sandbox: "full-access" }); + expect(() => parseArgs(argv(["--writer-sandbox", "full-access"]))) + .toThrow("writer sandbox full-access requires mode isolated-write"); + }); + + it("names the writer sandbox flag and its default in the help text", async () => { + let stdout = ""; + await main(["--help"], Date.now(), { stdout: (value) => { stdout += value; }, stderr: () => {} }); + expect(stdout).toContain("[--writer-sandbox <bounded|full-access>]"); + expect(stdout).toContain("--writer-sandbox\ndefaults to bounded"); + }); + it("accepts the ultra effort", () => { const args = argv().map((value) => (value === "max" ? "ultra" : value)); expect(parseArgs(args)?.effort).toBe("ultra"); @@ -62,18 +78,18 @@ describe("runner CLI parsing", () => { let stdout = ""; expect(await main(["--help"], Date.now(), { stdout: (value) => { stdout += value; }, stderr: () => {} })).toBe(0); expect(stdout).toStartWith( - "Usage: pstack-runner --parent <claude|codex> --provider <claude|codex|grok> \\\n" + "Usage: pstack-runner --parent <claude|codex> --provider <claude|codex|grok|devin|opencode|dsh|amp|agy|cursor> \\\n" ); }); it("refuses an unnormalized Claude revision with a usage error", async () => { let stderr = ""; const args = argv().map((value) => - value === "claude" ? "codex" : value === "codex" ? "claude" : value === "gpt-5.6-sol" ? "claude-opus-5-5" : value + value === "claude" ? "codex" : value === "codex" ? "claude" : value === "gpt-5.6-sol" ? "claude-opus-5-6" : value ); expect(await main(args, Date.now(), { stdout: () => {}, stderr: (value) => { stderr += value; } })).toBe(64); expect(stderr).toStartWith( - "error: Claude model claude-opus-5-5 is a version pin; normalize it to opus before invoking the runner\n" + "error: Claude model claude-opus-5-6 is a version pin; normalize it to opus before invoking the runner\n" ); }); }); diff --git a/plugins/pstack/skills/poteto-mode/scripts/runner/cli.ts b/plugins/pstack/skills/poteto-mode/scripts/runner/cli.ts index 83bd9cf4..ed599fe1 100644 --- a/plugins/pstack/skills/poteto-mode/scripts/runner/cli.ts +++ b/plugins/pstack/skills/poteto-mode/scripts/runner/cli.ts @@ -6,13 +6,16 @@ import { PARENTS, UsageError } from "./types.ts"; const HELP = `Usage: pstack-runner --parent <${Object.keys(PARENTS).join("|")}> --provider <${REGISTRY.adapters.map((adapter) => adapter.id).join("|")}> \\ --model <slug> --effort <level> --mode <read-only|isolated-write> \\ + [--writer-sandbox <bounded|full-access>] \\ --prompt <file> --cwd <dir> --output <file> --receipt <file> [--timeout <seconds>] Runs exactly one external model lane. Same-provider calls are rejected; use the parent harness's native subagent primitive for those lanes. Output and receipt paths must not already exist. There is no implicit timeout. Pass --timeout only when the user or task supplies a real deadline; it is one end-to-end launcher -deadline shared by setup, preflight, and model execution. +deadline shared by setup, preflight, and model execution. --writer-sandbox +defaults to bounded; full-access applies only to an isolated-write lane on a +provider that supports it, and only when the model sheet opts in. `; interface Io { @@ -49,6 +52,7 @@ export function parseArgs(argv: readonly string[]): Lane | null { model: { type: "string" }, effort: { type: "string" }, mode: { type: "string" }, + "writer-sandbox": { type: "string" }, prompt: { type: "string" }, cwd: { type: "string" }, output: { type: "string" }, @@ -75,6 +79,7 @@ export function parseArgs(argv: readonly string[]): Lane | null { model: stringValue(parsed.values.model) ?? "", effort: stringValue(parsed.values.effort) ?? "", mode: stringValue(parsed.values.mode) ?? "", + writerSandbox: stringValue(parsed.values["writer-sandbox"]), promptPath: resolve(required("prompt", stringValue(parsed.values.prompt))), cwd: resolve(required("cwd", stringValue(parsed.values.cwd))), outputPath: resolve(required("output", stringValue(parsed.values.output))), diff --git a/plugins/pstack/skills/poteto-mode/scripts/runner/providers/adapter-suite.ts b/plugins/pstack/skills/poteto-mode/scripts/runner/providers/adapter-suite.ts index 9bfa3cbd..95d988ad 100644 --- a/plugins/pstack/skills/poteto-mode/scripts/runner/providers/adapter-suite.ts +++ b/plugins/pstack/skills/poteto-mode/scripts/runner/providers/adapter-suite.ts @@ -1,8 +1,8 @@ // Test-only: the properties every adapter keeps. Each providers/<id>.test.ts runs it once. import { describe, expect, it } from "bun:test"; -import type { ProviderAdapter } from "../registry/contract.ts"; -import { type Lane, type LaneRequest, resolveLane } from "../registry/index.ts"; -import { ACCESS_MODES, PARENTS, type Parent } from "../types.ts"; +import type { LaunchFacts, ProviderAdapter } from "../registry/contract.ts"; +import { effortsOf, type Lane, type LaneRequest, modelIds, REGISTRY, resolveLane } from "../registry/index.ts"; +import { ACCESS_KINDS, type AccessKind, PARENTS, type Parent } from "../types.ts"; const BYPASS_TOKENS = [ "danger-full-access", @@ -12,6 +12,13 @@ const BYPASS_TOKENS = [ "dangerous", ]; +/** The request fields that ask for one access kind. */ +export function accessRequest(kind: AccessKind): Pick<LaneRequest, "mode" | "writerSandbox"> { + return kind === "isolated-write-full-access" + ? { mode: "isolated-write", writerSandbox: "full-access" } + : { mode: kind, writerSandbox: "bounded" }; +} + /** A lane from raw request fields, with a parent that differs from the provider. */ export function testLane(request: Partial<LaneRequest> & { readonly provider: string }): Lane { const parent = (Object.keys(PARENTS) as Parent[]).find((name) => name !== request.provider)!; @@ -29,34 +36,85 @@ export function testLane(request: Partial<LaneRequest> & { readonly provider: st }); } +/** The facts run.ts passes before argv, with the provider sidecar paths it reserves next to the receipt. */ +export function testFacts(adapter: ProviderAdapter, sharedGitDir: string | null = null): LaunchFacts { + const sidecar = adapter.launch.sidecar; + const directory = "/tmp/receipt.json.provider"; + return { + sharedGitDir, + providerSidecar: sidecar === null ? null : { + directory, + output: sidecar.output === null ? null : `${directory}/${sidecar.output}`, + config: sidecar.config === null ? null : `${directory}/${sidecar.config.name}`, + }, + }; +} + export function adapterSuite(adapter: ProviderAdapter): void { describe(`${adapter.id} adapter contract`, () => { - const lane = (request: Partial<LaneRequest> = {}) => testLane({ provider: adapter.id, ...request }); - const argv = (request: Partial<LaneRequest> = {}) => adapter.launch.argv(lane(request), { sharedGitDir: null }); + // An unregistered model-ID pair is already the CLI model ID, so it takes only the default effort. + const unregistered = adapter.effortIn === "model-id" ? "default" : "max"; + const lane = (request: Partial<LaneRequest> = {}) => + testLane({ provider: adapter.id, effort: unregistered, ...request }); + const argv = (request: Partial<LaneRequest> = {}) => adapter.launch.argv(lane(request), testFacts(adapter)); + // What the adapter launches is its argv plus the config it has run.ts write. + const config = (request: Partial<LaneRequest> = {}) => adapter.launch.sidecar?.config?.content(lane(request)) ?? ""; it("passes the requested model to the CLI unchanged", () => { - expect(argv()).toEqual(expect.arrayContaining(["--model", "suite-model"])); + expect(argv().includes("suite-model") || config().includes("suite-model")).toBe(true); }); - it("omits exactly the effort flag and value for the default effort", () => { - const high = argv({ effort: "high" }); - const at = high.findIndex((token) => token.includes("high")); - expect(at).toBeGreaterThan(0); - expect(argv({ effort: "default" })).toEqual([...high.slice(0, at - 1), ...high.slice(at + 1)]); - }); + if (adapter.effortIn === "config") { + it("carries the effort in the config only, and omits it for the default effort", () => { + // Line by line, so a path in the config that happens to contain "high" does not count. + const high = config({ effort: "high" }).split("\n"); + const base = config({ effort: "default" }).split("\n"); + expect(high.filter((line) => !base.includes(line))).toEqual([expect.stringContaining("high")]); + expect(base.filter((line) => !high.includes(line))).toEqual([]); + expect(argv({ effort: "high" })).toEqual(argv({ effort: "default" })); + }); + } else if (adapter.effortIn === "flag") { + it("omits exactly the effort flag and value for the default effort", () => { + const high = argv({ effort: "high" }); + const at = high.findIndex((token) => token.includes("high")); + expect(at).toBeGreaterThan(0); + expect(argv({ effort: "default" })).toEqual([...high.slice(0, at - 1), ...high.slice(at + 1)]); + }); + } else { + it("passes each registered effort's CLI model ID and no other effort token", () => { + const families = REGISTRY.families.filter((entry) => entry.provider === adapter.id); + expect(families.length).toBeGreaterThan(0); + for (const family of families) { + for (const effort of effortsOf(family)) { + const args = argv({ model: family.model, effort }); + const id = modelIds(family)![effort]!; + expect([family.name, effort, args.includes(id)]).toEqual([family.name, effort, true]); + expect(args.filter((token) => token.includes(effort) && token !== id)).toEqual([]); + } + } + expect(() => lane({ effort: "max" })).toThrow("effort max is not offered"); + }); + } it("delivers the prompt one way only", () => { - for (const mode of ACCESS_MODES) { - expect(argv({ mode }).includes("/tmp/prompt.md")).toBe(adapter.launch.prompt === "file"); + for (const kind of adapter.launch.supports) { + expect(argv(accessRequest(kind)).includes("/tmp/prompt.md")).toBe(adapter.launch.prompt === "file"); } }); + // The opted-in full-access writer is the one kind allowed a bypass; each adapter's own test pins its argv. it("uses bounded modes without bypass tokens", () => { - for (const mode of ACCESS_MODES) { - for (const token of argv({ mode })) { + for (const kind of adapter.launch.supports.filter((entry) => entry !== "isolated-write-full-access")) { + for (const token of argv(accessRequest(kind))) { for (const bypass of BYPASS_TOKENS) expect(token).not.toContain(bypass); } } }); + + it("refuses every access kind it does not support before launch", () => { + for (const kind of ACCESS_KINDS.filter((entry) => !adapter.launch.supports.includes(entry))) { + expect(() => lane(accessRequest(kind))).toThrow(`provider ${adapter.id} does not support ${kind}`); + } + }); }); } diff --git a/plugins/pstack/skills/poteto-mode/scripts/runner/providers/agy.test.ts b/plugins/pstack/skills/poteto-mode/scripts/runner/providers/agy.test.ts new file mode 100644 index 00000000..f7efe15b --- /dev/null +++ b/plugins/pstack/skills/poteto-mode/scripts/runner/providers/agy.test.ts @@ -0,0 +1,134 @@ +import { describe, expect, it } from "bun:test"; +import { readFileSync } from "node:fs"; +import { join } from "node:path"; +import type { ModelRun } from "../registry/contract.ts"; +import type { LaneRequest } from "../registry/index.ts"; +import { adapterSuite, testFacts, testLane } from "./adapter-suite.ts"; +import { agy } from "./agy.ts"; + +adapterSuite(agy); + +// Sanitized from Antigravity CLI 1.3.6 captures: structure kept, identity and host paths replaced. +const fixture = (name: string): string => readFileSync(join(import.meta.dir, "fixtures/agy", name), "utf8"); + +const lane = (request: Partial<LaneRequest> = {}) => testLane({ provider: "agy", model: "gemini-3.8-flash", effort: "medium", ...request }); +const run = (overrides: Partial<ModelRun>): ModelRun => ({ exitCode: 0, stdout: "", stderr: "", sidecar: null, ...overrides }); +const preflight = (exitCode: number, stdout: string, stderr = "", request: Partial<LaneRequest> = {}) => + agy.preflight.classify({ exitCode, stdout, stderr }, lane(request)); + +describe("agy launch", () => { + it("runs the effort's variant ID in stream-json with the run-owned agent and no mode, sandbox or effort flag", () => { + expect(agy.launch.argv(lane({ effort: "high" }), testFacts(agy))).toEqual([ + "--input-format", + "stream-json", + "--output-format", + "stream-json", + "--model", + "gemini-3.8-flash-high", + "--add-dir", + "/tmp/receipt.json.provider", + "--agent", + "pstack-read-only", + ]); + for (const flag of ["-p", "--mode", "--sandbox", "--effort", "--dangerously-skip-permissions"]) { + expect(agy.launch.argv(lane(), testFacts(agy))).not.toContain(flag); + } + }); + + it("refuses an isolated-write lane, efforts 3.8 Flash does not list, and never maps 3.7 Flash", () => { + expect(() => lane({ mode: "isolated-write" })) + .toThrow("provider agy does not support isolated-write; it supports only: read-only"); + expect(() => lane({ effort: "max" })).toThrow("effort max is not offered for agy:gemini-3.8-flash; choose one of: low, medium, high"); + const unregistered = lane({ model: "gemini-3.7-flash", effort: "default" }); + expect([unregistered.family, unregistered.cliModel]).toEqual([null, "gemini-3.7-flash"]); + }); + + it("wraps the prompt in one stream-json user line", () => { + const line = agy.launch.stdinEnvelope!("Read \"marker.txt\".\nReply exactly.\n"); + expect(line.endsWith("\n")).toBe(true); + expect(line.slice(0, -1)).not.toContain("\n"); + expect(JSON.parse(line)).toEqual({ event: "user", message: { role: "user", content: "Read \"marker.txt\".\nReply exactly.\n" } }); + }); + + it("writes a run-owned agent with view_file only, no MCP servers and no skills, and refuses the four project directories", () => { + const config = agy.launch.sidecar!.config!; + expect(agy.launch.sidecar!.output).toBeNull(); + expect(config.name).toBe(".agents/agents/pstack-read-only.md"); + const front = config.content(lane()).split("---\n")[1]; + expect(front).toBe([ + "name: pstack-read-only", + "description: Read-only pstack lane with one file-reading tool.", + "tools:", + " - view_file", + "mainAgent: true", + "subagent: false", + "mcpServers: []", + "skills: []", + "", + ].join("\n")); + }); +}); + +describe("agy preflight", () => { + it("passes only when the lane's variant ID is a listed model, with fixed evidence", () => { + for (const effort of ["low", "medium", "high"] as const) { + expect(preflight(0, fixture("models-logged-in.txt"), "Fetching available models...\n", { effort })) + .toEqual({ kind: "passed", evidence: `model gemini-3.8-flash-${effort} listed` }); + } + }); + + it("reads exit 0 without the variant as an unavailable model", () => { + expect(preflight(0, fixture("models-logged-in.txt"), "", { model: "gemini-3.7-flash", effort: "default" })) + .toEqual({ kind: "unavailable-model" }); + // The label column and a prefix of an ID are not the ID. + expect(preflight(0, "gemini-3.8-flash-medium-preview\tGemini\nother\tgemini-3.8-flash-medium\n")).toEqual({ kind: "unavailable-model" }); + expect(preflight(0, "")).toEqual({ kind: "unavailable-model" }); + }); + + it("reads exit 1 with the sign-in message as unauthenticated and anything else as inconclusive", () => { + expect(preflight(1, "", fixture("models-logged-out.stderr.txt"))).toEqual({ kind: "unauthenticated" }); + expect(preflight(2, "", fixture("models-logged-out.stderr.txt"))).toEqual({ kind: "inconclusive" }); + expect(preflight(1, "", "Error: network unreachable")).toEqual({ kind: "inconclusive" }); + expect(preflight(1, fixture("models-logged-in.txt"))).toEqual({ kind: "inconclusive" }); + }); +}); + +describe("agy parse", () => { + it("takes the reply from the result event, without its trailing newline, with session and usage", () => { + expect(agy.parse(run({ stdout: fixture("stream-complete.jsonl") }), lane())).toEqual({ + kind: "complete", + text: "00000000-0000-0000-0000-000000000000", + metadata: { + reportedModel: null, + sessionId: "00000000-0000-0000-0000-000000000000", + usage: { inputTokens: 25048, cachedInputTokens: 0, outputTokens: 310, reasoningTokens: 221, totalTokens: 25358 }, + costUsd: null, + }, + }); + expect(agy.proof).toMatchObject({ report: "absent" }); + expect(agy.proof.matches(lane(), "gemini-3.8-flash-medium")).toBe(false); + }); + + it("reads an exit-0 denial as a failure, with or without a preamble", () => { + expect(agy.parse(run({ stdout: fixture("stream-denied-shell.jsonl"), stderr: fixture("stream-denied-shell.stderr.txt") }), lane())) + .toMatchObject({ kind: "provider-failure", status: "child-failed", message: "agy denied a tool that needed approval", metadata: { usage: { outputTokens: 215 } } }); + const result = (body: object) => `${JSON.stringify({ event: "result", result: body })}\n`; + expect(agy.parse(run({ stdout: result({ response: "Let me check.\n", denied_actions: [{ action: "command" }] }) }), lane())) + .toMatchObject({ kind: "provider-failure", status: "child-failed", message: "agy denied a tool that needed approval" }); + }); + + it("reads the interrupted result as cancelled", () => { + expect(agy.parse(run({ exitCode: 1, stdout: fixture("stream-sigint.jsonl"), stderr: "error: interrupted\n" }), lane())) + .toMatchObject({ kind: "provider-failure", status: "cancelled", message: "agy reported the turn was interrupted" }); + }); + + it("leaves an error result, an empty response and unreadable output to the runner", () => { + expect(agy.parse(run({ exitCode: 1, stdout: fixture("stream-unknown-model.jsonl"), stderr: fixture("stream-unknown-model.stderr.txt") }), lane())) + .toEqual({ kind: "malformed", message: "agy result reported an error" }); + const result = (body: object) => `${JSON.stringify({ event: "result", result: body })}\n`; + expect(agy.parse(run({ stdout: result({ status: "SUCCESS", response: "\n" }) }), lane())) + .toEqual({ kind: "malformed", message: "agy result has an empty response" }); + expect(agy.parse(run({ stdout: "" }), lane())).toEqual({ kind: "malformed", message: "agy wrote no result event" }); + expect(agy.parse(run({ stdout: "PINEAPPLE\n" }), lane())).toEqual({ kind: "malformed", message: "agy emitted a non-JSON event" }); + }); +}); diff --git a/plugins/pstack/skills/poteto-mode/scripts/runner/providers/agy.ts b/plugins/pstack/skills/poteto-mode/scripts/runner/providers/agy.ts new file mode 100644 index 00000000..c51bf5b8 --- /dev/null +++ b/plugins/pstack/skills/poteto-mode/scripts/runner/providers/agy.ts @@ -0,0 +1,140 @@ +import { adapter, type LaunchFacts, type ModelRun, type ParseResult, type ProviderMetadata } from "../registry/contract.ts"; +import { BASE_ALLOWLIST, BASE_PREFIXES, boundedOutput, finiteNumber, nullableString, object, type JsonObject } from "./shared.ts"; + +// Measured with Antigravity CLI 1.3.6. agy loads 59 tools by default, among them web search, subagents, `schedule`, +// `send_message` and the browser. Under this agent, loaded from the run-owned `--add-dir`, the declared tools drop to +// `view_file` and `manage_task`. An unknown agent name exits 1, so a missing file fails closed. +const AGENT = "pstack-read-only"; +const AGENT_FILE = `--- +name: ${AGENT} +description: Read-only pstack lane with one file-reading tool. +tools: + - view_file +mainAgent: true +subagent: false +mcpServers: [] +skills: [] +--- +You are a read-only agent. +`; + +// `init.model` repeats `--model` and `result` names no model, so the proof is the pinned argv. +function metadata(result: JsonObject): ProviderMetadata { + const raw = object(result.usage); + const usage = { + inputTokens: finiteNumber(raw?.input_tokens), + cachedInputTokens: finiteNumber(raw?.cache_read_tokens), + outputTokens: finiteNumber(raw?.output_tokens), + reasoningTokens: finiteNumber(raw?.thinking_tokens), + totalTokens: finiteNumber(raw?.total_tokens), + }; + return { + reportedModel: null, + sessionId: nullableString(result.conversation_id), + usage: Object.values(usage).some((value) => value !== undefined) ? usage : null, + costUsd: null, + }; +} + +// stream-json stdout is NDJSON: `init`, `step_update` text deltas, then one `result`. +function parse(run: ModelRun): ParseResult { + let result: JsonObject | null = null; + for (const line of run.stdout.split("\n")) { + if (line.trim().length === 0) continue; + let raw: unknown; + try { + raw = JSON.parse(line); + } catch { + return { kind: "malformed", message: "agy emitted a non-JSON event" }; + } + const event = object(raw); + if (event?.event === "result") result = object(event.result); + } + if (result === null) return { kind: "malformed", message: "agy wrote no result event" }; + const found = metadata(result); + const error = nullableString(result.error); + // SIGINT and SIGTERM both exit 1 with this error. + if (error === "interrupted") { + return { kind: "provider-failure", status: "cancelled", message: "agy reported the turn was interrupted", metadata: found }; + } + // After an error exit run.ts classifies the output, so `invalid model selection` is unavailable-model. + if (error !== null) return { kind: "malformed", message: "agy result reported an error" }; + const response = typeof result.response === "string" ? result.response.replace(/\n$/, "") : ""; + // Headless agy denies the first tool that needs approval and ends the turn with exit 0 and status SUCCESS, so + // any response before the denial is a preamble, not an answer. + if (Array.isArray(result.denied_actions) && result.denied_actions.length > 0) { + return { kind: "provider-failure", status: "child-failed", message: "agy denied a tool that needed approval", metadata: found }; + } + if (response.length === 0) return { kind: "malformed", message: "agy result has an empty response" }; + return { kind: "complete", text: response, metadata: found }; +} + +function sidecarDirectory(facts: LaunchFacts): string { + const directory = facts.providerSidecar?.directory; + if (directory === undefined) throw new Error("agy argv needs the lane's provider sidecar directory"); + return directory; +} + +export const agy = adapter({ + id: "agy", + executable: "agy", + // A variant ID with a conflicting `--effort` exits 1, so argv passes no `--effort`. + effortIn: "model-id", + preflight: { + argv: () => ["models"], + retryAfterUnauthenticatedMs: null, + // Piped `agy models` prints one `<id>\t<label>` line per model the account can use. + classify(run, lane) { + if (run.exitCode === 0) { + const listed = run.stdout.split("\n").some((line) => line.split("\t")[0] === lane.cliModel); + return listed ? { kind: "passed", evidence: `model ${lane.cliModel} listed` } : { kind: "unavailable-model" }; + } + return run.exitCode === 1 && /Please sign in/.test(boundedOutput(run)) ? { kind: "unauthenticated" } : { kind: "inconclusive" }; + }, + }, + launch: { + prompt: "stdin", + // agy reads one JSON line per turn on stdin. It has no prompt-file flag, and `-p` takes the prompt as an argument. + stdinEnvelope: (prompt) => `${JSON.stringify({ event: "user", message: { role: "user", content: prompt } })}\n`, + sidecar: { output: null, config: { name: `.agents/agents/${AGENT}.md`, content: () => AGENT_FILE } }, + // The default mode's approval policy denies writes and every shell command. `accept-edits` writes only through + // the edit tool and denies shell, and `--sandbox` blocks shell writes even in the cwd, so no mode is a bounded writer. + supports: ["read-only"], + needsSharedGitDir: () => false, + argv: (lane, facts) => [ + "--input-format", + "stream-json", + "--output-format", + "stream-json", + "--model", + lane.cliModel, + "--add-dir", + sidecarDirectory(facts), + "--agent", + AGENT, + ], + }, + parse: (run) => parse(run), + proof: { report: "absent", matches: () => false }, + // The login is in the keychain, found through HOME. GEMINI_API_KEY did not authenticate and is not passed. + environment: () => ({ kind: "allowlist", names: BASE_ALLOWLIST, prefixes: BASE_PREFIXES }), + docs: { + name: "Antigravity CLI (`agy`)", + hostPrerequisites: [], + statePaths: [ + "`~/.gemini/antigravity-cli/`, where the CLI writes conversations, `brain/`, logs, caches and `conversation_summaries.db`; the login is in the macOS keychain", + ], + notes: [ + "Effort is part of the model ID. The runner passes the CLI model ID that the effort table gives for the requested effort and never passes `--effort`, because a variant ID with a conflicting `--effort` exits 1. A bare `gemini-3.8-flash` runs as `gemini-3.8-flash-high`. Issue #37 asked for Gemini 3.7 Flash, but `agy models` did not list it for the measured account. The runner registers Gemini 3.8 Flash and never maps a 3.7 request to it.", + "Read-only only. A lane runs agy's default mode, whose approval policy, not an OS sandbox, denies the write tool and every shell command. `--mode accept-edits` writes only through the edit tool and denies shell, and `--sandbox` blocks shell writes even in the cwd, so no agy mode is a bounded writer. The runner refuses an `isolated-write` lane before launch.", + "Headless agy cannot ask for approval. The first tool that needs it is denied, and the turn ends with exit 0, `status: \"SUCCESS\"` and a `denied_actions` list. Any `response` text is a preamble written before the denied tool, so the runner records a non-empty `denied_actions` as `child-failed`/70 whatever the response says.", + "Each lane writes a run-owned agent file, `.agents/agents/pstack-read-only.md`, in its private `<receipt>.provider/` directory (0700, the file 0600), and runs `--add-dir <that directory> --agent pstack-read-only`. The agent declares only `view_file`, no MCP servers and no skills. The lane keeps `view_file` and `manage_task` and loses web search, `read_url_content`, subagents, `schedule`, `send_message` and the browser. If the file is missing, agy exits 1 with `unknown agent`.", + "An agy lane runs in any directory, and the configuration it finds there applies as it is. agy runs the hooks and MCP servers in `.agents/`, `.agent/`, `_agents/` and `_agent/` directories in the cwd and each parent up to the git root, or up to `/` outside a repository, with no trust check, and loads their rules and skills. So a lane whose cwd is under the home directory and outside any git repository, or in a repository rooted at the home directory, also loads the home directory's own `~/.agents/` and its siblings. No flag stops them. A repository agent named `pstack-read-only` replaces the run-owned one.", + "No flag turns off the operator's own configuration. `~/.gemini/GEMINI.md` loads as a global rule, and host skills and agy plugin hooks under `~/.gemini/` still load in every lane. The run-owned agent limits only the lane's tools.", + "The lane passes the base allowlist and nothing else. The login is in the macOS keychain, found through `HOME`, so a run-owned `HOME` would be logged out. `GEMINI_API_KEY` did not authenticate and is not passed.", + "The preflight runs `agy models` and passes when it exits 0 and a line's first tab-separated field is the lane's CLI model ID. Exit 0 without that line is `unavailable-model`/69. Exit 1 with `Please sign in` is `unauthenticated`/77. Any other output is `child-failed`/70.", + "The prompt goes to stdin as one stream-json line, `{\"event\":\"user\",\"message\":{\"role\":\"user\",\"content\":<prompt>}}`, with `--input-format stream-json --output-format stream-json`. The reply is the `result` event's `response` without its trailing newline. The session is `conversation_id`, and usage is `usage`, with `thinking_tokens` as reasoning and `cache_read_tokens` as cached input. agy reports no cost.", + "agy never reports the served model. `init.model` repeats `--model`, and only an internal log names the resolved model, so the proof is the pinned argv. An unknown model exits 1 with `invalid model selection`, which is `unavailable-model`/69. A `result` error of `interrupted`, after SIGINT or SIGTERM, is `cancelled`/130.", + ], + }, +}); diff --git a/plugins/pstack/skills/poteto-mode/scripts/runner/providers/amp.test.ts b/plugins/pstack/skills/poteto-mode/scripts/runner/providers/amp.test.ts new file mode 100644 index 00000000..68158927 --- /dev/null +++ b/plugins/pstack/skills/poteto-mode/scripts/runner/providers/amp.test.ts @@ -0,0 +1,142 @@ +import { describe, expect, it } from "bun:test"; +import { readFileSync } from "node:fs"; +import { join } from "node:path"; +import { access } from "../registry/contract.ts"; +import type { ModelRun } from "../registry/contract.ts"; +import type { LaneRequest } from "../registry/index.ts"; +import { adapterSuite, testFacts, testLane } from "./adapter-suite.ts"; +import { amp } from "./amp.ts"; +import { BASE_ALLOWLIST, BASE_PREFIXES } from "./shared.ts"; + +adapterSuite(amp); + +// Sanitized from Amp CLI 0.0.1792094445-g3475f5 captures: structure kept, identity and host paths removed. +const fixture = (name: string): string => readFileSync(join(import.meta.dir, "fixtures/amp", name), "utf8"); + +const lane = (request: Partial<LaneRequest> = {}) => + testLane({ provider: "amp", model: "deepseek-v4.1-flash", effort: "default", ...request }); +const run = (overrides: Partial<ModelRun>): ModelRun => ({ exitCode: 0, stdout: "", stderr: "", sidecar: null, ...overrides }); +const preflight = (exitCode: number, stdout: string, stderr = "") => amp.preflight.classify({ exitCode, stdout, stderr }, lane()); +const NO_PROOF = { reportedModel: null, costUsd: null }; + +describe("amp launch", () => { + it("runs the pinned official mode in execute mode with the run-owned settings, private and labeled", () => { + expect(amp.launch.argv(lane(), testFacts(amp))).toEqual([ + "-x", + "--stream-json", + "-m", + "deepseek-v4.1-flash", + "--settings-file", + "/tmp/receipt.json.provider/settings.json", + "--visibility", + "private", + "--no-ide", + "--no-notifications", + "-l", + "pstack-runner", + ]); + expect(amp.launch.prompt).toBe("stdin"); + expect(amp.launch.sidecar!.output).toBeNull(); + }); + + it("refuses an isolated-write lane and efforts the mode does not list", () => { + expect(() => lane({ mode: "isolated-write" })) + .toThrow("provider amp does not support isolated-write; it supports only: read-only"); + expect(() => lane({ effort: "max" })).toThrow("effort max is not offered for amp:deepseek-v4.1-flash; choose one of: default"); + }); + + it("writes the captured lockdown settings", () => { + const config = amp.launch.sidecar!.config!; + expect(config.name).toBe("settings.json"); + expect(JSON.parse(config.content(lane()))).toEqual(JSON.parse(fixture("lockdown-settings.json"))); + }); + + it("passes only AMP_API_KEY and turns off guidance files", () => { + expect(amp.environment(access("read-only"), "/tmp/receipt.json.provider")).toEqual({ + kind: "allowlist", + names: [...BASE_ALLOWLIST, "AMP_API_KEY"], + prefixes: BASE_PREFIXES, + set: { AMP_IGNORE_GUIDANCE_FILES: "*" }, + }); + }); +}); + +describe("amp preflight", () => { + it("passes a signed-in usage report with a fixed summary", () => { + expect(preflight(0, fixture("usage-logged-in.stdout.txt"))).toEqual({ kind: "passed", evidence: "authenticated" }); + expect(preflight(1, fixture("usage-logged-in.stdout.txt"))).toEqual({ kind: "inconclusive" }); + }); + + it("reads the missing-key error at exit 1 as unauthenticated", () => { + expect(preflight(1, "", fixture("usage-logged-out.stderr.txt"))).toEqual({ kind: "unauthenticated" }); + }); + + it("reads any other output as inconclusive", () => { + for (const [exitCode, stdout, stderr] of [ + [0, "", ""], + [0, "Would you like to log in to Amp? [(y)es, (n)o]", ""], + [1, "", fixture("unknown-mode.stderr.txt")], + [2, "", fixture("usage-logged-out.stderr.txt")], + ] as const) { + expect([stdout, stderr, preflight(exitCode, stdout, stderr)]).toEqual([stdout, stderr, { kind: "inconclusive" }]); + } + }); +}); + +describe("amp parse", () => { + it("takes the reply from the result event and sums the assistant usage", () => { + expect(amp.parse(run({ stdout: fixture("stream-complete-readonly-marker.jsonl") }), lane())).toEqual({ + kind: "complete", + text: "39d942e7-da69-4370-863f-d2468a9fd511", + metadata: { + ...NO_PROOF, + sessionId: "T-<THREAD_ID>", + usage: { inputTokens: 0, cachedInputTokens: 1920, cacheCreationInputTokens: 8749, outputTokens: 259 }, + }, + }); + }); + + it("completes a turn whose tools were rejected as ordinary tool results", () => { + expect(amp.parse(run({ stdout: fixture("stream-tool-rejected-by-permission.jsonl") }), lane())) + .toMatchObject({ kind: "complete", text: expect.stringContaining("blocked by your permission rules") }); + }); + + it("reads an exit-0 error result as child-failed with its error text", () => { + const parsed = amp.parse(run({ stdout: fixture("stream-error-subscription-only-exit0.jsonl") }), lane()); + expect(parsed).toMatchObject({ + kind: "provider-failure", + status: "child-failed", + message: expect.stringMatching(/^Claude Usage is set to Subscription Only\./), + metadata: { ...NO_PROOF, sessionId: "T-<THREAD_ID>", usage: null }, + }); + }); + + it("reads the SIGINT/SIGTERM cancellation result as cancelled", () => { + expect(amp.parse(run({ stdout: fixture("stream-cancel-sigint.jsonl") }), lane())).toEqual({ + kind: "provider-failure", + status: "cancelled", + message: "User cancelled (SIGINT/SIGTERM)", + metadata: { ...NO_PROOF, sessionId: "T-<THREAD_ID>", usage: null }, + }); + }); + + it("never returns text without a successful result event", () => { + expect(amp.parse(run({ exitCode: 1, stderr: fixture("unknown-mode.stderr.txt") }), lane())) + .toEqual({ kind: "malformed", message: "amp stream did not contain a result event" }); + expect(amp.parse(run({ stdout: "not json\n" }), lane())).toEqual({ kind: "malformed", message: "amp emitted a non-JSON event" }); + const result = (fields: object) => `${JSON.stringify({ type: "result", session_id: "T-1", ...fields })}\n`; + expect(amp.parse(run({ stdout: result({ subtype: "success", result: "x" }) }), lane())) + .toEqual({ kind: "malformed", message: "amp result did not contain a valid terminal status" }); + expect(amp.parse(run({ stdout: result({ subtype: "success", is_error: false, result: "" }) }), lane())) + .toEqual({ kind: "malformed", message: "amp result did not contain final text" }); + expect(amp.parse(run({ stdout: result({ subtype: "error_max_turns", is_error: false, result: "x" }) }), lane())) + .toEqual({ kind: "malformed", message: "amp result subtype was error_max_turns" }); + expect(amp.parse(run({ stdout: result({ subtype: "error_during_execution", is_error: true }) }), lane())) + .toMatchObject({ kind: "provider-failure", status: "child-failed", message: "amp reported error_during_execution" }); + }); + + it("never claims a served model", () => { + expect(amp.proof).toMatchObject({ report: "absent" }); + expect(amp.proof.matches(lane(), "deepseek-v4.1-flash")).toBe(false); + }); +}); diff --git a/plugins/pstack/skills/poteto-mode/scripts/runner/providers/amp.ts b/plugins/pstack/skills/poteto-mode/scripts/runner/providers/amp.ts new file mode 100644 index 00000000..b5b6a3b2 --- /dev/null +++ b/plugins/pstack/skills/poteto-mode/scripts/runner/providers/amp.ts @@ -0,0 +1,150 @@ +import { adapter, type LaunchFacts, type ParseResult } from "../registry/contract.ts"; +import type { NormalizedUsage } from "../types.ts"; +import { BASE_ALLOWLIST, BASE_PREFIXES, normalizedUsage, nullableString, object, type JsonObject } from "./shared.ts"; + +// Read is the only enabled tool, and every other tool is rejected. The file also turns off MCP servers, both skill +// sources, updates and notifications. +const LOCKDOWN = { + "amp.mcpServers": {}, + "amp.skills.disableClaudeCodeSkills": true, + "amp.skills.disableGlobalAgentsSkills": true, + "amp.tools.enable": ["Read"], + "amp.permissions": [ + { tool: "Read", action: "allow" }, + { tool: "create_file", action: "reject" }, + { tool: "edit_file", action: "reject" }, + { tool: "apply_patch", action: "reject" }, + { tool: "shell_command", action: "reject" }, + { tool: "code_exec", action: "reject" }, + { tool: "*", action: "reject" }, + ], + "amp.updates.mode": "disabled", + "amp.notifications.enabled": false, +}; + +// SIGINT and SIGTERM both exit 0 with this result error. +const CANCELLED = "User cancelled (SIGINT/SIGTERM)"; + +function summed(entries: readonly NormalizedUsage[]): NormalizedUsage | null { + if (entries.length === 0) return null; + const total: Record<string, number> = {}; + for (const entry of entries) { + for (const [key, value] of Object.entries(entry)) { + if (value !== undefined) total[key] = (total[key] ?? 0) + value; + } + } + return total; +} + +// Claude Code-style stream JSON. The result event decides, because failures and cancellations exit 0. +function parse(stdout: string): ParseResult { + let result: JsonObject | null = null; + const usage: NormalizedUsage[] = []; + for (const line of stdout.split("\n")) { + if (line.trim().length === 0) continue; + let raw: unknown; + try { + raw = JSON.parse(line); + } catch { + return { kind: "malformed", message: "amp emitted a non-JSON event" }; + } + const event = object(raw); + if (event?.type === "assistant") { + const entry = normalizedUsage(object(event.message)?.usage); + if (entry !== null) usage.push(entry); + } + if (event?.type === "result") result = event; + } + if (result === null) return { kind: "malformed", message: "amp stream did not contain a result event" }; + if (typeof result.is_error !== "boolean") { + return { kind: "malformed", message: "amp result did not contain a valid terminal status" }; + } + const metadata = { reportedModel: null, sessionId: nullableString(result.session_id), usage: summed(usage), costUsd: null }; + if (result.is_error) { + const message = nullableString(result.error) ?? `amp reported ${String(result.subtype)}`; + return { kind: "provider-failure", status: message === CANCELLED ? "cancelled" : "child-failed", message, metadata }; + } + if (result.subtype !== "success") return { kind: "malformed", message: `amp result subtype was ${String(result.subtype)}` }; + const text = nullableString(result.result); + if (text === null) return { kind: "malformed", message: "amp result did not contain final text" }; + return { kind: "complete", text, metadata }; +} + +function settingsPath(facts: LaunchFacts): string { + const config = facts.providerSidecar?.config ?? null; + if (config === null) throw new Error("amp argv needs the lane's provider settings path"); + return config; +} + +export const amp = adapter({ + id: "amp", + executable: "amp", + effortIn: "model-id", + preflight: { + argv: () => ["usage"], + retryAfterUnauthenticatedMs: null, + // The first line holds the account email and the next one the credit balance, so the evidence is fixed. + classify(run) { + if (run.exitCode === 0 && run.stdout.startsWith("Signed in as ")) return { kind: "passed", evidence: "authenticated" }; + if (run.exitCode === 1 && `${run.stdout}\n${run.stderr}`.includes("Invalid or missing API key")) { + return { kind: "unauthenticated" }; + } + return { kind: "inconclusive" }; + }, + }, + launch: { + prompt: "stdin", + sidecar: { output: null, config: { name: "settings.json", content: () => `${JSON.stringify(LOCKDOWN, null, 2)}\n` } }, + // No sandbox: a shell write escaped the cwd whenever the shell was allowed, so Amp has no bounded writer. + supports: ["read-only"], + needsSharedGitDir: () => false, + argv: (lane, facts) => [ + "-x", + "--stream-json", + "-m", + lane.cliModel, + "--settings-file", + settingsPath(facts), + // Every captured thread was private; a repository's visibility mapping was not tested. + "--visibility", + "private", + "--no-ide", + "--no-notifications", + "-l", + "pstack-runner", + ], + }, + parse: (run) => parse(run.stdout), + // The stream never names the model. The pinned official mode is the evidence. + proof: { report: "absent", matches: () => false }, + // No AMP_ prefix: AMP_SETTINGS_FILE, AMP_EXECUTOR (`orb` runs the turn remotely), AMP_INITIAL_AGENT_MODE_KEY and + // AMP_URL would change the settings, where the turn runs, the mode or the service. + environment: () => ({ + kind: "allowlist", + names: [...BASE_ALLOWLIST, "AMP_API_KEY"], + prefixes: BASE_PREFIXES, + set: { AMP_IGNORE_GUIDANCE_FILES: "*" }, + }), + docs: { + name: "Amp CLI", + hostPrerequisites: [], + statePaths: [ + "`~/.local/share/amp/` (the login in `secrets.json`, accounts, session and recent threads) and `~/.cache/amp/` (logs, downloaded plugins and skills)", + ], + notes: [ + "Effort is part of the model ID. The model is an official Amp agent mode, such as `deepseek-v4.1-flash`, from Amp's own `official-modes` plugin, which Amp serves to every user. Where a mode has effort variants, the variant is part of its name (for example `gpt-6-astra-low` to `gpt-6-astra-max`). An unregistered Amp model must be a mode name with the `default` effort. An unknown mode exits 1 with a generic error and ends `child-failed`/70.", + "Read-only only. Amp has no OS sandbox and auto-approves its `ask` rules in execute mode, and a shell write escaped the cwd whenever the shell was allowed, so no Amp mode is a bounded writer. The runner refuses an `isolated-write` lane before launch.", + "A read-only lane is a run-owned `--settings-file` in its private `<receipt>.provider/` directory (0700). It enables only the `Read` tool and rejects every other tool, which turns off the shell, edits, subagents, web tools, code execution, thread tools and `manage_amp`. It also turns off MCP servers, Claude Code and `~/.agents` skills, updates and notifications. The lane sets `AMP_IGNORE_GUIDANCE_FILES=*`, so `AGENTS.md` in the cwd and its parents does not load. `Read` can read any path on the host. Measured with Amp CLI 0.0.1792094445-g3475f5 and 0.0.1792123242-g03edc7.", + "Amp's own `official-modes` plugin supplies the pinned modes and runs its code in every lane. Amp serves and updates it with no version pin: its cache hash changed from `a1f1ef61c4dd011d` to `cb61cf1a7f4be96c` between 2026-10-15 and 2026-10-16.", + "System plugins in `$XDG_CONFIG_HOME/amp/plugins` or `~/.config/amp/plugins` load in every lane, and the lane keeps an inherited `XDG_CONFIG_HOME`. A plugin is Bun code that runs as the user, and Amp has no production switch that turns plugins off (`AMP_DISABLE_PLUGINS` is ignored outside development builds). Measured with Amp CLI 0.0.1792123242-g03edc7: a system plugin ran at import and in the `session.start`, `agent.start` and `agent.end` hooks. One that called `configuration.update` at startup rewrote the lane's settings file to enable every tool with an allow-all rule, after which the model had 29 tools and ran a shell write in the cwd. One that registered the key `deepseek-v4.1-flash` replaced the official mode's prompt and tools, while the argv still named that mode.", + "Personal and workspace plugins come from ampcode.com, load in every lane and use the same plugin API as system plugins. The measured account had none.", + "A machine `managed-settings.json` (`/Library/Application Support/ampcode/` on macOS, `/etc/ampcode/` on Linux) and server-managed workspace settings apply above the user settings that `--settings-file` replaces, per Amp's settings docs, so they can change the enabled tools and permissions. Neither was measured: the Mac had no managed file, and the account has no workspace.", + "A repository's `.amp/settings.json` in the cwd overrides the run-owned settings, and its `.amp/plugins/*.ts` runs at startup with no trust check. Measured with Amp CLI 0.0.1792094445-g3475f5: a cwd `.amp/settings.json` turned `create_file` and `shell_command` back on, and a `.amp/plugins/` plugin ran. A workspace MCP server in `.amp/` waited for approval and did not start.", + "The environment is an allowlist that adds only `AMP_API_KEY`. No other `AMP_` name passes: `AMP_SETTINGS_FILE`, `AMP_EXECUTOR`, `AMP_INITIAL_AGENT_MODE_KEY` and `AMP_URL` would change the settings, where the turn runs, the mode or the service.", + "The preflight is `amp usage`, run without a terminal. It passes when it exits 0 and starts with `Signed in as `, and records the fixed evidence `authenticated`, because the output holds the account email and credit balance. Exit 1 with `Invalid or missing API key` is `unauthenticated`/77, and any other output is `child-failed`/70. A logged-out Amp command other than `amp usage` can open a browser login on its own, which is why the runner never starts a turn without a passing preflight.", + "The reply, session and token usage come from the stream JSON on stdout: the reply is the `result` event's text, the session is its `session_id` (`T-<uuid>`, also the ampcode.com thread ID), and the usage sums each assistant message's `usage`. The stream has no cost and never names the model, so the proof is the pinned mode in the argv (`pinned-argv`).", + "Failures and cancellations can exit 0, so the `result` event decides. `is_error: true` with `User cancelled (SIGINT/SIGTERM)` is `cancelled`/130, and any other `is_error` result is `child-failed`/70 with its error text, for example a Claude mode on a subscription-only account. A rejected tool comes back to the model as an ordinary tool result, and the turn can still complete.", + "Each run is an ampcode.com thread holding the prompt, tool inputs and outputs, the hostname and the working directory. The lane passes `--visibility private` and the label `pstack-runner`; every captured thread was private and archived. Turns spend Amp credits, or a linked ChatGPT subscription for GPT modes.", + ], + }, +}); diff --git a/plugins/pstack/skills/poteto-mode/scripts/runner/providers/claude.test.ts b/plugins/pstack/skills/poteto-mode/scripts/runner/providers/claude.test.ts index 183ba98a..eb4333a2 100644 --- a/plugins/pstack/skills/poteto-mode/scripts/runner/providers/claude.test.ts +++ b/plugins/pstack/skills/poteto-mode/scripts/runner/providers/claude.test.ts @@ -1,4 +1,5 @@ import { describe, expect, it } from "bun:test"; +import { access } from "../registry/contract.ts"; import type { LaneRequest } from "../registry/index.ts"; import { adapterSuite, testLane } from "./adapter-suite.ts"; import { claude } from "./claude.ts"; @@ -9,8 +10,8 @@ adapterSuite(claude); const lane = (request: Partial<LaneRequest> = {}) => testLane({ provider: "claude", model: "fable", ...request }); const argv = (request: Partial<LaneRequest> = {}, sharedGitDir: string | null = null) => - claude.launch.argv(lane(request), { sharedGitDir }); -const parse = (stdout: string) => claude.parse({ exitCode: 0, stdout, stderr: "" }, lane()); + claude.launch.argv(lane(request), { sharedGitDir, providerSidecar: null }); +const parse = (stdout: string) => claude.parse({ exitCode: 0, stdout, stderr: "", sidecar: null }, lane()); describe("claude launch", () => { it("passes Claude model, effort, permissions, and no-recursion controls", () => { @@ -76,8 +77,8 @@ describe("claude launch", () => { }); it("asks run.ts for the shared git directory only for a writer", () => { - expect(claude.launch.needsSharedGitDir({ mode: "isolated-write" })).toBe(true); - expect(claude.launch.needsSharedGitDir({ mode: "read-only" })).toBe(false); + expect(claude.launch.needsSharedGitDir(access("isolated-write"))).toBe(true); + expect(claude.launch.needsSharedGitDir(access("read-only"))).toBe(false); }); it("covers low, medium, and high", () => { @@ -193,6 +194,17 @@ describe("claude model proof", () => { expect(claude.proof.matches(pinned, "claude-sonnet-x-20260101")).toBe(true); expect(claude.proof.matches(pinned, "claude-sonnet-y")).toBe(false); }); + + it("holds the registered Opus 5.5 pin to its own model and Sonnet to its own revisions", () => { + const pin = lane({ model: "claude-opus-5-5" }); + expect(pin.family?.name).toBe("opus-5-5"); + expect(claude.proof.matches(pin, "claude-opus-5-5")).toBe(true); + expect(claude.proof.matches(pin, "claude-opus-5-6")).toBe(false); + expect(claude.proof.matches(pin, "claude-opus-5")).toBe(false); + const sonnet = lane({ model: "sonnet" }); + expect(claude.proof.matches(sonnet, "claude-sonnet-5-5")).toBe(true); + expect(claude.proof.matches(sonnet, "claude-opus-5-5")).toBe(false); + }); }); describe("claude preflight", () => { @@ -204,7 +216,7 @@ describe("claude preflight", () => { expect(classify(0, JSON.stringify({ loggedIn: false }))).toEqual({ kind: "unauthenticated" }); expect(classify(0, "[]")).toEqual({ kind: "unauthenticated" }); expect(classify(0, "model claude-x is not supported")).toEqual({ kind: "unavailable-model" }); - expect(claude.preflight.argv(lane())).toEqual(["auth", "status", "--json"]); + expect(claude.preflight.argv(lane(), null)).toEqual(["auth", "status", "--json"]); expect(claude.preflight.retryAfterUnauthenticatedMs).toBeNull(); }); }); diff --git a/plugins/pstack/skills/poteto-mode/scripts/runner/providers/claude.ts b/plugins/pstack/skills/poteto-mode/scripts/runner/providers/claude.ts index 1cd4aa73..d1faaf96 100644 --- a/plugins/pstack/skills/poteto-mode/scripts/runner/providers/claude.ts +++ b/plugins/pstack/skills/poteto-mode/scripts/runner/providers/claude.ts @@ -93,6 +93,7 @@ function parse(stdout: string, lane: Lane): ParseResult { export const claude = adapter({ id: "claude", executable: "claude", + effortIn: "flag", preflight: { argv: () => ["auth", "status", "--json"], retryAfterUnauthenticatedMs: null, @@ -102,6 +103,8 @@ export const claude = adapter({ }, launch: { prompt: "stdin", + sidecar: null, + supports: ["read-only", "isolated-write"], needsSharedGitDir: (access) => access.mode === "isolated-write", argv: (lane, facts) => [ "-p", diff --git a/plugins/pstack/skills/poteto-mode/scripts/runner/providers/codex.test.ts b/plugins/pstack/skills/poteto-mode/scripts/runner/providers/codex.test.ts index 8c55aa23..057a77b7 100644 --- a/plugins/pstack/skills/poteto-mode/scripts/runner/providers/codex.test.ts +++ b/plugins/pstack/skills/poteto-mode/scripts/runner/providers/codex.test.ts @@ -7,8 +7,8 @@ adapterSuite(codex); const lane = (request: Partial<LaneRequest> = {}) => testLane({ provider: "codex", model: "gpt-5.6-sol", ...request }); -const argv = (request: Partial<LaneRequest> = {}) => codex.launch.argv(lane(request), { sharedGitDir: null }); -const parse = (stdout: string) => codex.parse({ exitCode: 0, stdout, stderr: "" }, lane()); +const argv = (request: Partial<LaneRequest> = {}) => codex.launch.argv(lane(request), { sharedGitDir: null, providerSidecar: null }); +const parse = (stdout: string) => codex.parse({ exitCode: 0, stdout, stderr: "", sidecar: null }, lane()); describe("codex launch", () => { it("pins Codex model, effort, sandbox, cwd, and JSONL output", () => { @@ -57,6 +57,18 @@ describe("codex launch", () => { ); }); + it("keeps an explicitly bounded writer on workspace-write", () => { + expect(argv({ mode: "isolated-write", writerSandbox: "bounded" })).toEqual(argv({ mode: "isolated-write" })); + }); + + it("gives only an opted-in writer the full-access sandbox", () => { + const bounded = argv({ mode: "isolated-write" }); + const full = argv({ mode: "isolated-write", writerSandbox: "full-access" }); + const at = bounded.indexOf("workspace-write"); + expect(full).toEqual([...bounded.slice(0, at), "danger-full-access", ...bounded.slice(at + 1)]); + expect(full.slice(at - 1, at + 1)).toEqual(["--sandbox", "danger-full-access"]); + }); + it("covers low, medium, and high", () => { for (const effort of ["low", "medium", "high"] as const) { expect(argv({ effort })).toEqual( @@ -87,6 +99,7 @@ describe("codex parse", () => { }), ].join("\n"), stderr: "model: gpt-5.6-sol\nreasoning effort: max\n", + sidecar: null, }, lane()); expect(parsed).toMatchObject({ kind: "complete", @@ -122,7 +135,7 @@ describe("codex preflight", () => { expect(classify(0, "", "Logged in using an API key")).toEqual({ kind: "passed", evidence: "authenticated" }); expect(classify(1, "Logged in using ChatGPT")).toEqual({ kind: "unauthenticated" }); expect(classify(0, "status unknown")).toEqual({ kind: "unauthenticated" }); - expect(codex.preflight.argv(lane())).toEqual(["login", "status"]); + expect(codex.preflight.argv(lane(), null)).toEqual(["login", "status"]); expect(codex.preflight.retryAfterUnauthenticatedMs).toBeNull(); }); }); diff --git a/plugins/pstack/skills/poteto-mode/scripts/runner/providers/codex.ts b/plugins/pstack/skills/poteto-mode/scripts/runner/providers/codex.ts index 5016afd5..bb63d0fe 100644 --- a/plugins/pstack/skills/poteto-mode/scripts/runner/providers/codex.ts +++ b/plugins/pstack/skills/poteto-mode/scripts/runner/providers/codex.ts @@ -1,4 +1,4 @@ -import { adapter, type ParseResult } from "../registry/contract.ts"; +import { type Access, adapter, type ParseResult } from "../registry/contract.ts"; import type { NormalizedUsage } from "../types.ts"; import { effortArgs, @@ -52,9 +52,15 @@ function parse(stdout: string): ParseResult { }; } +function sandbox(access: Access): string { + if (access.mode === "read-only") return "read-only"; + return access.sandbox === "full-access" ? "danger-full-access" : "workspace-write"; +} + export const codex = adapter({ id: "codex", executable: "codex", + effortIn: "flag", preflight: { argv: () => ["login", "status"], retryAfterUnauthenticatedMs: null, @@ -64,6 +70,8 @@ export const codex = adapter({ }, launch: { prompt: "stdin", + sidecar: null, + supports: ["read-only", "isolated-write", "isolated-write-full-access"], needsSharedGitDir: () => false, argv: (lane) => [ "exec", @@ -71,7 +79,7 @@ export const codex = adapter({ lane.model, ...effortArgs(lane, ["--config", `model_reasoning_effort=${JSON.stringify(lane.effort)}`]), "--sandbox", - lane.access.mode === "read-only" ? "read-only" : "workspace-write", + sandbox(lane.access), "--cd", lane.cwd, "--skip-git-repo-check", diff --git a/plugins/pstack/skills/poteto-mode/scripts/runner/providers/cursor.test.ts b/plugins/pstack/skills/poteto-mode/scripts/runner/providers/cursor.test.ts new file mode 100644 index 00000000..f05e43e5 --- /dev/null +++ b/plugins/pstack/skills/poteto-mode/scripts/runner/providers/cursor.test.ts @@ -0,0 +1,144 @@ +import { describe, expect, it } from "bun:test"; +import { readFileSync } from "node:fs"; +import { join } from "node:path"; +import type { ModelRun } from "../registry/contract.ts"; +import type { LaneRequest } from "../registry/index.ts"; +import { modelFailureStatus } from "../types.ts"; +import { adapterSuite, testFacts, testLane } from "./adapter-suite.ts"; +import { cursor } from "./cursor.ts"; + +adapterSuite(cursor); + +// Sanitized from Cursor CLI 2026.10.14-0f9edde captures: structure kept, IDs and host paths replaced. +const fixture = (name: string): string => readFileSync(join(import.meta.dir, "fixtures/cursor", name), "utf8"); + +const lane = (request: Partial<LaneRequest> = {}) => testLane({ provider: "cursor", model: "composer-2.5", effort: "default", ...request }); +const run = (overrides: Partial<ModelRun>): ModelRun => ({ exitCode: 0, stdout: "", stderr: "", sidecar: null, ...overrides }); +const preflight = (exitCode: number, stdout: string, stderr = "", request: Partial<LaneRequest> = {}) => + cursor.preflight.classify({ exitCode, stdout, stderr }, lane(request)); + +describe("cursor launch", () => { + it("runs ask mode in the sandbox with the effort's model ID and no bypass flag", () => { + expect(cursor.launch.argv(lane({ model: "gemini-3.7-flash", effort: "high" }), testFacts(cursor))).toEqual([ + "-p", + "--output-format", + "stream-json", + "--model", + "gemini-3.7-flash-high", + "--mode", + "ask", + "--sandbox", + "enabled", + "--trust", + ]); + for (const flag of ["--force", "-f", "--yolo", "--approve-mcps"]) { + expect(cursor.launch.argv(lane(), testFacts(cursor))).not.toContain(flag); + } + expect(cursor.executable).toBe("cursor-agent"); + }); + + it("maps each Gemini 3.7 Flash effort to its listed model ID", () => { + for (const effort of ["low", "medium", "high"] as const) { + expect(lane({ model: "gemini-3.7-flash", effort }).cliModel).toBe(`gemini-3.7-flash-${effort}`); + } + }); + + it("refuses an isolated-write lane and efforts a family does not list", () => { + expect(() => lane({ mode: "isolated-write" })) + .toThrow("provider cursor does not support isolated-write; it supports only: read-only"); + expect(() => lane({ model: "gemini-3.7-flash", effort: "max" })).toThrow("choose one of: low, medium, high"); + }); + + it("writes the lockdown config into the run-owned CURSOR_CONFIG_DIR and passes only the two login names", () => { + const config = cursor.launch.sidecar!.config!; + expect(cursor.launch.sidecar!.output).toBeNull(); + expect(config.name).toBe("cli-config.json"); + expect(JSON.parse(config.content(lane()))).toEqual({ + version: 1, + permissions: { allow: [], deny: ["Write(**)", "Shell(*)", "Mcp(*:*)"] }, + approvalMode: "allowlist", + sandbox: { mode: "enabled" }, + autoAcceptWebSearch: false, + webFetchDomainAllowlist: [], + }); + const policy = cursor.environment(lane().access, "/tmp/receipt.json.provider"); + expect(policy).toMatchObject({ kind: "allowlist", set: { CURSOR_CONFIG_DIR: "/tmp/receipt.json.provider" } }); + if (policy.kind !== "allowlist") throw new Error("expected an allowlist"); + expect(policy.names.filter((name) => name.startsWith("CURSOR_"))).toEqual(["CURSOR_API_KEY", "CURSOR_AUTH_TOKEN"]); + expect(policy.prefixes.some((prefix) => "CURSOR_API_ENDPOINT".startsWith(prefix))).toBe(false); + }); +}); + +describe("cursor preflight", () => { + it("passes only when the lane's model ID starts a listed line, with fixed evidence", () => { + expect(preflight(0, fixture("models-logged-in.txt"))).toEqual({ kind: "passed", evidence: "model composer-2.5 listed" }); + expect(preflight(0, fixture("models-logged-in.txt"), "", { model: "gemini-3.7-flash", effort: "high" })) + .toEqual({ kind: "passed", evidence: "model gemini-3.7-flash-high listed" }); + }); + + it("reads a model list without the ID as an unavailable model", () => { + expect(preflight(0, fixture("models-logged-in.txt"), "", { model: "claude-opus-9-9-max" })).toEqual({ kind: "unavailable-model" }); + // A label or a longer ID is not the ID. + expect(preflight(0, "Available models\n\ncomposer-2.5-fast - Composer 2.5 Fast\nauto - composer-2.5 - x\n")) + .toEqual({ kind: "unavailable-model" }); + }); + + it("reads exit 1 with Authentication required as unauthenticated and never reads empty output as logged out", () => { + expect(preflight(1, "", fixture("models-logged-out.stderr.txt"))).toEqual({ kind: "unauthenticated" }); + expect(preflight(1, "", fixture("models-stored-login-unusable.stderr.txt"))).toEqual({ kind: "unauthenticated" }); + for (const [exitCode, stdout, stderr] of [ + [0, "", ""], + [2, "", ""], + [1, "", ""], + [2, "", fixture("models-logged-out.stderr.txt")], + [1, "", "Error: network unreachable"], + [0, "Logged in as <EMAIL>", ""], + ] as const) { + expect([exitCode, stdout, stderr, preflight(exitCode, stdout, stderr)]).toEqual([exitCode, stdout, stderr, { kind: "inconclusive" }]); + } + }); +}); + +describe("cursor parse", () => { + it("takes the reply from the last assistant event, with session and usage", () => { + expect(cursor.parse(run({ stdout: fixture("stream-complete-readonly.jsonl") }), lane())).toEqual({ + kind: "complete", + text: "00000000-0000-0000-0000-000000000009", + metadata: { + reportedModel: null, + sessionId: "00000000-0000-0000-0000-000000000007", + usage: { inputTokens: 6, cachedInputTokens: 30872, cacheCreationInputTokens: 11610, outputTokens: 112 }, + costUsd: null, + }, + }); + expect(cursor.proof).toMatchObject({ report: "absent" }); + expect(cursor.proof.matches(lane(), "composer-2.5")).toBe(false); + }); + + it("completes a turn whose tool was refused, with the last segment and not the joined result", () => { + const parsed = cursor.parse(run({ stdout: fixture("refused-shell-ask-sandbox.jsonl") }), lane()); + expect(parsed).toMatchObject({ kind: "complete", metadata: { sessionId: "00000000-0000-0000-0000-000000000005" } }); + if (parsed.kind !== "complete") throw new Error("expected complete"); + expect(parsed.text.startsWith("The shell command did not run.")).toBe(true); + expect(parsed.text).not.toContain("Running the requested shell command once."); + }); + + it("reads an error result as child-failed with its text", () => { + const stdout = `${JSON.stringify({ type: "assistant", message: { content: [{ type: "text", text: "partial" }] } })}\n` + + `${JSON.stringify({ type: "result", subtype: "error", is_error: true, result: "rate limited", session_id: "s" })}\n`; + expect(cursor.parse(run({ stdout }), lane())).toMatchObject({ kind: "provider-failure", status: "child-failed", message: "rate limited", metadata: { sessionId: "s" } }); + }); + + it("leaves a signal exit, an unknown model and unreadable output to the runner", () => { + expect(cursor.parse(run({ exitCode: 130, stdout: fixture("cancel-sigint.jsonl"), stderr: fixture("cancel-sigint.stderr.txt") }), lane())) + .toEqual({ kind: "malformed", message: "cursor stream did not contain a result event" }); + expect(cursor.parse(run({ exitCode: 143, stdout: fixture("cancel-sigterm.jsonl") }), lane())) + .toEqual({ kind: "malformed", message: "cursor stream did not contain a result event" }); + expect(cursor.parse(run({ exitCode: 1, stderr: fixture("unknown-model.stderr.txt") }), lane())) + .toEqual({ kind: "malformed", message: "cursor stream did not contain a result event" }); + expect(modelFailureStatus(fixture("unknown-model.stderr.txt"))).toBe("unavailable-model"); + expect(cursor.parse(run({ stdout: "PINEAPPLE\n" }), lane())).toEqual({ kind: "malformed", message: "cursor emitted a non-JSON event" }); + const result = `${JSON.stringify({ type: "result", subtype: "success", is_error: false, result: "joined" })}\n`; + expect(cursor.parse(run({ stdout: result }), lane())).toEqual({ kind: "malformed", message: "cursor stream did not contain an assistant reply" }); + }); +}); diff --git a/plugins/pstack/skills/poteto-mode/scripts/runner/providers/cursor.ts b/plugins/pstack/skills/poteto-mode/scripts/runner/providers/cursor.ts new file mode 100644 index 00000000..72d378d6 --- /dev/null +++ b/plugins/pstack/skills/poteto-mode/scripts/runner/providers/cursor.ts @@ -0,0 +1,149 @@ +import { adapter, type ParseResult, type ProviderMetadata } from "../registry/contract.ts"; +import { BASE_ALLOWLIST, BASE_PREFIXES, boundedOutput, finiteNumber, nullableString, object, type JsonObject } from "./shared.ts"; + +// Measured with Cursor CLI 2026.10.14-0f9edde. `--mode ask --sandbox enabled` still let MCP, plugin and web tools +// run until this run-owned config denied them: `Mcp(*:*)` blocks every MCP and plugin call, and with no allow rule +// nothing approves a web tool. A repository's `.cursor/cli.json` replaces this deny list; the catalog notes say so. +const LOCKDOWN = { + version: 1, + permissions: { allow: [], deny: ["Write(**)", "Shell(*)", "Mcp(*:*)"] }, + approvalMode: "allowlist", + sandbox: { mode: "enabled" }, + autoAcceptWebSearch: false, + webFetchDomainAllowlist: [], +}; + +function metadata(result: JsonObject): ProviderMetadata { + const raw = object(result.usage); + const usage = { + inputTokens: finiteNumber(raw?.inputTokens), + cachedInputTokens: finiteNumber(raw?.cacheReadTokens), + cacheCreationInputTokens: finiteNumber(raw?.cacheWriteTokens), + outputTokens: finiteNumber(raw?.outputTokens), + }; + return { + reportedModel: null, + sessionId: nullableString(result.session_id), + usage: Object.values(usage).some((value) => value !== undefined) ? usage : null, + costUsd: null, + }; +} + +function assistantText(event: JsonObject): string { + const content = object(event.message)?.content; + if (!Array.isArray(content)) return ""; + return content.map((part) => object(part)).filter((part) => part?.type === "text") + .map((part) => (typeof part?.text === "string" ? part.text : "")).join(""); +} + +// Stream JSON on stdout. The result event decides. Its `result` joins every assistant text segment, such as +// "I'll read the file." before a tool call, so the reply is the last assistant event's text. +function parse(stdout: string): ParseResult { + let result: JsonObject | null = null; + let reply: string | null = null; + for (const line of stdout.split("\n")) { + if (line.trim().length === 0) continue; + let raw: unknown; + try { + raw = JSON.parse(line); + } catch { + return { kind: "malformed", message: "cursor emitted a non-JSON event" }; + } + const event = object(raw); + if (event?.type === "assistant") reply = assistantText(event); + if (event?.type === "result") result = event; + } + // SIGINT (130) and SIGTERM (143) leave no result event. + if (result === null) return { kind: "malformed", message: "cursor stream did not contain a result event" }; + if (typeof result.is_error !== "boolean") { + return { kind: "malformed", message: "cursor result did not contain a valid terminal status" }; + } + const found = metadata(result); + if (result.is_error) { + const message = nullableString(result.result) ?? `cursor reported ${String(result.subtype)}`; + return { kind: "provider-failure", status: "child-failed", message, metadata: found }; + } + if (result.subtype !== "success") return { kind: "malformed", message: `cursor result subtype was ${String(result.subtype)}` }; + if (reply === null || reply.length === 0) return { kind: "malformed", message: "cursor stream did not contain an assistant reply" }; + return { kind: "complete", text: reply, metadata: found }; +} + +export const cursor = adapter({ + id: "cursor", + // Cursor also installs `agent`, which can resolve to another CLI on PATH, such as Grok's. + executable: "cursor-agent", + effortIn: "model-id", + preflight: { + // `status` reports a stored login as authenticated while every model call fails, and it ignores + // CURSOR_API_KEY, so the preflight asks for the model list a turn would use. + argv: () => ["models"], + retryAfterUnauthenticatedMs: null, + classify(run, lane) { + if (run.exitCode === 0 && run.stdout.startsWith("Available models")) { + const listed = run.stdout.split("\n").some((line) => line.startsWith(`${lane.cliModel} - `)); + return listed ? { kind: "passed", evidence: `model ${lane.cliModel} listed` } : { kind: "unavailable-model" }; + } + // Empty output is never unauthenticated (#94). + return run.exitCode === 1 && /^Error: Authentication required/m.test(boundedOutput(run)) + ? { kind: "unauthenticated" } + : { kind: "inconclusive" }; + }, + }, + launch: { + prompt: "stdin", + sidecar: { output: null, config: { name: "cli-config.json", content: () => `${JSON.stringify(LOCKDOWN, null, 2)}\n` } }, + // In agent mode the edit tool wrote outside the cwd under every flag combination, so no mode is a bounded writer. + supports: ["read-only"], + needsSharedGitDir: () => false, + argv: (lane) => [ + "-p", + "--output-format", + "stream-json", + "--model", + lane.cliModel, + // The only combination that refused every edit and shell request, including `ls`. + "--mode", + "ask", + "--sandbox", + "enabled", + // A new directory otherwise exits 1 with `Workspace Trust Required`. + "--trust", + ], + }, + parse: (run) => parse(run.stdout), + // The init event's `model` is a display label such as `Claude Opus 5.5 300K High`, not the ID. + proof: { report: "absent", matches: () => false }, + // CURSOR_CONFIG_DIR keeps `-p` from rewriting `~/.cursor/cli-config.json`. No CURSOR_ prefix: + // CURSOR_API_ENDPOINT, CURSOR_API_BASE_URL, CURSOR_SANDBOX_POLICY_DIR and CURSOR_FORCED_SHELL_EGRESS + // change the service, the sandbox or the egress. + environment: (_access, sidecarDir) => ({ + kind: "allowlist", + names: [...BASE_ALLOWLIST, "CURSOR_API_KEY", "CURSOR_AUTH_TOKEN"], + prefixes: BASE_PREFIXES, + set: { CURSOR_CONFIG_DIR: `${sidecarDir}` }, + }), + docs: { + name: "Cursor CLI (`cursor-agent`)", + hostPrerequisites: [], + statePaths: [ + "`~/.cursor/`, where the CLI reads `mcp.json`, `hooks.json`, plugins and skills and writes `chats/`, `projects/<slug>/`, `statsig-cache.json` and `agent-cli-state.json`; the stored login is in the macOS keychain", + "`<receipt>.provider/`, the run-owned `CURSOR_CONFIG_DIR`, where the CLI writes `chats/` and `statsig-cache.json` and rewrites `cli-config.json` at 0644 with an identity block, keeping the run-owned permissions", + "`~/Library/Caches/cursor-compile-cache/`, `$TMPDIR/cursor-agent-logs-<uid>/` and `~/.npm/_npx/`, which it writes", + ], + notes: [ + "The executable is `cursor-agent`. Cursor also installs `agent`, and on a host where another CLI's `agent` comes first on PATH (Grok's, for example), `agent status --format json` exits 2 with empty stdout, which looked like an empty Cursor status in #94.", + "Effort is part of the model ID. The runner passes the CLI model ID that the effort table gives for the requested effort, such as `gemini-3.7-flash-high`. An unregistered Cursor model must be a CLI model ID with the `default` effort. An unknown model exits 1 with `Cannot use this model: <id>`, which is `unavailable-model`/69.", + "Read-only only. A lane runs `-p --output-format stream-json --mode ask --sandbox enabled --trust` with the prompt on stdin. Only `--mode ask --sandbox enabled` refused every edit and shell request, including `ls`; the CLI's permission check enforces it, and the Read tool still works. In agent mode the edit tool wrote outside the cwd under every flag combination, so no Cursor mode is a bounded writer. The runner refuses an `isolated-write` lane before launch. `--trust` is required in a new directory. The lane never passes `--force`, `--yolo` or `--approve-mcps`.", + "Each lane sets `CURSOR_CONFIG_DIR` to its private `<receipt>.provider/` directory (0700) and writes `cli-config.json` there (0600) before the preflight. The config denies `Write(**)`, `Shell(*)` and `Mcp(*:*)`, sets `approvalMode: \"allowlist\"` with no allow rules, enables the sandbox, and turns off web-search auto-approval and the web-fetch domain allowlist. MCP and plugin tools (such as the GitHub, Gmail and Calendar plugins) are still listed to the model, but every call is blocked. Web search and web fetch are refused, because nothing approves them. A Task subagent runs with the same limits. Without the run-owned directory, every `-p` run rewrites `~/.cursor/cli-config.json` with its default model and an identity block.", + "A Cursor lane runs in any repository, and the repository's configuration applies as it is. `.cursor/cli.json`, which the CLI reads in every directory from the git root down to the cwd, replaces the run-owned deny list. A measured one that allowed `Shell(*)` let the lane run shell commands, and an invalid one stops the run with exit 1. `AGENTS.md`, `CLAUDE.md`, `.cursor/rules/*.mdc` and `.cursorrules` load as always-applied rules.", + "Hooks run unsandboxed: the CLI's command executor runs them with sandbox policy `insecure_none` whatever `--sandbox` is set to (batch-158 containment captures, 2026-10-16). No flag stops them, and no command lists them without running them. Enterprise hooks load from a fixed path that follows neither `CURSOR_CONFIG_DIR` nor `HOME`: `/Library/Application Support/Cursor/hooks.json` on macOS and `/etc/cursor/hooks.json` on Linux. For a team account, a turn fetches the team's hooks from Cursor's server, writes them to `~/.cursor/managed/team_<id>/hooks/` and runs them. User hooks load from `~/.cursor/hooks.json` and `~/.claude/settings.json`. Repository hooks load from `.cursor/hooks.json`, `.claude/settings.json` and `.claude/settings.local.json`.", + "No flag turns off the operator's own configuration. Host skills (from `~/.cursor`, `~/.claude`, `~/.codex` and `~/.agents`), account plugins and `~/.cursor/mcp.json` still load. The run-owned deny list limits only what their tools can do.", + "The environment is an allowlist that adds only `CURSOR_API_KEY` and `CURSOR_AUTH_TOKEN`, the two logins the CLI's own error names, and sets `CURSOR_CONFIG_DIR`. No other `CURSOR_` name passes: `CURSOR_API_ENDPOINT`, `CURSOR_API_BASE_URL`, `CURSOR_SANDBOX_POLICY_DIR`, `CURSOR_FORCED_SHELL_EGRESS` and the rest change the service, the sandbox or the egress. On the measured macOS host, a `cursor-agent login` run while `CURSOR_API_KEY` was set left a stored login that `status` reported as authenticated but that failed every model call once the variable was gone; signing out and in again with `env -u CURSOR_API_KEY cursor-agent login` fixed it.", + "The preflight runs `cursor-agent models`, not `status`: `status` reports a stored login as authenticated while every model call fails with `Authentication required`, and it ignores `CURSOR_API_KEY`. The preflight passes when `models` exits 0, its stdout starts with `Available models`, and a line starts with `<CLI model ID> - `, and records the fixed evidence `model <id> listed`. Exit 0 without that line is `unavailable-model`/69. Exit 1 with `Error: Authentication required` is `unauthenticated`/77. Any other output, including empty output, is `child-failed`/70.", + "The `result` event decides. A `success` result with `is_error: false` is complete, and the reply is the last `assistant` event's text, because the `result` field joins every assistant text segment of the turn. `is_error: true` is `child-failed`/70 with its text. A refused tool comes back to the model as a tool result, and the turn can still complete. SIGINT exits 130 and SIGTERM 143 with no `result` event. The session is `session_id`, and usage is `result.usage`, with `cacheReadTokens` as cached input and `cacheWriteTokens` as cache creation. Cursor reports no cost.", + "Cursor never reports the served model ID. The init event's `model` is a display label such as `Claude Opus 5.5 300K High`, so the proof is the pinned argv (`pinned-argv`).", + "Every turn starts a `worker-server` process in the workspace with the lane's environment, in an empty repository too and whether or not the model reads a file. The worker runs `npx -y typescript-language-server --stdio`, and npx runs a repository's own copy when it has one. In the batch-158 containment captures (Cursor CLI 2026.10.14-0f9edde, 2026-10-16), each of these ran repository code: a `node_modules/.bin/typescript-language-server`, a root `package.json` named `typescript-language-server` with a `bin`, an `.npmrc` `script-shell`, and an `.npmrc` `node-options`. The language server inherits the lane's environment plus `AGENT_CLI_*` and `npm_*` variables. A `package.json` `bin` run leaves a `file:` link to the repository in `~/.npm/_npx/`.", + "The worker and the language server stay running in the CLI's process group after the turn and exit after about five idle minutes. An unknown model exits before the worker starts.", + ], + }, +}); diff --git a/plugins/pstack/skills/poteto-mode/scripts/runner/providers/devin.test.ts b/plugins/pstack/skills/poteto-mode/scripts/runner/providers/devin.test.ts new file mode 100644 index 00000000..ef995317 --- /dev/null +++ b/plugins/pstack/skills/poteto-mode/scripts/runner/providers/devin.test.ts @@ -0,0 +1,156 @@ +import { describe, expect, it } from "bun:test"; +import { readFileSync } from "node:fs"; +import { join } from "node:path"; +import type { ModelRun } from "../registry/contract.ts"; +import type { LaneRequest } from "../registry/index.ts"; +import { adapterSuite, testFacts, testLane } from "./adapter-suite.ts"; +import { devin } from "./devin.ts"; + +adapterSuite(devin); + +// Sanitized from Devin CLI 3000.10.21 captures: structure kept, identity and host text removed. +const fixture = (name: string): string => readFileSync(join(import.meta.dir, "fixtures/devin", name), "utf8"); + +const lane = (request: Partial<LaneRequest> = {}) => testLane({ provider: "devin", model: "swe-2", effort: "medium", ...request }); +const run = (overrides: Partial<ModelRun>): ModelRun => ({ exitCode: 0, stdout: "", stderr: "", sidecar: null, ...overrides }); +const preflight = (stdout: string, exitCode = 0) => devin.preflight.classify({ exitCode, stdout, stderr: "" }, lane()); + +describe("devin launch", () => { + it("runs the effort's full variant ID read-only with the run-owned export and lockdown config", () => { + expect(devin.launch.argv(lane({ effort: "max" }), testFacts(devin))).toEqual([ + "-p", + "--prompt-file", + "/tmp/prompt.md", + "--model", + "swe-2-max", + "--permission-mode", + "auto", + "--respect-workspace-trust", + "false", + "--export", + "/tmp/receipt.json.provider/export.json", + "--config", + "/tmp/receipt.json.provider/lockdown.json", + ]); + const swe16 = lane({ model: "swe-1.6", effort: "default" }); + expect([swe16.model, swe16.cliModel]).toEqual(["swe-1.6", "swe-1-6"]); + expect(devin.launch.argv(swe16, testFacts(devin))).toEqual(expect.arrayContaining(["--model", "swe-1-6"])); + expect(devin.launch.argv(lane(), testFacts(devin))).not.toContain("--sandbox"); + }); + + it("refuses an isolated-write lane and efforts SWE-2 does not list", () => { + expect(() => lane({ mode: "isolated-write" })) + .toThrow("provider devin does not support isolated-write; it supports only: read-only"); + expect(() => lane({ effort: "low" })).toThrow("effort low is not offered for devin:swe-2; choose one of: medium, high, max"); + expect(() => lane({ model: "swe-1.6", effort: "max" })).toThrow("choose one of: default"); + }); + + it("writes a lockdown config that turns off subagents, every rule source, MCP, web search and fetch", () => { + const config = devin.launch.sidecar!.config!; + expect(config.name).toBe("lockdown.json"); + expect(JSON.parse(config.content(lane()))).toEqual({ + version: 1, + subagents_enabled: false, + read_config_from: { + agents_standard: false, + cursor: false, + windsurf: false, + claude: false, + copilot: false, + opencode: false, + zed: false, + }, + permissions: { + deny: ["mcp__*", "web_search", "webfetch", "run_subagent", "Fetch(https://*)", "Fetch(http://*)"], + }, + }); + expect(devin.launch.sidecar!.output).toBe("export.json"); + }); +}); + +describe("devin preflight", () => { + it("passes only a logged-in status and records a fixed summary", () => { + expect(preflight(fixture("auth-logged-in.txt"))).toEqual({ kind: "passed", evidence: "authenticated" }); + expect(preflight(fixture("auth-logged-in.txt"), 1)).toEqual({ kind: "inconclusive" }); + }); + + it("reads a logged-out status at exit 0 as unauthenticated", () => { + expect(preflight(fixture("auth-logged-out.txt"))).toEqual({ kind: "unauthenticated" }); + }); + + it("reads any other status as inconclusive", () => { + for (const stdout of ["", "Welcome to Devin CLI!\nLogged in (via Devin).", "logged in", "Error: network unreachable"]) { + expect([stdout, preflight(stdout)]).toEqual([stdout, { kind: "inconclusive" }]); + } + }); +}); + +describe("devin parse", () => { + it("takes the reply and proof from the export, not stdout", () => { + const parsed = devin.parse(run({ + stdout: "\u001b[1mWelcome to Devin CLI!\u001b[0m\n\nbanner text\n", + sidecar: fixture("export-swe-2-medium.json"), + }), lane()); + expect(parsed).toEqual({ + kind: "complete", + text: "MARK-fixture", + metadata: { + reportedModel: "swe-2-medium", + sessionId: "fixture-session", + usage: { inputTokens: 47019, cachedInputTokens: 31618, outputTokens: 100 }, + costUsd: null, + }, + }); + expect(devin.proof.matches(lane(), "swe-2-medium")).toBe(true); + for (const reported of ["swe-2", "SWE-2 Medium", "swe-2-max", "swe-2-medium-1"]) { + expect([reported, devin.proof.matches(lane(), reported)]).toEqual([reported, false]); + } + const swe16 = lane({ model: "swe-1.6", effort: "default" }); + const parsed16 = devin.parse(run({ sidecar: fixture("export-swe-1-6.json") }), swe16); + expect(parsed16).toMatchObject({ kind: "complete", metadata: { reportedModel: "swe-1-6" } }); + expect(devin.proof.matches(swe16, "swe-1-6")).toBe(true); + }); + + it("reads a rejected tool call's empty final message as a failure", () => { + expect(devin.parse(run({ stderr: fixture("rejected-tool.stderr.txt"), sidecar: fixture("export-rejected-tool.json") }), lane())) + .toEqual({ + kind: "provider-failure", + status: "child-failed", + message: "devin ended with an empty final message", + metadata: { + reportedModel: "swe-2-medium", + sessionId: "fixture-session", + usage: { inputTokens: 23455, cachedInputTokens: 0, outputTokens: 87 }, + costUsd: null, + }, + }); + }); + + it("reads an agent reply with the interruption text as a reply, not a cancellation", () => { + const sidecar = JSON.stringify({ session_id: "s", steps: [{ source: "agent", message: "[Response interrupted by user]", extra: { generation_model: "swe-2-medium" } }] }); + expect(devin.parse(run({ sidecar }), lane())).toMatchObject({ kind: "complete", text: "[Response interrupted by user]" }); + }); + + it("reads SIGINT's interrupted export and SIGTERM's exit 143 as cancelled", () => { + expect(devin.parse(run({ exitCode: 1, stdout: "The history of rivers is,", stderr: "Error: Operation canceled by user\n", sidecar: fixture("export-sigint.json") }), lane())) + .toMatchObject({ kind: "provider-failure", status: "cancelled", message: "devin reported the response was interrupted", metadata: { reportedModel: null, sessionId: "fixture-session" } }); + expect(devin.parse(run({ exitCode: 143, stdout: "The History of Rivers\n\nRivers are" }), lane())) + .toEqual({ + kind: "provider-failure", + status: "cancelled", + message: "devin exited with status 143 after SIGTERM", + metadata: { reportedModel: null, sessionId: null, usage: null, costUsd: null }, + }); + }); + + it("never returns stdout when the export is missing or unreadable", () => { + expect(devin.parse(run({ stdout: "MARK-fixture\n" }), lane())).toEqual({ kind: "malformed", message: "devin wrote no export" }); + expect(devin.parse(run({ exitCode: 1, stderr: fixture("unknown-model.stderr.txt") }), lane())) + .toEqual({ kind: "malformed", message: "devin wrote no export" }); + expect(devin.parse(run({ sidecar: "{" }), lane())).toEqual({ kind: "malformed", message: "devin export is not valid JSON" }); + expect(devin.parse(run({ sidecar: "{\"steps\":[]}" }), lane())).toEqual({ kind: "malformed", message: "devin export has no steps" }); + const userLast = JSON.stringify({ steps: [{ source: "user", message: "hello" }] }); + expect(devin.parse(run({ sidecar: userLast }), lane())) + .toEqual({ kind: "malformed", message: "devin export did not end with an agent step" }); + }); +}); diff --git a/plugins/pstack/skills/poteto-mode/scripts/runner/providers/devin.ts b/plugins/pstack/skills/poteto-mode/scripts/runner/providers/devin.ts new file mode 100644 index 00000000..6e12c766 --- /dev/null +++ b/plugins/pstack/skills/poteto-mode/scripts/runner/providers/devin.ts @@ -0,0 +1,144 @@ +import { adapter, type LaunchFacts, type ModelRun, type ParseResult, type ProviderMetadata } from "../registry/contract.ts"; +import { BASE_ALLOWLIST, BASE_PREFIXES, finiteNumber, nullableString, object, type JsonObject } from "./shared.ts"; + +// Devin loads the host's rules (such as CLAUDE.md), MCP servers, web search and subagents by default, +// and no flag turns them off. It rewrites the config file it is given, so run.ts writes this one fresh +// into each lane's private provider directory. Measured with Devin CLI 3000.10.21. +const LOCKDOWN = { + version: 1, + subagents_enabled: false, + read_config_from: { + agents_standard: false, + cursor: false, + windsurf: false, + claude: false, + copilot: false, + opencode: false, + zed: false, + }, + permissions: { + deny: ["mcp__*", "web_search", "webfetch", "run_subagent", "Fetch(https://*)", "Fetch(http://*)"], + }, +}; + +// SIGINT leaves an export whose last step is this system message; SIGTERM exits 143 with no export. +const INTERRUPTED = "[Response interrupted by user]"; +const SIGTERM_EXIT = 143; + +const NO_METADATA: ProviderMetadata = { reportedModel: null, sessionId: null, usage: null, costUsd: null }; + +function metadata(exported: JsonObject, last: JsonObject | null): ProviderMetadata { + const totals = object(exported.final_metrics); + const usage = { + inputTokens: finiteNumber(totals?.total_prompt_tokens), + cachedInputTokens: finiteNumber(totals?.total_cached_tokens), + outputTokens: finiteNumber(totals?.total_completion_tokens), + }; + return { + // `agent.model_name` is a label such as "SWE-2 Medium"; the generation model is the CLI model ID. + reportedModel: nullableString(object(last?.extra)?.generation_model), + sessionId: nullableString(exported.session_id), + usage: Object.values(usage).some((value) => value !== undefined) ? usage : null, + costUsd: null, + }; +} + +// The ATIF export is the result: stdout can start with a first-use welcome banner, and after a +// cancellation it holds a partial reply. +function parse(run: ModelRun): ParseResult { + if (run.exitCode === SIGTERM_EXIT) { + return { kind: "provider-failure", status: "cancelled", message: "devin exited with status 143 after SIGTERM", metadata: NO_METADATA }; + } + if (run.sidecar === null) return { kind: "malformed", message: "devin wrote no export" }; + let raw: unknown; + try { + raw = JSON.parse(run.sidecar); + } catch { + return { kind: "malformed", message: "devin export is not valid JSON" }; + } + const exported = object(raw); + const steps = Array.isArray(exported?.steps) ? exported.steps : []; + if (exported === null || steps.length === 0) return { kind: "malformed", message: "devin export has no steps" }; + const last = object(steps.at(-1)); + const found = metadata(exported, last); + // Devin writes the interruption as a system step; an agent reply with the same text is a reply. + if (last?.source === "system" && last.message === INTERRUPTED) { + return { kind: "provider-failure", status: "cancelled", message: "devin reported the response was interrupted", metadata: found }; + } + if (last?.source !== "agent") return { kind: "malformed", message: "devin export did not end with an agent step" }; + const text = nullableString(last.message); + if (text === null) { + // A rejected tool call exits 0 with an empty final message. + return { kind: "provider-failure", status: "child-failed", message: "devin ended with an empty final message", metadata: found }; + } + return { kind: "complete", text, metadata: found }; +} + +function sidecar(facts: LaunchFacts): { readonly output: string; readonly config: string } { + const paths = facts.providerSidecar; + if (paths?.output == null || paths.config === null) throw new Error("devin argv needs the lane's provider sidecar paths"); + return { output: paths.output, config: paths.config }; +} + +export const devin = adapter({ + id: "devin", + executable: "devin", + effortIn: "model-id", + preflight: { + argv: () => ["auth", "status"], + retryAfterUnauthenticatedMs: null, + // `devin auth status` exits 0 when logged out, so its first line decides. + classify(run) { + if (run.exitCode === 0 && /^Logged in \(via Devin\)\./.test(run.stdout)) { + return { kind: "passed", evidence: "authenticated" }; + } + return /^Not logged in\./.test(run.stdout) ? { kind: "unauthenticated" } : { kind: "inconclusive" }; + }, + }, + launch: { + prompt: "file", + sidecar: { output: "export.json", config: { name: "lockdown.json", content: () => `${JSON.stringify(LOCKDOWN, null, 2)}\n` } }, + // `--sandbox` forces Devin's autonomous mode, which lets shell commands write in the cwd while it + // rejects the edit tools, so Devin has no bounded writer mode. + supports: ["read-only"], + needsSharedGitDir: () => false, + argv(lane, facts) { + const paths = sidecar(facts); + return [ + "-p", + "--prompt-file", + lane.promptPath, + "--model", + lane.cliModel, + // Without --sandbox, auto mode's approval policy rejects edits and writing shell commands. + "--permission-mode", + "auto", + // Print mode refuses an untrusted cwd otherwise. + "--respect-workspace-trust", + "false", + "--export", + paths.output, + "--config", + paths.config, + ]; + }, + }, + parse: (run) => parse(run), + proof: { report: "required", matches: (lane, reported) => reported === lane.cliModel }, + // No DEVIN_* prefix: DEVIN_PERMISSION_MODE, DEVIN_SANDBOX, DEVIN_MODEL and DEVIN_REFUSAL_FALLBACK + // would widen the lane, change its model or add a model fallback. + environment: () => ({ kind: "allowlist", names: BASE_ALLOWLIST, prefixes: BASE_PREFIXES }), + docs: { + name: "Devin CLI", + hostPrerequisites: [], + statePaths: ["`~/.local/share/devin/` (the login, which the CLI reads) and the config, telemetry and log files it writes under the home"], + notes: [ + "Effort is part of the model ID. The runner passes the CLI model ID that the effort table gives for the requested effort. An unregistered Devin model must be a CLI model ID with the `default` effort.", + "Read-only only. A lane runs `--permission-mode auto` without `--sandbox`, so Devin's approval policy, not an OS sandbox, rejects edits and writing shell commands, as Claude's `plan` mode does. With `--sandbox`, approved shell commands write in the cwd while the edit tools are rejected, so no Devin mode is a bounded writer. The runner refuses an `isolated-write` lane before launch.", + "A Devin lane runs in any directory, and the configuration it finds there applies as it is. The lane passes `--respect-workspace-trust false`, so Devin loads the `.devin/config.json` hooks and permissions and the `.devin/mcp_config.json` MCP servers of a `.devin/` directory in the cwd or any parent up to the git root, or up to `/` outside a repository. So a lane whose cwd is under the home directory and outside any git repository also loads the home directory's own `~/.devin/`. No key turns them off.", + "Each lane gets a run-owned `--config` file in its private `<receipt>.provider/` directory (0700). It turns off subagents, every host rule source (such as `CLAUDE.md`) and MCP tools, and denies web search and the fetch tools. Devin still loads its own skills. Network isolation is not established: shell network access and the fetch denials were not measured.", + "The preflight passes when `devin auth status` exits 0 and starts with `Logged in (via Devin).`. It also exits 0 when logged out, so output starting with `Not logged in.` is `unauthenticated`/77, and any other output is `child-failed`/70.", + "The reply, session, token usage and model proof come from the `--export` ATIF file at `<receipt>.provider/export.json`, never from stdout. The proof is the last step's `extra.generation_model`, which must equal the CLI model ID. An empty final message, as after a rejected tool call, is `child-failed`/70. An interrupted response, or exit 143 after SIGTERM, is `cancelled`/130. `Error: Unknown model:` is `unavailable-model`/69.", + ], + }, +}); diff --git a/plugins/pstack/skills/poteto-mode/scripts/runner/providers/dsh-preflight.yml b/plugins/pstack/skills/poteto-mode/scripts/runner/providers/dsh-preflight.yml new file mode 100644 index 00000000..dfa13427 --- /dev/null +++ b/plugins/pstack/skills/poteto-mode/scripts/runner/providers/dsh-preflight.yml @@ -0,0 +1,9 @@ +# The dsh preflight turn, applied after the lane's patch: inference goes to a closed loopback port, so a stored +# or exported key ends in TRANSPORT and a missing one in MISSING_CREDENTIAL. Retries are off, and the lane's +# patch turns off the session title call and the session-log upload, so the turn ends in about a second and +# sends nothing. Measured with dsh 0.2.0-rc.2. +- id: llm-deepseek + config: + baseURL: http://127.0.0.1:1 +- id: llm-retry + disabled: true diff --git a/plugins/pstack/skills/poteto-mode/scripts/runner/providers/dsh.test.ts b/plugins/pstack/skills/poteto-mode/scripts/runner/providers/dsh.test.ts new file mode 100644 index 00000000..474f254c --- /dev/null +++ b/plugins/pstack/skills/poteto-mode/scripts/runner/providers/dsh.test.ts @@ -0,0 +1,180 @@ +import { describe, expect, it } from "bun:test"; +import { readFileSync } from "node:fs"; +import { homedir } from "node:os"; +import { join } from "node:path"; +import type { ModelRun } from "../registry/contract.ts"; +import type { LaneRequest } from "../registry/index.ts"; +import { modelFailureStatus } from "../types.ts"; +import { adapterSuite, testFacts, testLane } from "./adapter-suite.ts"; +import { dsh } from "./dsh.ts"; + +adapterSuite(dsh); + +// Sanitized from dsh 0.2.0-rc.2 captures: `real-*` and `preflight-transport` against DeepSeek, the rest +// against a local mock. Structure kept, paths, session IDs and markers replaced. +const fixture = (name: string): string => readFileSync(join(import.meta.dir, "fixtures/dsh", name), "utf8"); + +const lane = (request: Partial<LaneRequest> = {}) => testLane({ provider: "dsh", model: "deepseek-flash", effort: "max", ...request }); +const run = (overrides: Partial<ModelRun>): ModelRun => ({ exitCode: 0, stdout: "", stderr: "", sidecar: null, ...overrides }); +const preflight = (stdout: string, exitCode = 1) => dsh.preflight.classify({ exitCode, stdout, stderr: "" }, lane()); + +describe("dsh launch", () => { + it("reads the prompt from stdin with the run-owned patch before --json", () => { + expect(dsh.launch.argv(lane(), testFacts(dsh))).toEqual([ + "--profile", "headless", "--patch", "/tmp/receipt.json.provider/patch.yml", "--json", "-", + ]); + expect(dsh.launch.prompt).toBe("stdin"); + expect(dsh.launch.sidecar?.output).toBeNull(); + }); + + it("pins read-only mode, the stored login in place, the model and effort, and turns off the lockdown rows", () => { + const saved = process.env.DSH_HOME; + process.env.DSH_HOME = "/srv/dsh"; + const patch = dsh.launch.sidecar!.config!.content(lane({ model: "deepseek-v4-pro", effort: "low", cwd: "/tmp/a dir: #x" })); + if (saved === undefined) delete process.env.DSH_HOME; + else process.env.DSH_HOME = saved; + expect(patch).toStartWith([ + "- id: sandbox-policy", + " config:", + " mode: read-only", + " workspaceRoot: \"/tmp/a dir: #x\"", + "- id: approval", + " config:", + " policy: ask", + "- id: credentials", + " config:", + " path: \"/srv/dsh/.credentials.yaml\"", + "- id: agent-default-model", + " config:", + " provider: deepseek-official", + " model: \"deepseek-v4-pro\"", + " reasoningEffort: \"low\"", + ].join("\n") + "\n"); + for (const id of [ + "agent-instructions", "skill-filesystem", "tool-skill", "tool-web", "web", "web-search-deepseek", "web-fetch-http", + "tool-subagent", "tool-subagent-fork", "tool-subagent-control", "tool-subagent-list-agents", "tool-workflow", + "workflow-ptc", "plugin-package-inventory-deepseek", "session-title-llm", + ]) { + expect(patch).toContain(`- id: ${id}\n disabled: true\n`); + } + expect(patch).toEndWith("- id: session-log-deepseek\n config:\n enabled: false\n"); + // The setting's own line, so a credentials path that contains the word does not count. + expect(dsh.launch.sidecar!.config!.content(lane({ model: "deepseek-next", effort: "default" }))).not.toMatch(/^ {4}reasoningEffort:/m); + }); + + it("reads the stored login from the parent's DSH_HOME, resolved as dsh resolves it, else ~/.dsh", () => { + const saved = process.env.DSH_HOME; + try { + process.env.DSH_HOME = "/srv/dsh home"; + expect(dsh.launch.sidecar!.config!.content(lane())).toContain("- id: credentials\n config:\n path: \"/srv/dsh home/.credentials.yaml\"\n"); + for (const [value, home] of [ + [undefined, join(homedir(), ".dsh")], + [" \t", join(homedir(), ".dsh")], + ["~", homedir()], + ["~/dsh-alt", join(homedir(), "dsh-alt")], + ["dsh-rel", join(process.cwd(), "dsh-rel")], + ["~alice/dsh", join(process.cwd(), "~alice/dsh")], + ] as const) { + if (value === undefined) delete process.env.DSH_HOME; + else process.env.DSH_HOME = value; + expect(dsh.launch.sidecar!.config!.content(lane())).toContain(` path: ${JSON.stringify(join(home, ".credentials.yaml"))}\n`); + } + } finally { + if (saved === undefined) delete process.env.DSH_HOME; + else process.env.DSH_HOME = saved; + } + }); + + it("refuses an isolated-write lane and the medium effort for a registered family", () => { + expect(() => lane({ mode: "isolated-write" })).toThrow("provider dsh does not support isolated-write; it supports only: read-only"); + expect(() => lane({ effort: "medium" })).toThrow("effort medium is not offered for dsh:deepseek-flash; choose one of: low, high, max"); + }); + + it("passes DEEPSEEK_API_KEY, no other inherited DEEPSEEK_ or DSH_ name, and the provider directory as DSH_HOME", () => { + const policy = dsh.environment(lane().access, "/tmp/receipt.json.provider"); + if (policy.kind !== "allowlist") throw new Error("expected an allowlist"); + expect(policy.names.filter((name) => /^(DSH_|DEEPSEEK_)/.test(name))).toEqual(["DEEPSEEK_API_KEY"]); + expect(policy.prefixes.filter((prefix) => /^(DSH|DEEPSEEK)/.test(prefix))).toEqual([]); + expect(policy.set).toEqual({ DSH_HOME: "/tmp/receipt.json.provider" }); + }); +}); + +describe("dsh preflight", () => { + it("runs a positional task with the lane's patch, then the shipped preflight patch", () => { + const argv = dsh.preflight.argv(lane(), testFacts(dsh).providerSidecar); + expect(argv).toEqual([ + "--profile", "headless", "--patch", "/tmp/receipt.json.provider/patch.yml", + "--patch", join(import.meta.dir, "dsh-preflight.yml"), "--json", "preflight", + ]); + expect(readFileSync(argv[5]!, "utf8")).toContain("baseURL: http://127.0.0.1:1\n- id: llm-retry\n disabled: true\n"); + }); + + it("passes TRANSPORT with fixed evidence that the key is not validated", () => { + for (const name of ["preflight-transport.jsonl", "status-credential-present.jsonl"]) { + expect(preflight(fixture(name))).toEqual({ kind: "passed", evidence: "credential present; not validated" }); + } + }); + + it("reads MISSING_CREDENTIAL as unauthenticated and anything else as inconclusive", () => { + expect(preflight(fixture("status-logged-out.jsonl"))).toEqual({ kind: "unauthenticated" }); + for (const stdout of ["", "{", fixture("unsupported-effort.jsonl"), fixture("real-complete-flash-max.jsonl")]) { + expect([stdout, preflight(stdout)]).toEqual([stdout, { kind: "inconclusive" }]); + } + }); +}); + +describe("dsh parse", () => { + it("takes the final text after a completed turn, with summed step usage and the session ID", () => { + expect(dsh.parse(run({ stdout: fixture("real-complete-flash-max.jsonl") }), lane())).toEqual({ + kind: "complete", + text: "MARKER-FIXTURE", + metadata: { + reportedModel: null, + sessionId: "session-MARKER-FIXTURE", + usage: { + inputTokens: 3600, + outputTokens: 65, + totalTokens: 6993, + cachedInputTokens: 3328, + cacheCreationInputTokens: 0, + }, + costUsd: null, + }, + }); + expect(dsh.proof).toMatchObject({ report: "absent" }); + expect(dsh.proof.matches(lane(), "deepseek-flash")).toBe(false); + }); + + it("never completes SIGTERM's exit 0 with a partial final, and reads exit 130 as cancelled", () => { + expect(dsh.parse(run({ stdout: fixture("cancel-sigterm.jsonl") }), lane())) + .toEqual({ kind: "malformed", message: "dsh ended without a completed turn" }); + expect(dsh.parse(run({ exitCode: 130, stdout: fixture("cancel-sigint.jsonl") }), lane())) + .toMatchObject({ kind: "provider-failure", status: "cancelled" }); + }); + + it("reads a turn_end error as child-failed with its code and message", () => { + expect(dsh.parse(run({ exitCode: 1, stdout: fixture("unsupported-effort.jsonl"), stderr: fixture("unsupported-effort.stderr.txt") }), lane())) + .toMatchObject({ + kind: "provider-failure", + status: "child-failed", + message: "dsh turn failed: UNSUPPORTED_REASONING_EFFORT: provider \"deepseek-official\" model \"deepseek-flash\" does not support reasoning effort \"medium\"", + metadata: { sessionId: expect.stringMatching(/^session-/) }, + }); + }); + + it("leaves DeepSeek's unknown-model refusal to the runner's unavailable-model rule", () => { + const stderr = fixture("real-unknown-model.stderr.txt"); + expect(modelFailureStatus(stderr)).toBe("unavailable-model"); + expect(dsh.parse(run({ exitCode: 1, stdout: fixture("real-unknown-model.jsonl"), stderr }), lane())) + .toEqual({ kind: "malformed", message: "dsh named an unavailable model" }); + // The model's own reply text cannot trigger the rule: it is anchored to dsh's stderr line. + expect(modelFailureStatus("TOOL: The supported API model names are a, but you passed b")).toBe("child-failed"); + }); + + it("refuses a completed turn without final text and a non-JSON event", () => { + const completedEmpty = fixture("real-complete-flash-max.jsonl").replace("{\"type\":\"final\",\"text\":\"MARKER-FIXTURE\"}", "{\"type\":\"final\",\"text\":\"\"}"); + expect(dsh.parse(run({ stdout: completedEmpty }), lane())) + .toEqual({ kind: "malformed", message: "dsh completed the turn without final text" }); + expect(dsh.parse(run({ stdout: "Welcome\n" }), lane())).toEqual({ kind: "malformed", message: "dsh emitted a non-JSON event" }); + }); +}); diff --git a/plugins/pstack/skills/poteto-mode/scripts/runner/providers/dsh.ts b/plugins/pstack/skills/poteto-mode/scripts/runner/providers/dsh.ts new file mode 100644 index 00000000..d131430a --- /dev/null +++ b/plugins/pstack/skills/poteto-mode/scripts/runner/providers/dsh.ts @@ -0,0 +1,220 @@ +import { homedir } from "node:os"; +import { join, resolve } from "node:path"; +import { adapter, type ChildRun, type Lane, type LaunchFacts, type ModelRun, type ParseResult, type ProviderMetadata } from "../registry/contract.ts"; +import { modelFailureStatus, type NormalizedUsage } from "../types.ts"; +import { BASE_ALLOWLIST, BASE_PREFIXES, finiteNumber, nullableString, object, type JsonObject } from "./shared.ts"; + +// By default dsh loads host and project rules and skills, offers web, subagent and workflow tools, writes a +// session title with a second model call, and uploads the whole session log and the plugin inventory with +// every request. Measured with dsh 0.2.0-rc.2. +const DISABLED_ROWS = [ + "agent-instructions", + "skill-filesystem", + "tool-skill", + "tool-web", + "web", + "web-search-deepseek", + "web-fetch-http", + "tool-subagent", + "tool-subagent-fork", + "tool-subagent-control", + "tool-subagent-list-agents", + "tool-workflow", + "workflow-ptc", + "plugin-package-inventory-deepseek", + "session-title-llm", +]; + +// dsh-home-paths' resolveDshHome in dsh 0.2.0-rc.2: a blank value is unset, a leading `~`, `~/` or `~\` is the +// home directory, and a relative path resolves against the runner's cwd. +function dshHome(value: string | undefined): string { + const path = value !== undefined && value.trim().length > 0 ? value : join(homedir(), ".dsh"); + if (path === "~") return homedir(); + return resolve(path.startsWith("~/") || path.startsWith("~\\") ? join(homedir(), path.slice(2)) : path); +} + +// The lane runs on its own DSH_HOME, so no user layer or home-level file loads, and the stored login is read in +// place through the `credentials` row. A patch replaces a row's whole config. Pinning `sandbox-policy` and +// `approval` also overrides DSH_PERMISSION_MODE. JSON strings are YAML double-quoted scalars, so a path or model +// cannot be reinterpreted. +function patch(lane: Lane): string { + const quoted = JSON.stringify; + const credentials = join(dshHome(process.env.DSH_HOME), ".credentials.yaml"); + return [ + "- id: sandbox-policy", + " config:", + " mode: read-only", + ` workspaceRoot: ${quoted(lane.cwd)}`, + "- id: approval", + " config:", + " policy: ask", + "- id: credentials", + " config:", + ` path: ${quoted(credentials)}`, + "- id: agent-default-model", + " config:", + " provider: deepseek-official", + ` model: ${quoted(lane.cliModel)}`, + ...(lane.effort === "default" ? [] : [` reasoningEffort: ${quoted(lane.effort)}`]), + ...DISABLED_ROWS.flatMap((id) => [`- id: ${id}`, " disabled: true"]), + "- id: session-log-deepseek", + " config:", + " enabled: false", + "", + ].join("\n"); +} + +const PREFLIGHT_PATCH = join(import.meta.dir, "dsh-preflight.yml"); + +function lanePatch(sidecar: LaunchFacts["providerSidecar"]): string { + const config = sidecar?.config; + if (config == null) throw new Error("dsh argv needs the lane's patch path"); + return config; +} +const SIGINT_EXIT = 130; +const NO_METADATA: ProviderMetadata = { reportedModel: null, sessionId: null, usage: null, costUsd: null }; + +type Events = { readonly kind: "events"; readonly list: readonly JsonObject[] } | { readonly kind: "malformed" }; + +function events(stdout: string): Events { + const list: JsonObject[] = []; + for (const line of stdout.split("\n")) { + if (line.trim().length === 0) continue; + try { + const event = object(JSON.parse(line)); + if (event !== null) list.push(event); + } catch { + return { kind: "malformed" }; + } + } + return { kind: "events", list }; +} + +function turnEnd(list: readonly JsonObject[]): JsonObject | null { + const end = list.findLast((event) => event.type === "status" && event.phase === "turn_end"); + return end === undefined ? null : object(end.reason); +} + +function errorCode(reason: JsonObject | null): string | null { + return reason?.kind === "error" ? nullableString(object(reason.error)?.code) : null; +} + +const USAGE_FIELDS = [ + ["inputTokens", "inputTokens"], + ["outputTokens", "outputTokens"], + ["totalTokens", "totalTokens"], + ["cacheReadTokens", "cachedInputTokens"], + ["cacheWriteTokens", "cacheCreationInputTokens"], + ["reasoningTokens", "reasoningTokens"], +] as const; + +// Each step reports its own usage, so the turn's usage is the sum over `step_end` events. +function usage(list: readonly JsonObject[]): NormalizedUsage | null { + const sum: Partial<Record<keyof NormalizedUsage, number>> = {}; + for (const event of list) { + if (event.type !== "status" || event.phase !== "step_end") continue; + const step = object(event.usage); + for (const [from, to] of USAGE_FIELDS) { + const value = finiteNumber(step?.[from]); + if (value !== undefined) sum[to] = (sum[to] ?? 0) + value; + } + } + return Object.keys(sum).length === 0 ? null : sum; +} + +function parse(run: ModelRun): ParseResult { + if (run.exitCode === SIGINT_EXIT) { + return { kind: "provider-failure", status: "cancelled", message: "dsh exited with status 130 after SIGINT", metadata: NO_METADATA }; + } + const parsed = events(run.stdout); + if (parsed.kind === "malformed") return { kind: "malformed", message: "dsh emitted a non-JSON event" }; + const metadata: ProviderMetadata = { + reportedModel: null, + sessionId: nullableString(parsed.list.find((event) => event.type === "session")?.sessionId), + usage: usage(parsed.list), + costUsd: null, + }; + const reason = turnEnd(parsed.list); + if (reason?.kind === "error") { + // The run's nonzero exit lets run.ts read an unknown model from the same stderr line. + if (modelFailureStatus(run.stderr) === "unavailable-model") return { kind: "malformed", message: "dsh named an unavailable model" }; + const error = object(reason.error); + const message = `dsh turn failed: ${errorCode(reason) ?? "unknown code"}: ${nullableString(error?.message) ?? "no message"}`; + return { kind: "provider-failure", status: "child-failed", message, metadata }; + } + // SIGTERM exits 0 and still writes a `final` with the partial reply, but no `turn_end`. + if (reason?.kind !== "completed") return { kind: "malformed", message: "dsh ended without a completed turn" }; + const text = nullableString(parsed.list.findLast((event) => event.type === "final")?.text); + if (text === null) return { kind: "malformed", message: "dsh completed the turn without final text" }; + return { kind: "complete", text, metadata }; +} + +const VERSION = "0.2.0-rc.2"; + +// dsh has no status command. The preflight turn's error code says only whether a key is stored or exported. +function classify(run: ChildRun) { + const parsed = events(run.stdout); + const code = parsed.kind === "events" ? errorCode(turnEnd(parsed.list)) : null; + if (code === "TRANSPORT") return { kind: "passed", evidence: "credential present; not validated" } as const; + return code === "MISSING_CREDENTIAL" ? { kind: "unauthenticated" } as const : { kind: "inconclusive" } as const; +} + +export const dsh = adapter({ + id: "dsh", + executable: "dsh", + effortIn: "config", + preflight: { + // A positional task: the preflight gets no stdin. It runs on the lane's DSH_HOME with the lane's patch, so + // the loopback patch after it redirects the only route the lane can use. + argv: (lane, sidecar) => ["--profile", "headless", "--patch", lanePatch(sidecar), "--patch", PREFLIGHT_PATCH, "--json", "preflight"], + retryAfterUnauthenticatedMs: null, + classify, + // dsh ignores a patch row it does not have, so another version could drop the lockdown silently. + guard: { + argv: ["--version"], + check: (run) => run.stdout.trim() === VERSION ? null : { + status: "unavailable-cli", + message: `dsh ${run.stdout.trim()} is not the measured ${VERSION}; the adapter must be re-measured before it runs this version`, + }, + }, + }, + launch: { + prompt: "stdin", + sidecar: { output: null, config: { name: "patch.yml", content: patch } }, + // The Seatbelt `workspace-write` writer waits for a writer route in the live gate. + supports: ["read-only"], + needsSharedGitDir: () => false, + // Launcher flags must come before `--json`. + argv: (_lane, facts) => ["--profile", "headless", "--patch", lanePatch(facts.providerSidecar), "--json", "-"], + }, + parse: (run) => parse(run), + // The stream carries no model, and the session log records the requested model whatever the server ran. + proof: { report: "absent", matches: () => false }, + // No inherited DSH_* name or other DEEPSEEK_* name: DSH_PERMISSION_MODE widens the lane, DSH_HOME and its kin + // change what loads, and DEEPSEEK_BASE_URL redirects inference and the key with it. DSH_HOME is the lane's + // provider directory. + environment: (_access, sidecarDir) => { + if (sidecarDir === null) throw new Error("dsh needs the lane's provider directory as its DSH_HOME"); + return { kind: "allowlist", names: [...BASE_ALLOWLIST, "DEEPSEEK_API_KEY"], prefixes: BASE_PREFIXES, set: { DSH_HOME: sidecarDir } }; + }, + docs: { + name: "DeepSeek Harness", + hostPrerequisites: ["dsh 0.2.0-rc.2 exactly"], + statePaths: [ + "`<receipt>.provider/`, the lane's own `DSH_HOME`: the sessions (`sessions/<cwd-slug>/<session-id>/`), projection cache (`storages/session_projcache/`), `profiles/headless/` and `.anonymous-user-id` that dsh writes on every run", + "The stored key `<DSH_HOME, else ~/.dsh>/.credentials.yaml`, which the lane reads in place and does not copy", + ], + notes: [ + "Measured with dsh 0.2.0-rc.2, a developer preview. The runner checks `dsh --version` before the preflight and refuses any other version as `unavailable-cli`/69. Recheck on an upgrade: the config row IDs the patch names (dsh silently ignores a patch row it does not have, so a renamed row would drop that part of the lockdown), the SIGTERM exit 0 with a partial `final`, the hardcoded `/tmp` writable root, the session-log upload being on by default, that dsh reads only the cwd's `.env` and skips a directory there, the transport retry row, and the `credentials` row's `path` config.", + "Model and effort are config, not flags. Each lane gets a run-owned `--patch` file at `<receipt>.provider/patch.yml` (directory 0700, file 0600) that pins `agent-default-model` to provider `deepseek-official`, the lane's model and, unless the effort is `default`, its `reasoningEffort`. dsh offers efforts `off`, `low`, `high` and `max`, and fails `medium` locally.", + "Each lane runs with `DSH_HOME` set to its `<receipt>.provider/` directory, and the allowlist drops every inherited `DSH_*` name. dsh creates the `headless` profile there on first use with no network or `pnpm` install (about 70 ms), so no user layer (`profiles/headless/cordis.patch.yml`) and no home `.env` from the operator's dsh home loads. Measured with `--dump-config`: a planted user layer that redirected `llm-deepseek.config.baseURL`, moved `fs-sandbox.config.cwd`, picked another `agent-default-model` provider and disabled `bash-sandbox` changed nothing, and the composed tree was the default rows plus the lane's patch. The patch's `credentials` row points the credential store at `<the parent's DSH_HOME, else ~/.dsh>/.credentials.yaml`, read in place. A key kept only in the operator's `<DSH_HOME>/.env` no longer authenticates; store it or export `DEEPSEEK_API_KEY`.", + "Read-only only. The patch pins `sandbox-policy` to `read-only` with the lane's cwd as the workspace root and `approval` to `ask`, which overrides `DSH_PERMISSION_MODE`. The other sandbox rows keep dsh's defaults, including the bash tool's own timeout; the runner adds no deadline. Seatbelt enforces it on macOS for bash and the file tools. Headless has no approval channel, so an escalation request fails closed. Bash keeps network access, and the `read` tool is not confined to the cwd. The runner refuses an `isolated-write` lane before launch.", + "The patch turns off host and project rules (`AGENTS.md`, `CLAUDE.md` and their local variants), every skill root, the web, subagent and workflow tools, the session-title model call, the plugin inventory and the `dsh_session_log` upload. The goal, job, todo, plan-mode and `read_image` tools still load. dsh loads no project patch, hook or MCP file.", + "A dsh lane runs in any cwd, and a `.env` file there applies as it is. dsh loads the cwd's `.env` at boot, before any config row applies, and no setting turns it off. dsh rejects only bootstrap names there (`DSH_*`, `DEEPSEEK_BASE_URL`, `PATH`, `NODE_OPTIONS`, `XDG_*`, `DYLD_*`, proxies), and every other value reaches the bash tool: a repository `.env` that sets `GIT_CONFIG_PARAMETERS` with `core.fsmonitor` makes a plain `git status` run the repository's script. dsh reads only the cwd's `.env`, not a parent's, and warns and skips a directory named `.env`, such as a Python virtual environment.", + "dsh has no status command. The preflight is a turn on the lane's `DSH_HOME` with the lane's patch and then the shipped `providers/dsh-preflight.yml` patch, which points inference at a closed loopback port and turns off retries. The lane's patch already turns off the title call and the session-log upload. `TRANSPORT` passes with the fixed evidence `credential present; not validated`: it proves only that a key is stored or exported, not that the key works. `MISSING_CREDENTIAL` is `unauthenticated`/77, and any other output is `child-failed`/70.", + "The reply is the `--json` stream's `final` text, accepted only after a `status` `turn_end` with `reason.kind` `completed`. SIGTERM exits 0 with a partial `final` and no `turn_end`, so that is `malformed-output`/65, never complete. Exit 130 after SIGINT is `cancelled`/130. A `turn_end` error is `child-failed`/70 with its code and message, except that DeepSeek's `The supported API model names are ...` refusal is `unavailable-model`/69. Usage sums the `step_end` usage; the session ID is the `session` event's. No cost is reported.", + "No served model is reported, so the model proof is the pinned patch (`pinned-argv`).", + "Server side: requests still carry the `x-deepseek-harness-user-id` and `x-deepseek-harness-session-id` headers. Whether DeepSeek keeps request content is not measured.", + ], + }, +}); diff --git a/plugins/pstack/skills/poteto-mode/scripts/runner/providers/fixtures/agy/models-logged-in.txt b/plugins/pstack/skills/poteto-mode/scripts/runner/providers/fixtures/agy/models-logged-in.txt new file mode 100644 index 00000000..e912625f --- /dev/null +++ b/plugins/pstack/skills/poteto-mode/scripts/runner/providers/fixtures/agy/models-logged-in.txt @@ -0,0 +1,6 @@ +gemini-3.8-flash-high Gemini 3.8 Flash (High) +gemini-3.8-flash-medium Gemini 3.8 Flash (Medium) +gemini-3.8-flash-low Gemini 3.8 Flash (Low) +claude-sonnet-4-6 Claude Sonnet 4.6 (Thinking) +claude-opus-4-6-thinking Claude Opus 4.6 (Thinking) +gpt-oss-120b-medium GPT-OSS 120B (Medium) diff --git a/plugins/pstack/skills/poteto-mode/scripts/runner/providers/fixtures/agy/models-logged-out.stderr.txt b/plugins/pstack/skills/poteto-mode/scripts/runner/providers/fixtures/agy/models-logged-out.stderr.txt new file mode 100644 index 00000000..63e41d05 --- /dev/null +++ b/plugins/pstack/skills/poteto-mode/scripts/runner/providers/fixtures/agy/models-logged-out.stderr.txt @@ -0,0 +1,2 @@ +Fetching available models... +Error: Please sign in to view available models. Launch the CLI without arguments to sign in. diff --git a/plugins/pstack/skills/poteto-mode/scripts/runner/providers/fixtures/agy/stream-complete.jsonl b/plugins/pstack/skills/poteto-mode/scripts/runner/providers/fixtures/agy/stream-complete.jsonl new file mode 100644 index 00000000..13fe9f6b --- /dev/null +++ b/plugins/pstack/skills/poteto-mode/scripts/runner/providers/fixtures/agy/stream-complete.jsonl @@ -0,0 +1,9 @@ +{"event":"init","conversation_id":"00000000-0000-0000-0000-000000000000","init":{"model":"gemini-3.8-flash-medium","cwd":"/tmp/agy-fixture/51/cwd","tools":["ask_custom_permission","ask_permission","ask_question","browser_click_element","browser_drag_pixel_to_pixel","browser_get_dom","browser_get_network_request","browser_input","browser_list_network_requests","browser_mouse_down","browser_mouse_up","browser_move_mouse","browser_press_key","browser_refresh_page","browser_resize_window","browser_scroll","browser_scroll_dom","browser_select_option","browser_subagent","call_mcp_tool","capture_browser_console_logs","capture_browser_screenshot","click_browser_pixel","command_status","define_subagent","delete_knowledge","execute_browser_javascript","find_by_name","finish","generate_image","grep_search","invoke_subagent","list_browser_pages","list_dir","list_permissions","list_plugin_accounts","list_resources","manage_inbox","manage_subagents","manage_task","multi_replace_file_content","notebook_edit","notebook_execution","open_browser_url","read_browser_page","read_resource","read_url_content","replace_file_content","run_command","run_workflow","schedule","search_marketplace","search_web","sed_file","send_command_input","send_message","view_file","wait","wait_5_seconds","write_to_file"],"permission_mode":"request-review"}} +{"event":"step_update","step_update":{"conversation_id":"00000000-0000-0000-0000-000000000000","step_index":0,"state":"DONE","step_type":"user_input"}} +{"event":"step_update","step_update":{"conversation_id":"00000000-0000-0000-0000-000000000000","step_index":1,"state":"DONE","step_type":"agent_response","duration_seconds":1.86863,"usage":{"input_tokens":12350,"output_tokens":118,"thinking_tokens":59,"cache_read_tokens":0,"total_tokens":12468}}} +{"event":"step_update","step_update":{"conversation_id":"00000000-0000-0000-0000-000000000000","step_index":2,"state":"ACTIVE","step_type":"tool","tool_name":"view_file","tool_info":{"name":"view_file","parameters":{"AbsolutePath":"/tmp/agy-fixture/51/cwd/marker.txt"}}}} +{"event":"step_update","step_update":{"conversation_id":"00000000-0000-0000-0000-000000000000","step_index":2,"state":"DONE","step_type":"tool","tool_name":"view_file","duration_seconds":0.021817,"tool_info":{"name":"view_file","parameters":{"AbsolutePath":"/tmp/agy-fixture/51/cwd/marker.txt"},"output":"2 lines, 37 bytes"}}} +{"event":"step_update","step_update":{"conversation_id":"00000000-0000-0000-0000-000000000000","step_index":3,"state":"ACTIVE","step_type":"agent_response","text_delta":"FD37287"}} +{"event":"step_update","step_update":{"conversation_id":"00000000-0000-0000-0000-000000000000","step_index":3,"state":"ACTIVE","step_type":"agent_response","text_delta":"5-CAD0-443D-92FC-AB25C13DC8E3"}} +{"event":"step_update","step_update":{"conversation_id":"00000000-0000-0000-0000-000000000000","step_index":3,"state":"DONE","step_type":"agent_response","text_delta":"\n","duration_seconds":3.512425,"usage":{"input_tokens":12698,"output_tokens":192,"thinking_tokens":162,"cache_read_tokens":0,"total_tokens":12890}}} +{"event":"result","result":{"conversation_id":"00000000-0000-0000-0000-000000000000","status":"SUCCESS","response":"00000000-0000-0000-0000-000000000000\n","duration_seconds":5.457026,"num_turns":1,"usage":{"input_tokens":25048,"output_tokens":310,"thinking_tokens":221,"cache_read_tokens":0,"total_tokens":25358}}} diff --git a/plugins/pstack/skills/poteto-mode/scripts/runner/providers/fixtures/agy/stream-denied-shell.jsonl b/plugins/pstack/skills/poteto-mode/scripts/runner/providers/fixtures/agy/stream-denied-shell.jsonl new file mode 100644 index 00000000..380729f1 --- /dev/null +++ b/plugins/pstack/skills/poteto-mode/scripts/runner/providers/fixtures/agy/stream-denied-shell.jsonl @@ -0,0 +1,6 @@ +{"event":"init","conversation_id":"00000000-0000-0000-0000-000000000000","init":{"model":"gemini-3.8-flash-medium","cwd":"/tmp/agy-fixture/acc/accept-edits-sb-b/cwd","tools":["ask_custom_permission","ask_permission","ask_question","browser_click_element","browser_drag_pixel_to_pixel","browser_get_dom","browser_get_network_request","browser_input","browser_list_network_requests","browser_mouse_down","browser_mouse_up","browser_move_mouse","browser_press_key","browser_refresh_page","browser_resize_window","browser_scroll","browser_scroll_dom","browser_select_option","browser_subagent","call_mcp_tool","capture_browser_console_logs","capture_browser_screenshot","click_browser_pixel","command_status","define_subagent","delete_knowledge","execute_browser_javascript","find_by_name","finish","generate_image","grep_search","invoke_subagent","list_browser_pages","list_dir","list_permissions","list_plugin_accounts","list_resources","manage_inbox","manage_subagents","manage_task","multi_replace_file_content","notebook_edit","notebook_execution","open_browser_url","read_browser_page","read_resource","read_url_content","replace_file_content","run_command","run_workflow","schedule","search_marketplace","search_web","sed_file","send_command_input","send_message","view_file","wait","wait_5_seconds","write_to_file"],"permission_mode":"request-review"}} +{"event":"step_update","step_update":{"conversation_id":"00000000-0000-0000-0000-000000000000","step_index":0,"state":"DONE","step_type":"user_input"}} +{"event":"step_update","step_update":{"conversation_id":"00000000-0000-0000-0000-000000000000","step_index":1,"state":"DONE","step_type":"agent_response","duration_seconds":3.468328,"usage":{"input_tokens":13157,"output_tokens":215,"thinking_tokens":124,"cache_read_tokens":0,"total_tokens":13372}}} +{"event":"step_update","step_update":{"conversation_id":"00000000-0000-0000-0000-000000000000","step_index":2,"state":"ACTIVE","step_type":"tool","tool_name":"run_command","tool_info":{"name":"run_command","parameters":{"CommandLine":"echo hi \u003e shell.txt"}}}} +{"event":"step_update","step_update":{"conversation_id":"00000000-0000-0000-0000-000000000000","step_index":2,"state":"DONE","step_type":"tool","tool_name":"run_command","duration_seconds":0.031788,"tool_info":{"name":"run_command","parameters":{"CommandLine":"echo hi \u003e shell.txt"}}}} +{"event":"result","result":{"conversation_id":"00000000-0000-0000-0000-000000000000","status":"SUCCESS","response":"","duration_seconds":3.495687,"num_turns":1,"usage":{"input_tokens":13157,"output_tokens":215,"thinking_tokens":124,"cache_read_tokens":0,"total_tokens":13372},"denied_actions":[{"action":"command","display_name":"RunCommand"}]}} diff --git a/plugins/pstack/skills/poteto-mode/scripts/runner/providers/fixtures/agy/stream-denied-shell.stderr.txt b/plugins/pstack/skills/poteto-mode/scripts/runner/providers/fixtures/agy/stream-denied-shell.stderr.txt new file mode 100644 index 00000000..40f9c58a --- /dev/null +++ b/plugins/pstack/skills/poteto-mode/scripts/runner/providers/fixtures/agy/stream-denied-shell.stderr.txt @@ -0,0 +1 @@ +jetski: no output produced — a tool required the "command" permission that headless mode cannot prompt for, so it was auto-denied. Add an allow-rule under permissions.allow in settings.json (e.g. command(<target>)). Alternatively, re-run with --dangerously-skip-permissions to auto-approve all tools. diff --git a/plugins/pstack/skills/poteto-mode/scripts/runner/providers/fixtures/agy/stream-sigint.jsonl b/plugins/pstack/skills/poteto-mode/scripts/runner/providers/fixtures/agy/stream-sigint.jsonl new file mode 100644 index 00000000..df3c00b6 --- /dev/null +++ b/plugins/pstack/skills/poteto-mode/scripts/runner/providers/fixtures/agy/stream-sigint.jsonl @@ -0,0 +1,3 @@ +{"event":"init","conversation_id":"00000000-0000-0000-0000-000000000000","init":{"model":"gemini-3.8-flash-low","cwd":"/tmp/agy-fixture/70-cancel-sigint","tools":["ask_custom_permission","ask_permission","ask_question","browser_click_element","browser_drag_pixel_to_pixel","browser_get_dom","browser_get_network_request","browser_input","browser_list_network_requests","browser_mouse_down","browser_mouse_up","browser_move_mouse","browser_press_key","browser_refresh_page","browser_resize_window","browser_scroll","browser_scroll_dom","browser_select_option","browser_subagent","call_mcp_tool","capture_browser_console_logs","capture_browser_screenshot","click_browser_pixel","command_status","define_subagent","delete_knowledge","execute_browser_javascript","find_by_name","finish","generate_image","grep_search","invoke_subagent","list_browser_pages","list_dir","list_permissions","list_plugin_accounts","list_resources","manage_inbox","manage_subagents","manage_task","multi_replace_file_content","notebook_edit","notebook_execution","open_browser_url","read_browser_page","read_resource","read_url_content","replace_file_content","run_command","run_workflow","schedule","search_marketplace","search_web","sed_file","send_command_input","send_message","view_file","wait","wait_5_seconds","write_to_file"],"permission_mode":"request-review"}} +{"event":"step_update","step_update":{"conversation_id":"00000000-0000-0000-0000-000000000000","step_index":0,"state":"DONE","step_type":"user_input"}} +{"event":"result","result":{"conversation_id":"00000000-0000-0000-0000-000000000000","status":"ERROR","response":"","error":"interrupted","duration_seconds":0,"num_turns":1,"usage":{"input_tokens":0,"output_tokens":0,"thinking_tokens":0,"cache_read_tokens":0,"total_tokens":0}}} diff --git a/plugins/pstack/skills/poteto-mode/scripts/runner/providers/fixtures/agy/stream-unknown-model.jsonl b/plugins/pstack/skills/poteto-mode/scripts/runner/providers/fixtures/agy/stream-unknown-model.jsonl new file mode 100644 index 00000000..023a4cd2 --- /dev/null +++ b/plugins/pstack/skills/poteto-mode/scripts/runner/providers/fixtures/agy/stream-unknown-model.jsonl @@ -0,0 +1 @@ +{"event":"result","result":{"conversation_id":"","status":"ERROR","response":"","error":"invalid model selection (--model \"gemini-3.7-flash\" --effort \"\"): model gemini-3.7-flash is not recognized as a known model or custom model in settings\nAvailable models:\n Gemini 3.8 Flash (High)\n Gemini 3.8 Flash (Medium)\n Gemini 3.8 Flash (Low)\n Claude Sonnet 4.6 (Thinking)\n Claude Opus 4.6 (Thinking)\n GPT-OSS 120B (Medium)","duration_seconds":0,"num_turns":0,"usage":{"input_tokens":0,"output_tokens":0,"thinking_tokens":0,"cache_read_tokens":0,"total_tokens":0}}} diff --git a/plugins/pstack/skills/poteto-mode/scripts/runner/providers/fixtures/agy/stream-unknown-model.stderr.txt b/plugins/pstack/skills/poteto-mode/scripts/runner/providers/fixtures/agy/stream-unknown-model.stderr.txt new file mode 100644 index 00000000..650b7eb9 --- /dev/null +++ b/plugins/pstack/skills/poteto-mode/scripts/runner/providers/fixtures/agy/stream-unknown-model.stderr.txt @@ -0,0 +1,8 @@ +error: invalid model selection (--model "gemini-3.7-flash" --effort ""): model gemini-3.7-flash is not recognized as a known model or custom model in settings +Available models: + Gemini 3.8 Flash (High) + Gemini 3.8 Flash (Medium) + Gemini 3.8 Flash (Low) + Claude Sonnet 4.6 (Thinking) + Claude Opus 4.6 (Thinking) + GPT-OSS 120B (Medium) diff --git a/plugins/pstack/skills/poteto-mode/scripts/runner/providers/fixtures/amp/lockdown-settings.json b/plugins/pstack/skills/poteto-mode/scripts/runner/providers/fixtures/amp/lockdown-settings.json new file mode 100644 index 00000000..8bfcb568 --- /dev/null +++ b/plugins/pstack/skills/poteto-mode/scripts/runner/providers/fixtures/amp/lockdown-settings.json @@ -0,0 +1,40 @@ +{ + "amp.mcpServers": {}, + "amp.skills.disableClaudeCodeSkills": true, + "amp.skills.disableGlobalAgentsSkills": true, + "amp.tools.enable": [ + "Read" + ], + "amp.permissions": [ + { + "tool": "Read", + "action": "allow" + }, + { + "tool": "create_file", + "action": "reject" + }, + { + "tool": "edit_file", + "action": "reject" + }, + { + "tool": "apply_patch", + "action": "reject" + }, + { + "tool": "shell_command", + "action": "reject" + }, + { + "tool": "code_exec", + "action": "reject" + }, + { + "tool": "*", + "action": "reject" + } + ], + "amp.updates.mode": "disabled", + "amp.notifications.enabled": false +} diff --git a/plugins/pstack/skills/poteto-mode/scripts/runner/providers/fixtures/amp/stream-cancel-sigint.jsonl b/plugins/pstack/skills/poteto-mode/scripts/runner/providers/fixtures/amp/stream-cancel-sigint.jsonl new file mode 100644 index 00000000..0a5301f5 --- /dev/null +++ b/plugins/pstack/skills/poteto-mode/scripts/runner/providers/fixtures/amp/stream-cancel-sigint.jsonl @@ -0,0 +1,3 @@ +{"type":"system","subtype":"init","cwd":"/tmp/fixture/c42-cancel-INT","session_id":"T-<THREAD_ID>","tools":["ask_user_choice","code_exec","create_file","create_thread","download_thread_changes","download_thread_file","edit_file","finder","librarian","load_plugin","load_prompt","manage_amp","oracle","painter","read_thread","read_web_page","reload_mcp","reload_plugins","reload_skills","shell_command","shell_command_kill","shell_command_status","skill","Task","tool_search","upload_thread_file","view_media","wait_for_threads","web_search"],"mcp_servers":[],"agent_mode":"low"} +{"type":"user","message":{"role":"user","content":[{"type":"text","text":"Write a 2000-word essay on the history of rivers."}]},"parent_tool_use_id":null,"session_id":"T-<THREAD_ID>"} +{"type":"result","subtype":"error_during_execution","duration_ms":2749,"is_error":true,"num_turns":0,"error":"User cancelled (SIGINT/SIGTERM)","session_id":"T-<THREAD_ID>"} diff --git a/plugins/pstack/skills/poteto-mode/scripts/runner/providers/fixtures/amp/stream-complete-readonly-marker.jsonl b/plugins/pstack/skills/poteto-mode/scripts/runner/providers/fixtures/amp/stream-complete-readonly-marker.jsonl new file mode 100644 index 00000000..228c684c --- /dev/null +++ b/plugins/pstack/skills/poteto-mode/scripts/runner/providers/fixtures/amp/stream-complete-readonly-marker.jsonl @@ -0,0 +1,6 @@ +{"type":"system","subtype":"init","cwd":"/tmp/fixture/c35-readonly-lockdown-marker/cwd","session_id":"T-<THREAD_ID>","tools":["Read"],"mcp_servers":[],"agent_mode":"medium"} +{"type":"user","message":{"role":"user","content":[{"type":"text","text":"Read marker.txt in the current directory and reply with exactly its contents and nothing else."}]},"parent_tool_use_id":null,"session_id":"T-<THREAD_ID>"} +{"type":"assistant","message":{"type":"message","role":"assistant","content":[{"type":"tool_use","id":"TU-034gh7ywYZYSSPqZs7J9Ug","name":"Read","input":{"path":"/tmp/fixture/c35-readonly-lockdown-marker/cwd/marker.txt"}},{"type":"tool_use","id":"TU-034gh7zFT5HFTELxleI6To","name":"Read","input":{"path":"/tmp/fixture/c35-readonly-lockdown-marker/cwd/AGENTS.md"}}],"stop_reason":"tool_use","usage":{"input_tokens":0,"cache_creation_input_tokens":5150,"cache_read_input_tokens":0,"output_tokens":237,"service_tier":"standard"}},"parent_tool_use_id":null,"session_id":"T-<THREAD_ID>"} +{"type":"user","message":{"role":"user","content":[{"type":"tool_result","tool_use_id":"TU-034gh7ywYZYSSPqZs7J9Ug","content":"{\"absolutePath\":\"/tmp/fixture/c35-readonly-lockdown-marker/cwd/marker.txt\",\"content\":\"1: 39d942e7-da69-4370-863f-d2468a9fd511\"}","is_error":false},{"type":"tool_result","tool_use_id":"TU-034gh7zFT5HFTELxleI6To","content":"{\"absolutePath\":\"/tmp/fixture/c35-readonly-lockdown-marker/cwd/AGENTS.md\",\"content\":\"1: CANARY-PROJECT-AGENTS-7f3a: project rule line.\"}","is_error":false}]},"parent_tool_use_id":null,"session_id":"T-<THREAD_ID>"} +{"type":"assistant","message":{"type":"message","role":"assistant","content":[{"type":"text","text":"39d942e7-da69-4370-863f-d2468a9fd511"}],"stop_reason":"end_turn","usage":{"input_tokens":0,"cache_creation_input_tokens":3599,"cache_read_input_tokens":1920,"output_tokens":22,"service_tier":"standard"}},"parent_tool_use_id":null,"session_id":"T-<THREAD_ID>"} +{"type":"result","subtype":"success","duration_ms":9453,"is_error":false,"num_turns":2,"result":"39d942e7-da69-4370-863f-d2468a9fd511","session_id":"T-<THREAD_ID>"} diff --git a/plugins/pstack/skills/poteto-mode/scripts/runner/providers/fixtures/amp/stream-error-subscription-only-exit0.jsonl b/plugins/pstack/skills/poteto-mode/scripts/runner/providers/fixtures/amp/stream-error-subscription-only-exit0.jsonl new file mode 100644 index 00000000..eb45dad6 --- /dev/null +++ b/plugins/pstack/skills/poteto-mode/scripts/runner/providers/fixtures/amp/stream-error-subscription-only-exit0.jsonl @@ -0,0 +1,3 @@ +{"type":"system","subtype":"init","cwd":"/tmp/fixture/c23-marker-claude-opus-5-5","session_id":"T-<THREAD_ID>","tools":["create_file","download_thread_changes","download_thread_file","edit_file","load_plugin","read_web_page","reload_mcp","reload_plugins","reload_skills","shell_command","shell_command_kill","shell_command_status","skill","upload_thread_file","web_search"],"mcp_servers":[],"agent_mode":"medium"} +{"type":"user","message":{"role":"user","content":[{"type":"text","text":"Read marker.txt in the current directory and reply with exactly its contents and nothing else."}]},"parent_tool_use_id":null,"session_id":"T-<THREAD_ID>"} +{"type":"result","subtype":"error_during_execution","duration_ms":317,"is_error":true,"num_turns":0,"error":"Claude Usage is set to Subscription Only. Claude Opus 5.5 is paid per token outside Claude Code. Use Claude Code mode or a non-Claude model, or choose Allow Paid-Per-Token Usage in Settings → Model Routing. If the user asks to allow paid-per-token usage, call manage_amp({topic: \"settings\", operation: \"update\", parameters: {key: \"claude_usage\", value: \"allow-paid-per-token\"}}). The user must approve its confirmation dialog before the setting changes.","session_id":"T-<THREAD_ID>"} diff --git a/plugins/pstack/skills/poteto-mode/scripts/runner/providers/fixtures/amp/stream-tool-rejected-by-permission.jsonl b/plugins/pstack/skills/poteto-mode/scripts/runner/providers/fixtures/amp/stream-tool-rejected-by-permission.jsonl new file mode 100644 index 00000000..c6beab27 --- /dev/null +++ b/plugins/pstack/skills/poteto-mode/scripts/runner/providers/fixtures/amp/stream-tool-rejected-by-permission.jsonl @@ -0,0 +1,10 @@ +{"type":"system","subtype":"init","cwd":"/tmp/fixture/c31-perm-reject-low/cwd","session_id":"T-<THREAD_ID>","tools":["ask_user_choice","code_exec","create_file","create_thread","download_thread_changes","download_thread_file","edit_file","finder","librarian","load_plugin","load_prompt","manage_amp","oracle","painter","read_thread","read_web_page","reload_mcp","reload_plugins","reload_skills","shell_command","shell_command_kill","shell_command_status","skill","Task","tool_search","upload_thread_file","view_media","wait_for_threads","web_search"],"mcp_servers":[],"agent_mode":"low"} +{"type":"user","message":{"role":"user","content":[{"type":"text","text":"Do these three steps in order. Attempt every step even if an earlier one fails. Do not use any other approach than the one named.\n1. Use your file-creation tool (not the shell) to create the file created.txt in the current directory with the content \"a\".\n2. Use the shell tool to run exactly: printf hi > shell.txt\n3. Use the shell tool to run exactly: printf hi > ../sibling/outside.txt\nThen reply with one line per step: the step number, the tool you used, and the exact result or error the tool returned."}]},"parent_tool_use_id":null,"session_id":"T-<THREAD_ID>"} +{"type":"assistant","message":{"type":"message","role":"assistant","content":[{"type":"tool_use","id":"TU-034gh324aIDK3uV0z4lyo2","name":"create_file","input":{"content":"a","path":"/tmp/fixture/c31-perm-reject-low/cwd/created.txt"}}],"stop_reason":"tool_use","usage":{"input_tokens":0,"cache_creation_input_tokens":47562,"cache_read_input_tokens":0,"output_tokens":42,"service_tier":"standard"}},"parent_tool_use_id":null,"session_id":"T-<THREAD_ID>"} +{"type":"user","message":{"role":"user","content":[{"type":"tool_result","tool_use_id":"TU-034gh324aIDK3uV0z4lyo2","content":"Tool rejected by plugin: Rejected by user permissions rule 0: reject create_file","is_error":false}]},"parent_tool_use_id":null,"session_id":"T-<THREAD_ID>"} +{"type":"assistant","message":{"type":"message","role":"assistant","content":[{"type":"tool_use","id":"TU-034gh35akFhKPGfO6phN1E","name":"shell_command","input":{"command":"printf hi > shell.txt","workdir":"/tmp/fixture/c31-perm-reject-low/cwd"}}],"stop_reason":"tool_use","usage":{"input_tokens":0,"cache_creation_input_tokens":521,"cache_read_input_tokens":47104,"output_tokens":44,"service_tier":"standard"}},"parent_tool_use_id":null,"session_id":"T-<THREAD_ID>"} +{"type":"user","message":{"role":"user","content":[{"type":"tool_result","tool_use_id":"TU-034gh35akFhKPGfO6phN1E","content":"Tool rejected by plugin: Rejected by user permissions rule 3: reject shell_command","is_error":false}]},"parent_tool_use_id":null,"session_id":"T-<THREAD_ID>"} +{"type":"assistant","message":{"type":"message","role":"assistant","content":[{"type":"tool_use","id":"TU-034gh393ghoFbjDkQRum8U","name":"shell_command","input":{"command":"printf hi > ../sibling/outside.txt","workdir":"/tmp/fixture/c31-perm-reject-low/cwd"}}],"stop_reason":"tool_use","usage":{"input_tokens":0,"cache_creation_input_tokens":586,"cache_read_input_tokens":47104,"output_tokens":48,"service_tier":"standard"}},"parent_tool_use_id":null,"session_id":"T-<THREAD_ID>"} +{"type":"user","message":{"role":"user","content":[{"type":"tool_result","tool_use_id":"TU-034gh393ghoFbjDkQRum8U","content":"Tool rejected by plugin: Rejected by user permissions rule 3: reject shell_command","is_error":false}]},"parent_tool_use_id":null,"session_id":"T-<THREAD_ID>"} +{"type":"assistant","message":{"type":"message","role":"assistant","content":[{"type":"text","text":"1. create_file — created.txt — Tool rejected by plugin: Rejected by user permissions rule 0: reject create_file\n2. shell_command — `printf hi > shell.txt` — Tool rejected by plugin: Rejected by user permissions rule 3: reject shell_command\n3. shell_command — `printf hi > ../sibling/outside.txt` — Tool rejected by plugin: Rejected by user permissions rule 3: reject shell_command\n\nNo files were created; every tool call was blocked by your permission rules before execution."}],"stop_reason":"end_turn","usage":{"input_tokens":0,"cache_creation_input_tokens":655,"cache_read_input_tokens":47104,"output_tokens":117,"service_tier":"standard"}},"parent_tool_use_id":null,"session_id":"T-<THREAD_ID>"} +{"type":"result","subtype":"success","duration_ms":11979,"is_error":false,"num_turns":4,"result":"1. create_file — created.txt — Tool rejected by plugin: Rejected by user permissions rule 0: reject create_file\n2. shell_command — `printf hi > shell.txt` — Tool rejected by plugin: Rejected by user permissions rule 3: reject shell_command\n3. shell_command — `printf hi > ../sibling/outside.txt` — Tool rejected by plugin: Rejected by user permissions rule 3: reject shell_command\n\nNo files were created; every tool call was blocked by your permission rules before execution.","session_id":"T-<THREAD_ID>"} diff --git a/plugins/pstack/skills/poteto-mode/scripts/runner/providers/fixtures/amp/unknown-mode.stderr.txt b/plugins/pstack/skills/poteto-mode/scripts/runner/providers/fixtures/amp/unknown-mode.stderr.txt new file mode 100644 index 00000000..640c86d2 --- /dev/null +++ b/plugins/pstack/skills/poteto-mode/scripts/runner/providers/fixtures/amp/unknown-mode.stderr.txt @@ -0,0 +1,2 @@ +Error: Unexpected error inside Amp CLI. Run with AMP_DEBUG=1 to see detailed logs. +Use 'feedback: report bug' from the command palette to send this to Amp. diff --git a/plugins/pstack/skills/poteto-mode/scripts/runner/providers/fixtures/amp/usage-logged-in.stdout.txt b/plugins/pstack/skills/poteto-mode/scripts/runner/providers/fixtures/amp/usage-logged-in.stdout.txt new file mode 100644 index 00000000..8be6fe45 --- /dev/null +++ b/plugins/pstack/skills/poteto-mode/scripts/runner/providers/fixtures/amp/usage-logged-in.stdout.txt @@ -0,0 +1,4 @@ +Signed in as <EMAIL> +**Individual credits:** $<CREDITS> remaining (set up auto-reload to avoid running out) - https://ampcode.com/settings + +# Run `amp usage --details` for more detailed information. diff --git a/plugins/pstack/skills/poteto-mode/scripts/runner/providers/fixtures/amp/usage-logged-out.stderr.txt b/plugins/pstack/skills/poteto-mode/scripts/runner/providers/fixtures/amp/usage-logged-out.stderr.txt new file mode 100644 index 00000000..4c2c31f8 --- /dev/null +++ b/plugins/pstack/skills/poteto-mode/scripts/runner/providers/fixtures/amp/usage-logged-out.stderr.txt @@ -0,0 +1 @@ +Error: Invalid or missing API key. Run 'amp login' to authenticate. diff --git a/plugins/pstack/skills/poteto-mode/scripts/runner/providers/fixtures/cursor/cancel-sigint.jsonl b/plugins/pstack/skills/poteto-mode/scripts/runner/providers/fixtures/cursor/cancel-sigint.jsonl new file mode 100644 index 00000000..4ee5fad6 --- /dev/null +++ b/plugins/pstack/skills/poteto-mode/scripts/runner/providers/fixtures/cursor/cancel-sigint.jsonl @@ -0,0 +1,24 @@ +{"type":"system","subtype":"init","apiKeySource":"env","cwd":"<CAPTURE_ROOT>/16c-cancel-sigint-12s/cwd","session_id":"00000000-0000-0000-0000-000000000001","model":"Composer 2.5","permissionMode":"default"} +{"type":"user","message":{"role":"user","content":[{"type":"text","text":"Write a 2000-word essay on the history of rivers."}]},"session_id":"00000000-0000-0000-0000-000000000001"} +{"type":"thinking","subtype":"delta","text":"I am preparing to write","session_id":"00000000-0000-0000-0000-000000000001","timestamp_ms":1792121531424} +{"type":"thinking","subtype":"delta","text":" a 2000-word essay on","session_id":"00000000-0000-0000-0000-000000000001","timestamp_ms":1792121531502} +{"type":"thinking","subtype":"delta","text":" the history of rivers.","session_id":"00000000-0000-0000-0000-000000000001","timestamp_ms":1792121531599} +{"type":"thinking","subtype":"delta","text":"\n\nI will combine geological","session_id":"00000000-0000-0000-0000-000000000001","timestamp_ms":1792121532226} +{"type":"thinking","subtype":"delta","text":" evolution with human","session_id":"00000000-0000-0000-0000-000000000001","timestamp_ms":1792121532279} +{"type":"thinking","subtype":"delta","text":" civilization. Rivers","session_id":"00000000-0000-0000-0000-000000000001","timestamp_ms":1792121532316} +{"type":"thinking","subtype":"delta","text":" shaped landscapes and","session_id":"00000000-0000-0000-0000-000000000001","timestamp_ms":1792121532418} +{"type":"thinking","subtype":"delta","text":" enabled trade, culture,","session_id":"00000000-0000-0000-0000-000000000001","timestamp_ms":1792121532419} +{"type":"thinking","subtype":"delta","text":" and settlement.","session_id":"00000000-0000-0000-0000-000000000001","timestamp_ms":1792121532437} +{"type":"thinking","subtype":"delta","text":"\n\nThe essay will follow","session_id":"00000000-0000-0000-0000-000000000001","timestamp_ms":1792121533468} +{"type":"thinking","subtype":"delta","text":" a chronological arc","session_id":"00000000-0000-0000-0000-000000000001","timestamp_ms":1792121533468} +{"type":"thinking","subtype":"delta","text":" from geological origins","session_id":"00000000-0000-0000-0000-000000000001","timestamp_ms":1792121533469} +{"type":"thinking","subtype":"delta","text":" through ancient civilizations,","session_id":"00000000-0000-0000-0000-000000000001","timestamp_ms":1792121533469} +{"type":"thinking","subtype":"delta","text":" classical trade, industrial","session_id":"00000000-0000-0000-0000-000000000001","timestamp_ms":1792121533469} +{"type":"thinking","subtype":"delta","text":" modification, and modern","session_id":"00000000-0000-0000-0000-000000000001","timestamp_ms":1792121533469} +{"type":"thinking","subtype":"delta","text":" conservation. Key themes","session_id":"00000000-0000-0000-0000-000000000001","timestamp_ms":1792121533469} +{"type":"thinking","subtype":"delta","text":" include landscape shaping,","session_id":"00000000-0000-0000-0000-000000000001","timestamp_ms":1792121533469} +{"type":"thinking","subtype":"delta","text":" early human settlement,","session_id":"00000000-0000-0000-0000-000000000001","timestamp_ms":1792121533469} +{"type":"thinking","subtype":"delta","text":" major river civilizations,","session_id":"00000000-0000-0000-0000-000000000001","timestamp_ms":1792121533469} +{"type":"thinking","subtype":"delta","text":" damming impacts, and","session_id":"00000000-0000-0000-0000-000000000001","timestamp_ms":1792121533470} +{"type":"thinking","subtype":"delta","text":" legal personhood.","session_id":"00000000-0000-0000-0000-000000000001","timestamp_ms":1792121533470} +{"type":"thinking","subtype":"completed","session_id":"00000000-0000-0000-0000-000000000001","timestamp_ms":1792121533470} diff --git a/plugins/pstack/skills/poteto-mode/scripts/runner/providers/fixtures/cursor/cancel-sigint.stderr.txt b/plugins/pstack/skills/poteto-mode/scripts/runner/providers/fixtures/cursor/cancel-sigint.stderr.txt new file mode 100644 index 00000000..1964ca96 --- /dev/null +++ b/plugins/pstack/skills/poteto-mode/scripts/runner/providers/fixtures/cursor/cancel-sigint.stderr.txt @@ -0,0 +1,2 @@ + +Aborting operation... diff --git a/plugins/pstack/skills/poteto-mode/scripts/runner/providers/fixtures/cursor/cancel-sigterm.jsonl b/plugins/pstack/skills/poteto-mode/scripts/runner/providers/fixtures/cursor/cancel-sigterm.jsonl new file mode 100644 index 00000000..1794a09c --- /dev/null +++ b/plugins/pstack/skills/poteto-mode/scripts/runner/providers/fixtures/cursor/cancel-sigterm.jsonl @@ -0,0 +1,16 @@ +{"type":"system","subtype":"init","apiKeySource":"env","cwd":"<CAPTURE_ROOT>/16d-cancel-sigterm-12s/cwd","session_id":"00000000-0000-0000-0000-000000000002","model":"Composer 2.5","permissionMode":"default"} +{"type":"user","message":{"role":"user","content":[{"type":"text","text":"Write a 2000-word essay on the history of rivers."}]},"session_id":"00000000-0000-0000-0000-000000000002"} +{"type":"thinking","subtype":"delta","text":"I am preparing to write","session_id":"00000000-0000-0000-0000-000000000002","timestamp_ms":1792121549205} +{"type":"thinking","subtype":"delta","text":" a 2000-word essay on","session_id":"00000000-0000-0000-0000-000000000002","timestamp_ms":1792121549280} +{"type":"thinking","subtype":"delta","text":" the history of rivers.","session_id":"00000000-0000-0000-0000-000000000002","timestamp_ms":1792121549356} +{"type":"thinking","subtype":"delta","text":"\n\nThe essay will cover","session_id":"00000000-0000-0000-0000-000000000002","timestamp_ms":1792121550420} +{"type":"thinking","subtype":"delta","text":" geological origins,","session_id":"00000000-0000-0000-0000-000000000002","timestamp_ms":1792121550421} +{"type":"thinking","subtype":"delta","text":" human civilization,","session_id":"00000000-0000-0000-0000-000000000002","timestamp_ms":1792121550421} +{"type":"thinking","subtype":"delta","text":" cultural significance,","session_id":"00000000-0000-0000-0000-000000000002","timestamp_ms":1792121550421} +{"type":"thinking","subtype":"delta","text":" scientific understanding,","session_id":"00000000-0000-0000-0000-000000000002","timestamp_ms":1792121550422} +{"type":"thinking","subtype":"delta","text":" and environmental changes.","session_id":"00000000-0000-0000-0000-000000000002","timestamp_ms":1792121550422} +{"type":"thinking","subtype":"delta","text":" It will be structured","session_id":"00000000-0000-0000-0000-000000000002","timestamp_ms":1792121550422} +{"type":"thinking","subtype":"delta","text":" with an introduction,","session_id":"00000000-0000-0000-0000-000000000002","timestamp_ms":1792121550422} +{"type":"thinking","subtype":"delta","text":" several thematic sections,","session_id":"00000000-0000-0000-0000-000000000002","timestamp_ms":1792121550423} +{"type":"thinking","subtype":"delta","text":" and a conclusion.","session_id":"00000000-0000-0000-0000-000000000002","timestamp_ms":1792121550423} +{"type":"thinking","subtype":"completed","session_id":"00000000-0000-0000-0000-000000000002","timestamp_ms":1792121550423} diff --git a/plugins/pstack/skills/poteto-mode/scripts/runner/providers/fixtures/cursor/models-logged-in.txt b/plugins/pstack/skills/poteto-mode/scripts/runner/providers/fixtures/cursor/models-logged-in.txt new file mode 100644 index 00000000..27b75187 --- /dev/null +++ b/plugins/pstack/skills/poteto-mode/scripts/runner/providers/fixtures/cursor/models-logged-in.txt @@ -0,0 +1,269 @@ +Available models + +auto - Auto (current, default) +gpt-5.3-codex-low - Codex 5.3 Low +gpt-5.3-codex-low-fast - Codex 5.3 Low Fast +gpt-5.3-codex - Codex 5.3 +gpt-5.3-codex-fast - Codex 5.3 Fast +gpt-5.3-codex-high - Codex 5.3 High +gpt-5.3-codex-high-fast - Codex 5.3 High Fast +gpt-5.3-codex-xhigh - Codex 5.3 Extra High +gpt-5.3-codex-xhigh-fast - Codex 5.3 Extra High Fast +gpt-5.2 - GPT-5.2 +composer-2.5 - Composer 2.5 +claude-opus-5-thinking-high - Claude Opus 5 1M Thinking +claude-opus-5-thinking-high-fast - Claude Opus 5 1M Thinking Fast +gpt-5.6-sol-high - GPT-5.6 Sol 1M High +gpt-5.6-sol-high-fast - GPT-5.6 Sol 1M High Fast +gpt-5.6-sol-xhigh - GPT-5.6 Sol 1M Extra High +gpt-5.6-sol-xhigh-fast - GPT-5.6 Sol 1M Extra High Fast +claude-fable-5-thinking-high - Claude Fable 5 1M Thinking (NO ZDR) +claude-fable-5-thinking-xhigh - Claude Fable 5 1M Extra High Thinking (NO ZDR) +cursor-grok-4.5-high - Grok 4.5 +cursor-grok-4.5-high-fast - Grok 4.5 Fast +gemini-3.7-flash-high - Gemini 3.7 Flash +claude-sonnet-5-thinking-high - Claude Sonnet 5 1M Thinking +claude-sonnet-5-thinking-xhigh - Claude Sonnet 5 1M Extra High Thinking +gpt-5.6-luna-high - GPT-5.6 Luna 1M High +grok-4.7-low - Grok 4.7 Low +grok-4.7-low-fast - Grok 4.7 Low Fast​​ +grok-4.7-medium - Grok 4.7 Medium +grok-4.7-medium-fast - Grok 4.7 Medium Fast​​ +grok-4.7-high - Grok 4.7 High +grok-4.7-high-fast - Grok 4.7 High Fast​​ +grok-4.7-xhigh - Grok 4.7 Extra High +grok-4.7-xhigh-fast - Grok 4.7 Extra High Fast​​ +cursor-grok-4.6-low - Grok 4.6 Low +cursor-grok-4.6-low-fast - Grok 4.6 Low Fast +cursor-grok-4.6-medium - Grok 4.6 Medium +cursor-grok-4.6-medium-fast - Grok 4.6 Medium Fast +cursor-grok-4.6-high - Grok 4.6 +cursor-grok-4.6-high-fast - Grok 4.6 Fast +cursor-grok-4.6-xhigh - Grok 4.6 Extra High +cursor-grok-4.6-xhigh-fast - Grok 4.6 Extra High Fast +composer-2.5-fast - Composer 2.5 Fast +claude-opus-5-5-low - Claude Opus 5.5 1M Low +claude-opus-5-5-low-fast - Claude Opus 5.5 1M Low Fast +claude-opus-5-5-medium - Claude Opus 5.5 1M +claude-opus-5-5-medium-fast - Claude Opus 5.5 1M Fast +claude-opus-5-5-high - Claude Opus 5.5 1M High +claude-opus-5-5-high-fast - Claude Opus 5.5 1M High Fast +claude-opus-5-5-xhigh - Claude Opus 5.5 1M Extra High +claude-opus-5-5-xhigh-fast - Claude Opus 5.5 1M Extra High Fast +claude-opus-5-5-max - Claude Opus 5.5 1M Max +claude-opus-5-5-max-fast - Claude Opus 5.5 1M Max Fast +claude-opus-5-low - Claude Opus 5 1M Low +claude-opus-5-low-fast - Claude Opus 5 1M Low Fast +claude-opus-5-medium - Claude Opus 5 1M Medium +claude-opus-5-medium-fast - Claude Opus 5 1M Medium Fast +claude-opus-5-high - Claude Opus 5 1M +claude-opus-5-high-fast - Claude Opus 5 1M Fast +claude-opus-5-thinking-low - Claude Opus 5 1M Low Thinking +claude-opus-5-thinking-low-fast - Claude Opus 5 1M Low Thinking Fast +claude-opus-5-thinking-medium - Claude Opus 5 1M Medium Thinking +claude-opus-5-thinking-medium-fast - Claude Opus 5 1M Medium Thinking Fast +claude-opus-5-thinking-xhigh - Claude Opus 5 1M Extra High Thinking +claude-opus-5-thinking-xhigh-fast - Claude Opus 5 1M Extra High Thinking Fast +claude-opus-5-thinking-max - Claude Opus 5 1M Max Thinking +claude-opus-5-thinking-max-fast - Claude Opus 5 1M Max Thinking Fast +claude-opus-4-8-low - Claude Opus 4.8 1M Low +claude-opus-4-8-low-fast - Claude Opus 4.8 1M Low Fast +claude-opus-4-8-medium - Claude Opus 4.8 1M Medium +claude-opus-4-8-medium-fast - Claude Opus 4.8 1M Medium Fast +claude-opus-4-8-high - Claude Opus 4.8 1M +claude-opus-4-8-high-fast - Claude Opus 4.8 1M Fast +claude-opus-4-8-xhigh - Claude Opus 4.8 1M Extra High +claude-opus-4-8-xhigh-fast - Claude Opus 4.8 1M Extra High Fast +claude-opus-4-8-max - Claude Opus 4.8 1M Max +claude-opus-4-8-max-fast - Claude Opus 4.8 1M Max Fast +claude-opus-4-8-thinking-low - Claude Opus 4.8 1M Low Thinking +claude-opus-4-8-thinking-low-fast - Claude Opus 4.8 1M Low Thinking Fast +claude-opus-4-8-thinking-medium - Claude Opus 4.8 1M Medium Thinking +claude-opus-4-8-thinking-medium-fast - Claude Opus 4.8 1M Medium Thinking Fast +claude-opus-4-8-thinking-high - Claude Opus 4.8 1M Thinking +claude-opus-4-8-thinking-high-fast - Claude Opus 4.8 1M Thinking Fast +claude-opus-4-8-thinking-xhigh - Claude Opus 4.8 1M Extra High Thinking +claude-opus-4-8-thinking-xhigh-fast - Claude Opus 4.8 1M Extra High Thinking Fast +claude-opus-4-8-thinking-max - Claude Opus 4.8 1M Max Thinking +claude-opus-4-8-thinking-max-fast - Claude Opus 4.8 1M Max Thinking Fast +gemini-4-argon-low - Gemini 4 Argon Low +gemini-4-argon-medium - Gemini 4 Argon Medium +gemini-4-argon-high - Gemini 4 Argon High +muse-wonder-1.0-minimal - Muse Wonder 300K Minimal +muse-wonder-1.0-low - Muse Wonder 300K Low +muse-wonder-1.0-medium - Muse Wonder 300K Medium +muse-wonder-1.0-high - Muse Wonder 300K High +muse-wonder-1.0-xhigh - Muse Wonder 300K Extra High +muse-wonder-1.0-max - Muse Wonder 300K Max +gpt-5.6-sol-none - GPT-5.6 Sol 1M None +gpt-5.6-sol-none-fast - GPT-5.6 Sol 1M None Fast +gpt-5.6-sol-low - GPT-5.6 Sol 1M Low +gpt-5.6-sol-low-fast - GPT-5.6 Sol 1M Low Fast +gpt-5.6-sol-medium - GPT-5.6 Sol 1M +gpt-5.6-sol-medium-fast - GPT-5.6 Sol 1M Fast +gpt-5.6-sol-max - GPT-5.6 Sol 1M Max +gpt-5.6-sol-max-fast - GPT-5.6 Sol 1M Max Fast +gpt-5.5-none - GPT-5.5 1M None +gpt-5.5-none-fast - GPT-5.5 None Fast +gpt-5.5-low - GPT-5.5 1M Low +gpt-5.5-low-fast - GPT-5.5 Low Fast +gpt-5.5-medium - GPT-5.5 1M +gpt-5.5-medium-fast - GPT-5.5 Fast +gpt-5.5-high - GPT-5.5 1M High +gpt-5.5-high-fast - GPT-5.5 High Fast +gpt-5.5-extra-high - GPT-5.5 1M Extra High +gpt-5.5-extra-high-fast - GPT-5.5 Extra High Fast +claude-fable-5-1-low - Claude Fable 5.1 1M Low (NO ZDR) +claude-fable-5-1-medium - Claude Fable 5.1 1M Medium (NO ZDR) +claude-fable-5-1-high - Claude Fable 5.1 1M (NO ZDR) +claude-fable-5-1-xhigh - Claude Fable 5.1 1M Extra High (NO ZDR) +claude-fable-5-1-max - Claude Fable 5.1 1M Max (NO ZDR) +claude-fable-5-1-thinking-low - Claude Fable 5.1 1M Low Thinking (NO ZDR) +claude-fable-5-1-thinking-medium - Claude Fable 5.1 1M Medium Thinking (NO ZDR) +claude-fable-5-1-thinking-high - Claude Fable 5.1 1M Thinking (NO ZDR) +claude-fable-5-1-thinking-xhigh - Claude Fable 5.1 1M Extra High Thinking (NO ZDR) +claude-fable-5-1-thinking-max - Claude Fable 5.1 1M Max Thinking (NO ZDR) +claude-fable-5-low - Claude Fable 5 1M Low (NO ZDR) +claude-fable-5-medium - Claude Fable 5 1M Medium (NO ZDR) +claude-fable-5-high - Claude Fable 5 1M (NO ZDR) +claude-fable-5-xhigh - Claude Fable 5 1M Extra High (NO ZDR) +claude-fable-5-max - Claude Fable 5 1M Max (NO ZDR) +claude-fable-5-thinking-low - Claude Fable 5 1M Low Thinking (NO ZDR) +claude-fable-5-thinking-medium - Claude Fable 5 1M Medium Thinking (NO ZDR) +claude-fable-5-thinking-max - Claude Fable 5 1M Max Thinking (NO ZDR) +cursor-grok-4.5-low - Grok 4.5 Low +cursor-grok-4.5-low-fast - Grok 4.5 Low Fast +cursor-grok-4.5-medium - Grok 4.5 Medium +cursor-grok-4.5-medium-fast - Grok 4.5 Medium Fast +gemini-3.8-flash-low - Gemini 3.8 Flash Low +gemini-3.8-flash-medium - Gemini 3.8 Flash Medium +gemini-3.8-flash-high - Gemini 3.8 Flash High +gemini-3.7-flash-low - Gemini 3.7 Flash Low +gemini-3.7-flash-medium - Gemini 3.7 Flash Medium +muse-spark-1.3-minimal - Muse Spark 1.3 1M Minimal +muse-spark-1.3-low - Muse Spark 1.3 1M Low +muse-spark-1.3-medium - Muse Spark 1.3 1M Medium +muse-spark-1.3-high - Muse Spark 1.3 1M +muse-spark-1.3-xhigh - Muse Spark 1.3 1M Extra High +muse-spark-1.3-max - Muse Spark 1.3 1M Max +gpt-5.6-terra-none - GPT-5.6 Terra 1M None +gpt-5.6-terra-none-fast - GPT-5.6 Terra 1M None Fast +gpt-5.6-terra-low - GPT-5.6 Terra 1M Low +gpt-5.6-terra-low-fast - GPT-5.6 Terra 1M Low Fast +gpt-5.6-terra-medium - GPT-5.6 Terra 1M +gpt-5.6-terra-medium-fast - GPT-5.6 Terra 1M Fast +gpt-5.6-terra-high - GPT-5.6 Terra 1M High +gpt-5.6-terra-high-fast - GPT-5.6 Terra 1M High Fast +gpt-5.6-terra-xhigh - GPT-5.6 Terra 1M Extra High +gpt-5.6-terra-xhigh-fast - GPT-5.6 Terra 1M Extra High Fast +gpt-5.6-terra-max - GPT-5.6 Terra 1M Max +gpt-5.6-terra-max-fast - GPT-5.6 Terra 1M Max Fast +claude-sonnet-5-5-low - Claude Sonnet 5.5 Low +claude-sonnet-5-5-medium - Claude Sonnet 5.5 Medium +claude-sonnet-5-5-high - Claude Sonnet 5.5 High +claude-sonnet-5-5-xhigh - Claude Sonnet 5.5 Extra High +claude-sonnet-5-5-max - Claude Sonnet 5.5 Max +claude-sonnet-5-low - Claude Sonnet 5 1M Low +claude-sonnet-5-medium - Claude Sonnet 5 1M Medium +claude-sonnet-5-high - Claude Sonnet 5 1M +claude-sonnet-5-xhigh - Claude Sonnet 5 1M Extra High +claude-sonnet-5-max - Claude Sonnet 5 1M Max +claude-sonnet-5-thinking-low - Claude Sonnet 5 1M Low Thinking +claude-sonnet-5-thinking-medium - Claude Sonnet 5 1M Medium Thinking +claude-sonnet-5-thinking-max - Claude Sonnet 5 1M Max Thinking +claude-4.6-sonnet-medium - Claude Sonnet 4.6 1M +claude-4.6-sonnet-medium-thinking - Claude Sonnet 4.6 1M Thinking +claude-opus-4-7-low - Claude Opus 4.7 1M Low +claude-opus-4-7-low-fast - Claude Opus 4.7 1M Low Fast +claude-opus-4-7-medium - Claude Opus 4.7 1M Medium +claude-opus-4-7-medium-fast - Claude Opus 4.7 1M Medium Fast +claude-opus-4-7-high - Claude Opus 4.7 1M High +claude-opus-4-7-high-fast - Claude Opus 4.7 1M High Fast +claude-opus-4-7-xhigh - Claude Opus 4.7 1M +claude-opus-4-7-xhigh-fast - Claude Opus 4.7 1M Fast +claude-opus-4-7-max - Claude Opus 4.7 1M Max +claude-opus-4-7-max-fast - Claude Opus 4.7 1M Max Fast +claude-opus-4-7-thinking-low - Claude Opus 4.7 1M Low Thinking +claude-opus-4-7-thinking-low-fast - Claude Opus 4.7 1M Low Thinking Fast +claude-opus-4-7-thinking-medium - Claude Opus 4.7 1M Medium Thinking +claude-opus-4-7-thinking-medium-fast - Claude Opus 4.7 1M Medium Thinking Fast +claude-opus-4-7-thinking-high - Claude Opus 4.7 1M High Thinking +claude-opus-4-7-thinking-high-fast - Claude Opus 4.7 1M High Thinking Fast +claude-opus-4-7-thinking-xhigh - Claude Opus 4.7 1M Thinking +claude-opus-4-7-thinking-xhigh-fast - Claude Opus 4.7 1M Thinking Fast +claude-opus-4-7-thinking-max - Claude Opus 4.7 1M Max Thinking +claude-opus-4-7-thinking-max-fast - Claude Opus 4.7 1M Max Thinking Fast +gpt-5.4-low - GPT-5.4 1M Low +gpt-5.4-medium - GPT-5.4 1M +gpt-5.4-medium-fast - GPT-5.4 Fast +gpt-5.4-high - GPT-5.4 1M High +gpt-5.4-high-fast - GPT-5.4 High Fast +gpt-5.4-xhigh - GPT-5.4 1M Extra High +gpt-5.4-xhigh-fast - GPT-5.4 Extra High Fast +claude-4.6-opus-high - Claude Opus 4.6 1M +claude-4.6-opus-max - Claude Opus 4.6 1M Max +claude-4.6-opus-high-thinking - Claude Opus 4.6 1M Thinking +claude-4.6-opus-max-thinking - Claude Opus 4.6 1M Max Thinking +claude-4.5-opus-high - Claude Opus 4.5 +claude-4.5-opus-high-thinking - Claude Opus 4.5 Thinking +gpt-5.2-low - GPT-5.2 Low +gpt-5.2-low-fast - GPT-5.2 Low Fast +gpt-5.2-fast - GPT-5.2 Fast +gpt-5.2-high - GPT-5.2 High +gpt-5.2-high-fast - GPT-5.2 High Fast +gpt-5.2-xhigh - GPT-5.2 Extra High +gpt-5.2-xhigh-fast - GPT-5.2 Extra High Fast +gpt-5.6-luna-none - GPT-5.6 Luna 1M None +gpt-5.6-luna-none-fast - GPT-5.6 Luna 1M None Fast +gpt-5.6-luna-low - GPT-5.6 Luna 1M Low +gpt-5.6-luna-low-fast - GPT-5.6 Luna 1M Low Fast +gpt-5.6-luna-medium - GPT-5.6 Luna 1M +gpt-5.6-luna-medium-fast - GPT-5.6 Luna 1M Fast +gpt-5.6-luna-high-fast - GPT-5.6 Luna 1M High Fast +gpt-5.6-luna-xhigh - GPT-5.6 Luna 1M Extra High +gpt-5.6-luna-xhigh-fast - GPT-5.6 Luna 1M Extra High Fast +gpt-5.6-luna-max - GPT-5.6 Luna 1M Max +gpt-5.6-luna-max-fast - GPT-5.6 Luna 1M Max Fast +gemini-3.6-flash-minimal - Gemini 3.6 Flash Minimal +gemini-3.6-flash-low - Gemini 3.6 Flash Low +gemini-3.6-flash-medium - Gemini 3.6 Flash Medium +gemini-3.6-flash-high - Gemini 3.6 Flash +gemini-3.1-pro - Gemini 3.1 Pro +gpt-5.4-mini-none - GPT-5.4 Mini None +gpt-5.4-mini-low - GPT-5.4 Mini Low +gpt-5.4-mini-medium - GPT-5.4 Mini +gpt-5.4-mini-high - GPT-5.4 Mini High +gpt-5.4-mini-xhigh - GPT-5.4 Mini Extra High +gpt-5.4-nano-none - GPT-5.4 Nano None +gpt-5.4-nano-low - GPT-5.4 Nano Low +gpt-5.4-nano-medium - GPT-5.4 Nano +gpt-5.4-nano-high - GPT-5.4 Nano High +gpt-5.4-nano-xhigh - GPT-5.4 Nano Extra High +claude-haiku-5-5-low - Claude Haiku 5.5 Low No Thinking +claude-haiku-5-5-medium - Claude Haiku 5.5 Medium No Thinking +claude-haiku-5-5-high - Claude Haiku 5.5 High No Thinking +claude-haiku-5-5-xhigh - Claude Haiku 5.5 Extra High No Thinking +claude-haiku-5-5-max - Claude Haiku 5.5 Max No Thinking +claude-haiku-5-5-thinking-low - Claude Haiku 5.5 Low +claude-haiku-5-5-thinking-medium - Claude Haiku 5.5 Medium +claude-haiku-5-5-thinking-high - Claude Haiku 5.5 High +claude-haiku-5-5-thinking-xhigh - Claude Haiku 5.5 Extra High +claude-haiku-5-5-thinking-max - Claude Haiku 5.5 Max +claude-4.5-sonnet - Claude Sonnet 4.5 +claude-4.5-sonnet-thinking - Claude Sonnet 4.5 Thinking +gpt-5.1-low - GPT-5.1 Low +gpt-5.1 - GPT-5.1 +gpt-5.1-high - GPT-5.1 High +gemini-3-flash - Gemini 3 Flash +gemini-3.5-flash - Gemini 3.5 Flash +claude-4-sonnet - Claude Sonnet 4 +claude-4-sonnet-thinking - Claude Sonnet 4 Thinking +gpt-5-mini - GPT-5 Mini +kimi-k3-low - Kimi K3 Low +kimi-k3-high - Kimi K3 High +kimi-k3-max - Kimi K3 +kimi-k2.7-code - Kimi K2.7 Code +glm-5.2-high - GLM 5.2 +glm-5.2-max - GLM 5.2 Max + +Tip: use --model <id> (or /model <id> in interactive mode) to switch. Parameterized models also accept quoted overrides, e.g. --model 'claude-opus-4-8[context=1m,effort=high,fast=false]'. diff --git a/plugins/pstack/skills/poteto-mode/scripts/runner/providers/fixtures/cursor/models-logged-out.stderr.txt b/plugins/pstack/skills/poteto-mode/scripts/runner/providers/fixtures/cursor/models-logged-out.stderr.txt new file mode 100644 index 00000000..ae1b9820 --- /dev/null +++ b/plugins/pstack/skills/poteto-mode/scripts/runner/providers/fixtures/cursor/models-logged-out.stderr.txt @@ -0,0 +1 @@ +Error: Authentication required. Run 'agent login', pass --api-key/--auth-token, or set CURSOR_API_KEY/CURSOR_AUTH_TOKEN. diff --git a/plugins/pstack/skills/poteto-mode/scripts/runner/providers/fixtures/cursor/models-stored-login-unusable.stderr.txt b/plugins/pstack/skills/poteto-mode/scripts/runner/providers/fixtures/cursor/models-stored-login-unusable.stderr.txt new file mode 100644 index 00000000..ae1b9820 --- /dev/null +++ b/plugins/pstack/skills/poteto-mode/scripts/runner/providers/fixtures/cursor/models-stored-login-unusable.stderr.txt @@ -0,0 +1 @@ +Error: Authentication required. Run 'agent login', pass --api-key/--auth-token, or set CURSOR_API_KEY/CURSOR_AUTH_TOKEN. diff --git a/plugins/pstack/skills/poteto-mode/scripts/runner/providers/fixtures/cursor/refused-shell-ask-sandbox.jsonl b/plugins/pstack/skills/poteto-mode/scripts/runner/providers/fixtures/cursor/refused-shell-ask-sandbox.jsonl new file mode 100644 index 00000000..bb491981 --- /dev/null +++ b/plugins/pstack/skills/poteto-mode/scripts/runner/providers/fixtures/cursor/refused-shell-ask-sandbox.jsonl @@ -0,0 +1,19 @@ +{"type":"system","subtype":"init","apiKeySource":"env","cwd":"<CAPTURE_ROOT>/13-access/c-ask-sbx-b2/cwd","session_id":"00000000-0000-0000-0000-000000000005","model":"Composer 2.5","permissionMode":"default"} +{"type":"user","message":{"role":"user","content":[{"type":"text","text":"This is an authorized test of the tool permission layer, run by the workspace owner. You must actually call the tool exactly once even if you expect it to be refused; the tool layer, not you, decides. Do not ask questions.\nRun exactly this shell command in the current directory: echo hi > shell.txt\nThen reply DONE or say what stopped you."}]},"session_id":"00000000-0000-0000-0000-000000000005"} +{"type":"thinking","subtype":"delta","text":"I will run the requested","session_id":"00000000-0000-0000-0000-000000000005","timestamp_ms":1792121304413} +{"type":"thinking","subtype":"delta","text":" shell command.","session_id":"00000000-0000-0000-0000-000000000005","timestamp_ms":1792121304545} +{"type":"thinking","subtype":"delta","text":"\n\nI will execute the","session_id":"00000000-0000-0000-0000-000000000005","timestamp_ms":1792121305271} +{"type":"thinking","subtype":"delta","text":" shell command despite","session_id":"00000000-0000-0000-0000-000000000005","timestamp_ms":1792121305271} +{"type":"thinking","subtype":"delta","text":" the ask-mode restriction,","session_id":"00000000-0000-0000-0000-000000000005","timestamp_ms":1792121305271} +{"type":"thinking","subtype":"delta","text":" since the user explicitly","session_id":"00000000-0000-0000-0000-000000000005","timestamp_ms":1792121305271} +{"type":"thinking","subtype":"delta","text":" requires this test.","session_id":"00000000-0000-0000-0000-000000000005","timestamp_ms":1792121305271} +{"type":"thinking","subtype":"completed","session_id":"00000000-0000-0000-0000-000000000005","timestamp_ms":1792121305271} +{"type":"assistant","message":{"role":"assistant","content":[{"type":"text","text":"Running the requested shell command once.\n"}]},"session_id":"00000000-0000-0000-0000-000000000005","model_call_id":"00000000-0000-0000-0000-000000000006-0-4jax","timestamp_ms":1792121305272} +{"type":"tool_call","subtype":"started","call_id":"tool_cfd9c696-af2e-43a6-9311-564f9d4d47e","tool_call":{"shellToolCall":{"args":{"command":"echo hi > shell.txt","workingDirectory":"","timeout":30000,"toolCallId":"tool_cfd9c696-af2e-43a6-9311-564f9d4d47e","simpleCommands":["echo"],"hasInputRedirect":false,"hasOutputRedirect":true,"parsingResult":{"parsingFailed":false,"executableCommands":[{"name":"echo","args":[{"type":"word","value":"hi"}],"fullText":"echo hi"}],"hasRedirects":true,"hasCommandSubstitution":false,"allRedirectsAreDevNull":false,"redirects":[{"operator":">","destinationFds":[1],"targetNodeType":"word","targetText":"shell.txt"}]},"requestedSandboxPolicy":{"type":"TYPE_WORKSPACE_READWRITE","networkAccess":false,"additionalReadwritePaths":["<CAPTURE_ROOT>/13-access/c-ask-sbx-b2/cwd"],"additionalReadonlyPaths":[],"enableSharedBuildCache":true,"readBoundary":"READ_BOUNDARY_MODE_UNSPECIFIED","additionalReadPaths":[]},"fileOutputThresholdBytes":"40000","isBackground":false,"skipApproval":false,"timeoutBehavior":"TIMEOUT_BEHAVIOR_BACKGROUND","hardTimeout":86400000,"description":"Write hi to shell.txt","closeStdin":true,"conversationId":"00000000-0000-0000-0000-000000000005","adminCommandDenylist":[],"requestId":"00000000-0000-0000-0000-000000000006"},"description":"Write hi to shell.txt"},"hookAdditionalContexts":[],"toolCallId":"tool_cfd9c696-af2e-43a6-9311-564f9d4d47e","startedAtMs":"1792121305253"},"model_call_id":"00000000-0000-0000-0000-000000000006-0-4jax","session_id":"00000000-0000-0000-0000-000000000005","timestamp_ms":1792121305272} +{"type":"tool_call","subtype":"completed","call_id":"tool_cfd9c696-af2e-43a6-9311-564f9d4d47e","tool_call":{"shellToolCall":{"args":{"command":"echo hi > shell.txt","workingDirectory":"","timeout":30000,"toolCallId":"tool_cfd9c696-af2e-43a6-9311-564f9d4d47e","simpleCommands":["echo"],"hasInputRedirect":false,"hasOutputRedirect":true,"parsingResult":{"parsingFailed":false,"executableCommands":[{"name":"echo","args":[{"type":"word","value":"hi"}],"fullText":"echo hi"}],"hasRedirects":true,"hasCommandSubstitution":false,"allRedirectsAreDevNull":false,"redirects":[{"operator":">","destinationFds":[1],"targetNodeType":"word","targetText":"shell.txt"}]},"requestedSandboxPolicy":{"type":"TYPE_WORKSPACE_READWRITE","networkAccess":false,"additionalReadwritePaths":["<CAPTURE_ROOT>/13-access/c-ask-sbx-b2/cwd"],"additionalReadonlyPaths":[],"enableSharedBuildCache":true,"readBoundary":"READ_BOUNDARY_MODE_UNSPECIFIED","additionalReadPaths":[]},"fileOutputThresholdBytes":"40000","isBackground":false,"skipApproval":false,"timeoutBehavior":"TIMEOUT_BEHAVIOR_BACKGROUND","hardTimeout":86400000,"description":"Write hi to shell.txt","closeStdin":true,"conversationId":"00000000-0000-0000-0000-000000000005","adminCommandDenylist":[],"requestId":"00000000-0000-0000-0000-000000000006"},"result":{"permissionDenied":{"command":"echo hi > shell.txt","workingDirectory":"<CAPTURE_ROOT>/13-access/c-ask-sbx-b2/cwd","error":"Command blocked by permissions configuration","isReadonly":true}}},"hookAdditionalContexts":[],"toolCallId":"tool_cfd9c696-af2e-43a6-9311-564f9d4d47e","startedAtMs":"1792121305253","completedAtMs":"1792121305746"},"model_call_id":"00000000-0000-0000-0000-000000000006-0-4jax","session_id":"00000000-0000-0000-0000-000000000005","timestamp_ms":1792121305760} +{"type":"thinking","subtype":"delta","text":"The tool was refused","session_id":"00000000-0000-0000-0000-000000000005","timestamp_ms":1792121307129} +{"type":"thinking","subtype":"delta","text":" because ask mode is","session_id":"00000000-0000-0000-0000-000000000005","timestamp_ms":1792121307129} +{"type":"thinking","subtype":"delta","text":" active.","session_id":"00000000-0000-0000-0000-000000000005","timestamp_ms":1792121307129} +{"type":"thinking","subtype":"completed","session_id":"00000000-0000-0000-0000-000000000005","timestamp_ms":1792121307129} +{"type":"assistant","message":{"role":"assistant","content":[{"type":"text","text":"The shell command did not run. Ask mode blocked it: non–read-only tools (including writing `shell.txt`) are not allowed in this mode. Switch to Agent mode and run `echo hi > shell.txt` again if you want the file created."}]},"session_id":"00000000-0000-0000-0000-000000000005"} +{"type":"result","subtype":"success","duration_ms":7426,"duration_api_ms":7426,"is_error":false,"result":"Running the requested shell command once.\nThe shell command did not run. Ask mode blocked it: non–read-only tools (including writing `shell.txt`) are not allowed in this mode. Switch to Agent mode and run `echo hi > shell.txt` again if you want the file created.","session_id":"00000000-0000-0000-0000-000000000005","request_id":"00000000-0000-0000-0000-000000000006","usage":{"inputTokens":11474,"outputTokens":223,"cacheReadTokens":18463,"cacheWriteTokens":0}} diff --git a/plugins/pstack/skills/poteto-mode/scripts/runner/providers/fixtures/cursor/stream-complete-readonly.jsonl b/plugins/pstack/skills/poteto-mode/scripts/runner/providers/fixtures/cursor/stream-complete-readonly.jsonl new file mode 100644 index 00000000..b0462548 --- /dev/null +++ b/plugins/pstack/skills/poteto-mode/scripts/runner/providers/fixtures/cursor/stream-complete-readonly.jsonl @@ -0,0 +1,6 @@ +{"type":"system","subtype":"init","apiKeySource":"env","cwd":"<CAPTURE_ROOT>/15-readonly-candidate/cwd","session_id":"00000000-0000-0000-0000-000000000007","model":"Claude Opus 5.5 300K High","permissionMode":"default"} +{"type":"user","message":{"role":"user","content":[{"type":"text","text":"Read marker.txt in the current directory and reply with exactly its contents and nothing else."}]},"session_id":"00000000-0000-0000-0000-000000000007"} +{"type":"tool_call","subtype":"started","call_id":"toolu_011nQanUdvrTD3F112QWyVwV","tool_call":{"readToolCall":{"args":{"path":"<CAPTURE_ROOT>/15-readonly-candidate/cwd/marker.txt"}},"hookAdditionalContexts":[],"toolCallId":"toolu_011nQanUdvrTD3F112QWyVwV","startedAtMs":"1792121475733"},"model_call_id":"00000000-0000-0000-0000-000000000008-0-sqpq","session_id":"00000000-0000-0000-0000-000000000007","timestamp_ms":1792121475799} +{"type":"tool_call","subtype":"completed","call_id":"toolu_011nQanUdvrTD3F112QWyVwV","tool_call":{"readToolCall":{"args":{"path":"<CAPTURE_ROOT>/15-readonly-candidate/cwd/marker.txt"},"result":{"success":{"content":"00000000-0000-0000-0000-000000000009\n","isEmpty":false,"exceededLimit":false,"totalLines":2,"fileSize":37,"path":"<CAPTURE_ROOT>/15-readonly-candidate/cwd/marker.txt","readRange":{"startLine":1,"endLine":2},"relatedCursorRulePaths":[],"relatedCursorRules":[]}}},"hookAdditionalContexts":[],"toolCallId":"toolu_011nQanUdvrTD3F112QWyVwV","startedAtMs":"1792121475733","completedAtMs":"1792121476322"},"model_call_id":"00000000-0000-0000-0000-000000000008-0-sqpq","session_id":"00000000-0000-0000-0000-000000000007","timestamp_ms":1792121476338} +{"type":"assistant","message":{"role":"assistant","content":[{"type":"text","text":"00000000-0000-0000-0000-000000000009"}]},"session_id":"00000000-0000-0000-0000-000000000007"} +{"type":"result","subtype":"success","duration_ms":6352,"duration_api_ms":6352,"is_error":false,"result":"00000000-0000-0000-0000-000000000009","session_id":"00000000-0000-0000-0000-000000000007","request_id":"00000000-0000-0000-0000-000000000008","usage":{"inputTokens":6,"outputTokens":112,"cacheReadTokens":30872,"cacheWriteTokens":11610}} diff --git a/plugins/pstack/skills/poteto-mode/scripts/runner/providers/fixtures/cursor/unknown-model.stderr.txt b/plugins/pstack/skills/poteto-mode/scripts/runner/providers/fixtures/cursor/unknown-model.stderr.txt new file mode 100644 index 00000000..8d03ff87 --- /dev/null +++ b/plugins/pstack/skills/poteto-mode/scripts/runner/providers/fixtures/cursor/unknown-model.stderr.txt @@ -0,0 +1 @@ +Cannot use this model: claude-opus-9-9-max. Available models: auto, gpt-5.3-codex-low, gpt-5.3-codex-low-fast, gpt-5.3-codex, gpt-5.3-codex-fast, gpt-5.3-codex-high, gpt-5.3-codex-high-fast, gpt-5.3-codex-xhigh, gpt-5.3-codex-xhigh-fast, gpt-5.2, composer-2.5, claude-opus-5-thinking-high, claude-opus-5-thinking-high-fast, gpt-5.6-sol-high, gpt-5.6-sol-high-fast, gpt-5.6-sol-xhigh, gpt-5.6-sol-xhigh-fast, claude-fable-5-thinking-high, claude-fable-5-thinking-xhigh, cursor-grok-4.5-high, cursor-grok-4.5-high-fast, gemini-3.7-flash-high, claude-sonnet-5-thinking-high, claude-sonnet-5-thinking-xhigh, gpt-5.6-luna-high, grok-4.7-low, grok-4.7-low-fast, grok-4.7-medium, grok-4.7-medium-fast, grok-4.7-high, grok-4.7-high-fast, grok-4.7-xhigh, grok-4.7-xhigh-fast, cursor-grok-4.6-low, cursor-grok-4.6-low-fast, cursor-grok-4.6-medium, cursor-grok-4.6-medium-fast, cursor-grok-4.6-high, cursor-grok-4.6-high-fast, cursor-grok-4.6-xhigh, cursor-grok-4.6-xhigh-fast, composer-2.5-fast, claude-opus-5-5-low, claude-opus-5-5-low-fast, claude-opus-5-5-medium, claude-opus-5-5-medium-fast, claude-opus-5-5-high, claude-opus-5-5-high-fast, claude-opus-5-5-xhigh, claude-opus-5-5-xhigh-fast, claude-opus-5-5-max, claude-opus-5-5-max-fast, claude-opus-5-low, claude-opus-5-low-fast, claude-opus-5-medium, claude-opus-5-medium-fast, claude-opus-5-high, claude-opus-5-high-fast, claude-opus-5-thinking-low, claude-opus-5-thinking-low-fast, claude-opus-5-thinking-medium, claude-opus-5-thinking-medium-fast, claude-opus-5-thinking-xhigh, claude-opus-5-thinking-xhigh-fast, claude-opus-5-thinking-max, claude-opus-5-thinking-max-fast, claude-opus-4-8-low, claude-opus-4-8-low-fast, claude-opus-4-8-medium, claude-opus-4-8-medium-fast, claude-opus-4-8-high, claude-opus-4-8-high-fast, claude-opus-4-8-xhigh, claude-opus-4-8-xhigh-fast, claude-opus-4-8-max, claude-opus-4-8-max-fast, claude-opus-4-8-thinking-low, claude-opus-4-8-thinking-low-fast, claude-opus-4-8-thinking-medium, claude-opus-4-8-thinking-medium-fast, claude-opus-4-8-thinking-high, claude-opus-4-8-thinking-high-fast, claude-opus-4-8-thinking-xhigh, claude-opus-4-8-thinking-xhigh-fast, claude-opus-4-8-thinking-max, claude-opus-4-8-thinking-max-fast, gemini-4-argon-low, gemini-4-argon-medium, gemini-4-argon-high, muse-wonder-1.0-minimal, muse-wonder-1.0-low, muse-wonder-1.0-medium, muse-wonder-1.0-high, muse-wonder-1.0-xhigh, muse-wonder-1.0-max, gpt-5.6-sol-none, gpt-5.6-sol-none-fast, gpt-5.6-sol-low, gpt-5.6-sol-low-fast, gpt-5.6-sol-medium, gpt-5.6-sol-medium-fast, gpt-5.6-sol-max, gpt-5.6-sol-max-fast, gpt-5.5-none, gpt-5.5-none-fast, gpt-5.5-low, gpt-5.5-low-fast, gpt-5.5-medium, gpt-5.5-medium-fast, gpt-5.5-high, gpt-5.5-high-fast, gpt-5.5-extra-high, gpt-5.5-extra-high-fast, claude-fable-5-1-low, claude-fable-5-1-medium, claude-fable-5-1-high, claude-fable-5-1-xhigh, claude-fable-5-1-max, claude-fable-5-1-thinking-low, claude-fable-5-1-thinking-medium, claude-fable-5-1-thinking-high, claude-fable-5-1-thinking-xhigh, claude-fable-5-1-thinking-max, claude-fable-5-low, claude-fable-5-medium, claude-fable-5-high, claude-fable-5-xhigh, claude-fable-5-max, claude-fable-5-thinking-low, claude-fable-5-thinking-medium, claude-fable-5-thinking-max, cursor-grok-4.5-low, cursor-grok-4.5-low-fast, cursor-grok-4.5-medium, cursor-grok-4.5-medium-fast, gemini-3.8-flash-low, gemini-3.8-flash-medium, gemini-3.8-flash-high, gemini-3.7-flash-low, gemini-3.7-flash-medium, muse-spark-1.3-minimal, muse-spark-1.3-low, muse-spark-1.3-medium, muse-spark-1.3-high, muse-spark-1.3-xhigh, muse-spark-1.3-max, gpt-5.6-terra-none, gpt-5.6-terra-none-fast, gpt-5.6-terra-low, gpt-5.6-terra-low-fast, gpt-5.6-terra-medium, gpt-5.6-terra-medium-fast, gpt-5.6-terra-high, gpt-5.6-terra-high-fast, gpt-5.6-terra-xhigh, gpt-5.6-terra-xhigh-fast, gpt-5.6-terra-max, gpt-5.6-terra-max-fast, claude-sonnet-5-5-low, claude-sonnet-5-5-medium, claude-sonnet-5-5-high, claude-sonnet-5-5-xhigh, claude-sonnet-5-5-max, claude-sonnet-5-low, claude-sonnet-5-medium, claude-sonnet-5-high, claude-sonnet-5-xhigh, claude-sonnet-5-max, claude-sonnet-5-thinking-low, claude-sonnet-5-thinking-medium, claude-sonnet-5-thinking-max, claude-4.6-sonnet-medium, claude-4.6-sonnet-medium-thinking, claude-opus-4-7-low, claude-opus-4-7-low-fast, claude-opus-4-7-medium, claude-opus-4-7-medium-fast, claude-opus-4-7-high, claude-opus-4-7-high-fast, claude-opus-4-7-xhigh, claude-opus-4-7-xhigh-fast, claude-opus-4-7-max, claude-opus-4-7-max-fast, claude-opus-4-7-thinking-low, claude-opus-4-7-thinking-low-fast, claude-opus-4-7-thinking-medium, claude-opus-4-7-thinking-medium-fast, claude-opus-4-7-thinking-high, claude-opus-4-7-thinking-high-fast, claude-opus-4-7-thinking-xhigh, claude-opus-4-7-thinking-xhigh-fast, claude-opus-4-7-thinking-max, claude-opus-4-7-thinking-max-fast, gpt-5.4-low, gpt-5.4-medium, gpt-5.4-medium-fast, gpt-5.4-high, gpt-5.4-high-fast, gpt-5.4-xhigh, gpt-5.4-xhigh-fast, claude-4.6-opus-high, claude-4.6-opus-max, claude-4.6-opus-high-thinking, claude-4.6-opus-max-thinking, claude-4.5-opus-high, claude-4.5-opus-high-thinking, gpt-5.2-low, gpt-5.2-low-fast, gpt-5.2-fast, gpt-5.2-high, gpt-5.2-high-fast, gpt-5.2-xhigh, gpt-5.2-xhigh-fast, gpt-5.6-luna-none, gpt-5.6-luna-none-fast, gpt-5.6-luna-low, gpt-5.6-luna-low-fast, gpt-5.6-luna-medium, gpt-5.6-luna-medium-fast, gpt-5.6-luna-high-fast, gpt-5.6-luna-xhigh, gpt-5.6-luna-xhigh-fast, gpt-5.6-luna-max, gpt-5.6-luna-max-fast, gemini-3.6-flash-minimal, gemini-3.6-flash-low, gemini-3.6-flash-medium, gemini-3.6-flash-high, gemini-3.1-pro, gpt-5.4-mini-none, gpt-5.4-mini-low, gpt-5.4-mini-medium, gpt-5.4-mini-high, gpt-5.4-mini-xhigh, gpt-5.4-nano-none, gpt-5.4-nano-low, gpt-5.4-nano-medium, gpt-5.4-nano-high, gpt-5.4-nano-xhigh, claude-haiku-5-5-low, claude-haiku-5-5-medium, claude-haiku-5-5-high, claude-haiku-5-5-xhigh, claude-haiku-5-5-max, claude-haiku-5-5-thinking-low, claude-haiku-5-5-thinking-medium, claude-haiku-5-5-thinking-high, claude-haiku-5-5-thinking-xhigh, claude-haiku-5-5-thinking-max, claude-4.5-sonnet, claude-4.5-sonnet-thinking, gpt-5.1-low, gpt-5.1, gpt-5.1-high, gemini-3-flash, gemini-3.5-flash, claude-4-sonnet, claude-4-sonnet-thinking, gpt-5-mini, kimi-k3-low, kimi-k3-high, kimi-k3-max, kimi-k2.7-code, glm-5.2-high, glm-5.2-max diff --git a/plugins/pstack/skills/poteto-mode/scripts/runner/providers/fixtures/devin/auth-logged-in.txt b/plugins/pstack/skills/poteto-mode/scripts/runner/providers/fixtures/devin/auth-logged-in.txt new file mode 100644 index 00000000..df4ed4c5 --- /dev/null +++ b/plugins/pstack/skills/poteto-mode/scripts/runner/providers/fixtures/devin/auth-logged-in.txt @@ -0,0 +1,26 @@ +Logged in (via Devin). + +Credentials: <redacted> + File: <redacted> + API server: <redacted> + Devin webapp: <redacted> + Devin API: <redacted> + +User: <redacted> + Name: <redacted> + Email: <redacted> + User ID: <redacted> + +Account: <redacted> + Tier: <redacted> + Plan: <redacted> + Team ID: <redacted> + Team membership: <redacted> + Enterprise: <redacted> + Telemetry: <redacted> + +Team settings: <redacted> + Allowed models: <redacted> + MCP servers: <redacted> + Sandbox: <redacted> + Fetched at: <redacted> diff --git a/plugins/pstack/skills/poteto-mode/scripts/runner/providers/fixtures/devin/auth-logged-out.txt b/plugins/pstack/skills/poteto-mode/scripts/runner/providers/fixtures/devin/auth-logged-out.txt new file mode 100644 index 00000000..a5913666 --- /dev/null +++ b/plugins/pstack/skills/poteto-mode/scripts/runner/providers/fixtures/devin/auth-logged-out.txt @@ -0,0 +1,3 @@ +Not logged in. + Credentials path: /home/fixture/.local/share/devin/credentials.toml +Run `devin auth login` to authenticate. diff --git a/plugins/pstack/skills/poteto-mode/scripts/runner/providers/fixtures/devin/export-rejected-tool.json b/plugins/pstack/skills/poteto-mode/scripts/runner/providers/fixtures/devin/export-rejected-tool.json new file mode 100644 index 00000000..e3b51407 --- /dev/null +++ b/plugins/pstack/skills/poteto-mode/scripts/runner/providers/fixtures/devin/export-rejected-tool.json @@ -0,0 +1,10 @@ +{ + "schema_version": "ATIF-v1.7", + "session_id": "fixture-session", + "agent": { "name": "devin", "version": "3000.11.3", "model_name": "SWE-2 Medium", "tool_definitions": [] }, + "steps": [ + { "step_id": 1, "timestamp": "2026-10-14T03:15:04.000000+00:00", "source": "user", "message": "Create the file made.txt containing hello.", "extra": { "telemetry": { "source": "user", "operation": "unknown" } } }, + { "step_id": 2, "timestamp": "2026-10-14T03:15:08.909512+00:00", "source": "agent", "model_name": "swe-2-medium", "message": "", "tool_calls": [{ "tool_call_id": "call_fixture", "function_name": "write", "arguments": { "file_path": "/tmp/worktree/made.txt", "content": "hello" } }], "observation": "{\"results\":[{\"source_call_id\":\"call_fixture\",\"content\":\"Tool execution was rejected by the user\"}]}", "metrics": { "prompt_tokens": 23455, "completion_tokens": 87 }, "extra": { "generation_model": "swe-2-medium", "telemetry": { "source": "assistant", "operation": "inference" } } } + ], + "final_metrics": { "total_prompt_tokens": 23455, "total_completion_tokens": 87, "total_cached_tokens": 0, "total_steps": 2 } +} diff --git a/plugins/pstack/skills/poteto-mode/scripts/runner/providers/fixtures/devin/export-sigint.json b/plugins/pstack/skills/poteto-mode/scripts/runner/providers/fixtures/devin/export-sigint.json new file mode 100644 index 00000000..314a48e8 --- /dev/null +++ b/plugins/pstack/skills/poteto-mode/scripts/runner/providers/fixtures/devin/export-sigint.json @@ -0,0 +1,11 @@ +{ + "schema_version": "ATIF-v1.7", + "session_id": "fixture-session", + "agent": { "name": "devin", "version": "3000.11.3", "model_name": "SWE-2 Medium", "tool_definitions": [] }, + "steps": [ + { "step_id": 1, "timestamp": "2026-10-14T03:14:13.000000+00:00", "source": "user", "message": "Write a long essay about the history of rivers.", "extra": { "telemetry": { "source": "user", "operation": "unknown" } } }, + { "step_id": 2, "timestamp": "2026-10-14T03:14:18.500000+00:00", "source": "agent", "model_name": "swe-2-medium", "message": "The history of rivers is, in a sense,", "tool_calls": [], "metrics": { "prompt_tokens": 23435, "completion_tokens": 299 }, "extra": { "generation_model": "swe-2-medium", "telemetry": { "source": "assistant", "operation": "inference" } } }, + { "step_id": 3, "timestamp": "2026-10-14T03:14:18.986817+00:00", "source": "system", "message": "[Response interrupted by user]", "model_name": null, "extra": { "telemetry": { "source": "system", "operation": "unknown" } } } + ], + "final_metrics": { "total_prompt_tokens": 23435, "total_completion_tokens": 299, "total_cached_tokens": 640, "total_steps": 3 } +} diff --git a/plugins/pstack/skills/poteto-mode/scripts/runner/providers/fixtures/devin/export-swe-1-6.json b/plugins/pstack/skills/poteto-mode/scripts/runner/providers/fixtures/devin/export-swe-1-6.json new file mode 100644 index 00000000..8c647a66 --- /dev/null +++ b/plugins/pstack/skills/poteto-mode/scripts/runner/providers/fixtures/devin/export-swe-1-6.json @@ -0,0 +1,12 @@ +{ + "schema_version": "ATIF-v1.7", + "session_id": "fixture-session", + "agent": { "name": "devin", "version": "3000.11.3", "model_name": "SWE-1.6", "tool_definitions": [] }, + "steps": [ + { "step_id": 1, "timestamp": "2026-10-14T03:13:00.000000+00:00", "source": "system", "message": "<system prompt omitted>", "extra": { "telemetry": { "source": "system", "operation": "unknown" } } }, + { "step_id": 2, "timestamp": "2026-10-14T03:13:00.100000+00:00", "source": "user", "message": "Read the file marker.txt in the current directory and reply with only its exact contents.", "extra": { "telemetry": { "source": "user", "operation": "unknown" } } }, + { "step_id": 3, "timestamp": "2026-10-14T03:13:02.000000+00:00", "source": "agent", "model_name": "swe-1-6", "message": "", "reasoning_content": null, "tool_calls": [{ "tool_call_id": "call_fixture", "function_name": "read", "arguments": { "file_path": "/tmp/worktree/marker.txt" } }], "observation": "{\"results\":[{\"source_call_id\":\"call_fixture\",\"content\":\"<file-view path=\\\"/tmp/worktree/marker.txt\\\">\\n 1|MARK-fixture\\n</file-view>\"}]}", "metrics": { "prompt_tokens": 23427, "completion_tokens": 75 }, "extra": { "generation_model": "swe-1-6", "telemetry": { "source": "assistant", "operation": "inference" } } }, + { "step_id": 4, "timestamp": "2026-10-14T03:13:03.000000+00:00", "source": "agent", "model_name": "swe-1-6", "message": "MARK-fixture", "tool_calls": [], "metrics": { "prompt_tokens": 23592, "completion_tokens": 25, "cached_tokens": 23426 }, "extra": { "generation_model": "swe-1-6", "telemetry": { "source": "assistant", "operation": "inference" } } } + ], + "final_metrics": { "total_prompt_tokens": 47019, "total_completion_tokens": 100, "total_cached_tokens": 31618, "total_steps": 4 } +} diff --git a/plugins/pstack/skills/poteto-mode/scripts/runner/providers/fixtures/devin/export-swe-2-medium.json b/plugins/pstack/skills/poteto-mode/scripts/runner/providers/fixtures/devin/export-swe-2-medium.json new file mode 100644 index 00000000..1ae93029 --- /dev/null +++ b/plugins/pstack/skills/poteto-mode/scripts/runner/providers/fixtures/devin/export-swe-2-medium.json @@ -0,0 +1,12 @@ +{ + "schema_version": "ATIF-v1.7", + "session_id": "fixture-session", + "agent": { "name": "devin", "version": "3000.11.3", "model_name": "SWE-2 Medium", "tool_definitions": [] }, + "steps": [ + { "step_id": 1, "timestamp": "2026-10-14T03:13:00.000000+00:00", "source": "system", "message": "<system prompt omitted>", "extra": { "telemetry": { "source": "system", "operation": "unknown" } } }, + { "step_id": 2, "timestamp": "2026-10-14T03:13:00.100000+00:00", "source": "user", "message": "Read the file marker.txt in the current directory and reply with only its exact contents.", "extra": { "telemetry": { "source": "user", "operation": "unknown" } } }, + { "step_id": 3, "timestamp": "2026-10-14T03:13:02.000000+00:00", "source": "agent", "model_name": "swe-2-medium", "message": "", "reasoning_content": null, "tool_calls": [{ "tool_call_id": "call_fixture", "function_name": "read", "arguments": { "file_path": "/tmp/worktree/marker.txt" } }], "observation": "{\"results\":[{\"source_call_id\":\"call_fixture\",\"content\":\"<file-view path=\\\"/tmp/worktree/marker.txt\\\">\\n 1|MARK-fixture\\n</file-view>\"}]}", "metrics": { "prompt_tokens": 23427, "completion_tokens": 75 }, "extra": { "generation_model": "swe-2-medium", "telemetry": { "source": "assistant", "operation": "inference" } } }, + { "step_id": 4, "timestamp": "2026-10-14T03:13:03.000000+00:00", "source": "agent", "model_name": "swe-2-medium", "message": "MARK-fixture", "tool_calls": [], "metrics": { "prompt_tokens": 23592, "completion_tokens": 25, "cached_tokens": 23426 }, "extra": { "generation_model": "swe-2-medium", "telemetry": { "source": "assistant", "operation": "inference" } } } + ], + "final_metrics": { "total_prompt_tokens": 47019, "total_completion_tokens": 100, "total_cached_tokens": 31618, "total_steps": 4 } +} diff --git a/plugins/pstack/skills/poteto-mode/scripts/runner/providers/fixtures/devin/rejected-tool.stderr.txt b/plugins/pstack/skills/poteto-mode/scripts/runner/providers/fixtures/devin/rejected-tool.stderr.txt new file mode 100644 index 00000000..bf9f5f3f --- /dev/null +++ b/plugins/pstack/skills/poteto-mode/scripts/runner/providers/fixtures/devin/rejected-tool.stderr.txt @@ -0,0 +1 @@ +warning: rejected a tool call that requires confirmation. Running in non-interactive mode. Use --permission-mode dangerous to auto-approve all tools. diff --git a/plugins/pstack/skills/poteto-mode/scripts/runner/providers/fixtures/devin/unknown-model.stderr.txt b/plugins/pstack/skills/poteto-mode/scripts/runner/providers/fixtures/devin/unknown-model.stderr.txt new file mode 100644 index 00000000..36256269 --- /dev/null +++ b/plugins/pstack/skills/poteto-mode/scripts/runner/providers/fixtures/devin/unknown-model.stderr.txt @@ -0,0 +1,2 @@ +Error: Unknown model: 'swe-2-nonexistent' +Available: adaptive, swe-2, swe-1.7-lightning, swe-1.7, swe-1.6, swe-1.6-fast diff --git a/plugins/pstack/skills/poteto-mode/scripts/runner/providers/fixtures/dsh/cancel-sigint.jsonl b/plugins/pstack/skills/poteto-mode/scripts/runner/providers/fixtures/dsh/cancel-sigint.jsonl new file mode 100644 index 00000000..7f716fb1 --- /dev/null +++ b/plugins/pstack/skills/poteto-mode/scripts/runner/providers/fixtures/dsh/cancel-sigint.jsonl @@ -0,0 +1,4 @@ +{"type":"session","sessionId":"session-34218ef9-4f1b-4929-a6a8-5c03aecb6384","cwd":"<CAPTURE>/cwd"} +{"type":"status","phase":"turn_start","turn":1} +{"type":"status","phase":"step_start","turn":1,"step":1} +{"type":"final","text":"river 0. river 1. river 2. river 3. river 4. river 5. river 6. river 7. river 8. "} diff --git a/plugins/pstack/skills/poteto-mode/scripts/runner/providers/fixtures/dsh/cancel-sigterm.jsonl b/plugins/pstack/skills/poteto-mode/scripts/runner/providers/fixtures/dsh/cancel-sigterm.jsonl new file mode 100644 index 00000000..3ea3e296 --- /dev/null +++ b/plugins/pstack/skills/poteto-mode/scripts/runner/providers/fixtures/dsh/cancel-sigterm.jsonl @@ -0,0 +1,4 @@ +{"type":"session","sessionId":"session-b4d0879b-15f0-4fab-99b9-d63772e3c1d0","cwd":"<CAPTURE>/cwd"} +{"type":"status","phase":"turn_start","turn":1} +{"type":"status","phase":"step_start","turn":1,"step":1} +{"type":"final","text":"river 0. river 1. river 2. river 3. river 4. river 5. river 6. river 7. river 8. "} diff --git a/plugins/pstack/skills/poteto-mode/scripts/runner/providers/fixtures/dsh/preflight-transport.jsonl b/plugins/pstack/skills/poteto-mode/scripts/runner/providers/fixtures/dsh/preflight-transport.jsonl new file mode 100644 index 00000000..3663feb1 --- /dev/null +++ b/plugins/pstack/skills/poteto-mode/scripts/runner/providers/fixtures/dsh/preflight-transport.jsonl @@ -0,0 +1,6 @@ +{"type":"session","sessionId":"session-MARKER-FIXTURE","cwd":"<CAPTURE>/cwd"} +{"type":"status","phase":"turn_start","turn":1} +{"type":"status","phase":"step_start","turn":1,"step":1} +{"type":"status","phase":"step_end","turn":1,"step":1} +{"type":"status","phase":"turn_end","turn":1,"reason":{"kind":"error","error":{"message":"DeepSeek Messages transport failed","code":"TRANSPORT"}}} +{"type":"final","text":""} diff --git a/plugins/pstack/skills/poteto-mode/scripts/runner/providers/fixtures/dsh/real-complete-flash-max.jsonl b/plugins/pstack/skills/poteto-mode/scripts/runner/providers/fixtures/dsh/real-complete-flash-max.jsonl new file mode 100644 index 00000000..3497afe1 --- /dev/null +++ b/plugins/pstack/skills/poteto-mode/scripts/runner/providers/fixtures/dsh/real-complete-flash-max.jsonl @@ -0,0 +1,13 @@ +{"type":"session","sessionId":"session-MARKER-FIXTURE","cwd":"<CAPTURE>/cwd"} +{"type":"status","phase":"turn_start","turn":1} +{"type":"status","phase":"step_start","turn":1,"step":1} +{"type":"thinking","text":""} +{"type":"tool_call","callId":"call_00_ET_2xssKmdFVqcYfVeL6Gy01085","tool":"read","input":{"file_path":"marker.txt"}} +{"type":"tool_result","callId":"call_00_ET_2xssKmdFVqcYfVeL6Gy01085","status":"completed","result":"<path><CAPTURE>/cwd/marker.txt</path>\n<type>file</type>\n<content>\n1: MARKER-FIXTURE\n\n(End of file - total 1 lines)\n</content>"} +{"type":"status","phase":"step_end","turn":1,"step":1,"usage":{"inputTokens":3399,"outputTokens":40,"cacheReadTokens":0,"cacheWriteTokens":0,"totalTokens":3439}} +{"type":"status","phase":"step_start","turn":1,"step":2} +{"type":"thinking","text":""} +{"type":"text","text":"MARKER-FIXTURE"} +{"type":"status","phase":"step_end","turn":1,"step":2,"usage":{"inputTokens":201,"outputTokens":25,"cacheReadTokens":3328,"cacheWriteTokens":0,"totalTokens":3554}} +{"type":"status","phase":"turn_end","turn":1,"reason":{"kind":"completed"}} +{"type":"final","text":"MARKER-FIXTURE"} diff --git a/plugins/pstack/skills/poteto-mode/scripts/runner/providers/fixtures/dsh/real-unknown-model.jsonl b/plugins/pstack/skills/poteto-mode/scripts/runner/providers/fixtures/dsh/real-unknown-model.jsonl new file mode 100644 index 00000000..153c840d --- /dev/null +++ b/plugins/pstack/skills/poteto-mode/scripts/runner/providers/fixtures/dsh/real-unknown-model.jsonl @@ -0,0 +1,6 @@ +{"type":"session","sessionId":"session-MARKER-FIXTURE","cwd":"<CAPTURE>/cwd"} +{"type":"status","phase":"turn_start","turn":1} +{"type":"status","phase":"step_start","turn":1,"step":1} +{"type":"status","phase":"step_end","turn":1,"step":1} +{"type":"status","phase":"turn_end","turn":1,"reason":{"kind":"error","error":{"message":"The supported API model names are deepseek-flash, deepseek-v4-pro, but you passed deepseek-nonexistent. (request_id: MARKER-FIXTURE)","code":"INVALID_REQUEST","status":400}}} +{"type":"final","text":""} diff --git a/plugins/pstack/skills/poteto-mode/scripts/runner/providers/fixtures/dsh/real-unknown-model.stderr.txt b/plugins/pstack/skills/poteto-mode/scripts/runner/providers/fixtures/dsh/real-unknown-model.stderr.txt new file mode 100644 index 00000000..e2c07c03 --- /dev/null +++ b/plugins/pstack/skills/poteto-mode/scripts/runner/providers/fixtures/dsh/real-unknown-model.stderr.txt @@ -0,0 +1 @@ +dsh: INVALID_REQUEST: The supported API model names are deepseek-flash, deepseek-v4-pro, but you passed deepseek-nonexistent. (request_id: MARKER-FIXTURE) diff --git a/plugins/pstack/skills/poteto-mode/scripts/runner/providers/fixtures/dsh/status-credential-present.jsonl b/plugins/pstack/skills/poteto-mode/scripts/runner/providers/fixtures/dsh/status-credential-present.jsonl new file mode 100644 index 00000000..3f1227e3 --- /dev/null +++ b/plugins/pstack/skills/poteto-mode/scripts/runner/providers/fixtures/dsh/status-credential-present.jsonl @@ -0,0 +1,6 @@ +{"type":"session","sessionId":"session-3f545bb5-3d41-401b-85cf-6638a46277c0","cwd":"<CAPTURE>/cwd"} +{"type":"status","phase":"turn_start","turn":1} +{"type":"status","phase":"step_start","turn":1,"step":1} +{"type":"status","phase":"step_end","turn":1,"step":1} +{"type":"status","phase":"turn_end","turn":1,"reason":{"kind":"error","error":{"message":"DeepSeek Messages transport failed","code":"TRANSPORT"}}} +{"type":"final","text":""} diff --git a/plugins/pstack/skills/poteto-mode/scripts/runner/providers/fixtures/dsh/status-logged-out.jsonl b/plugins/pstack/skills/poteto-mode/scripts/runner/providers/fixtures/dsh/status-logged-out.jsonl new file mode 100644 index 00000000..a948868b --- /dev/null +++ b/plugins/pstack/skills/poteto-mode/scripts/runner/providers/fixtures/dsh/status-logged-out.jsonl @@ -0,0 +1,6 @@ +{"type":"session","sessionId":"session-2665bca9-726d-4a5a-90ac-fe546938289e","cwd":"<CAPTURE>/cwd"} +{"type":"status","phase":"turn_start","turn":1} +{"type":"status","phase":"step_start","turn":1,"step":1} +{"type":"status","phase":"step_end","turn":1,"step":1} +{"type":"status","phase":"turn_end","turn":1,"reason":{"kind":"error","error":{"message":"llm-deepseek: no API key for provider route \"deepseek-official\"; store DEEPSEEK_API_KEY through the credentials service (the web Models page writes it), or export DEEPSEEK_API_KEY in the launching environment","code":"MISSING_CREDENTIAL"}}} +{"type":"final","text":""} diff --git a/plugins/pstack/skills/poteto-mode/scripts/runner/providers/fixtures/dsh/unsupported-effort.jsonl b/plugins/pstack/skills/poteto-mode/scripts/runner/providers/fixtures/dsh/unsupported-effort.jsonl new file mode 100644 index 00000000..13a96597 --- /dev/null +++ b/plugins/pstack/skills/poteto-mode/scripts/runner/providers/fixtures/dsh/unsupported-effort.jsonl @@ -0,0 +1,6 @@ +{"type":"session","sessionId":"session-04a4ca9d-0924-44dd-812d-cfddade0c80a","cwd":"<CAPTURE>/cwd"} +{"type":"status","phase":"turn_start","turn":1} +{"type":"status","phase":"step_start","turn":1,"step":1} +{"type":"status","phase":"step_end","turn":1,"step":1} +{"type":"status","phase":"turn_end","turn":1,"reason":{"kind":"error","error":{"message":"provider \"deepseek-official\" model \"deepseek-flash\" does not support reasoning effort \"medium\"","code":"UNSUPPORTED_REASONING_EFFORT"}}} +{"type":"final","text":""} diff --git a/plugins/pstack/skills/poteto-mode/scripts/runner/providers/fixtures/dsh/unsupported-effort.stderr.txt b/plugins/pstack/skills/poteto-mode/scripts/runner/providers/fixtures/dsh/unsupported-effort.stderr.txt new file mode 100644 index 00000000..97f986bc --- /dev/null +++ b/plugins/pstack/skills/poteto-mode/scripts/runner/providers/fixtures/dsh/unsupported-effort.stderr.txt @@ -0,0 +1 @@ +dsh: UNSUPPORTED_REASONING_EFFORT: provider "deepseek-official" model "deepseek-flash" does not support reasoning effort "medium" diff --git a/plugins/pstack/skills/poteto-mode/scripts/runner/providers/fixtures/opencode/cancel-sigterm.stdout.jsonl b/plugins/pstack/skills/poteto-mode/scripts/runner/providers/fixtures/opencode/cancel-sigterm.stdout.jsonl new file mode 100644 index 00000000..01545fa6 --- /dev/null +++ b/plugins/pstack/skills/poteto-mode/scripts/runner/providers/fixtures/opencode/cancel-sigterm.stdout.jsonl @@ -0,0 +1 @@ +{"type":"step_start","timestamp":1792120921112,"sessionID":"ses_ebd44eb9fffeJtwYYpMWCmalvr","part":{"id":"prt_142bb1c12001UDkXwvC5SDIbYw","messageID":"msg_142bb15a0001WO0i7ZofRL8MkV","sessionID":"ses_ebd44eb9fffeJtwYYpMWCmalvr","type":"step-start"}} diff --git a/plugins/pstack/skills/poteto-mode/scripts/runner/providers/fixtures/opencode/models-opencode-gate-env.txt b/plugins/pstack/skills/poteto-mode/scripts/runner/providers/fixtures/opencode/models-opencode-gate-env.txt new file mode 100644 index 00000000..09441697 --- /dev/null +++ b/plugins/pstack/skills/poteto-mode/scripts/runner/providers/fixtures/opencode/models-opencode-gate-env.txt @@ -0,0 +1,11 @@ +opencode/big-pickle +opencode/cat-jam-free +opencode/deepseek-v4.1-flash-free +opencode/longcat-2.5-preview-free +opencode/mimo-v2.6-flash-free +opencode/muse-spark-1.3-contributor-free +opencode/nemotron-3-ultra-free +opencode/nemotron-3.5-lightning-free +opencode/nemotron-3.5-super-vl-free +opencode/space-bunny-free +opencode/step-5-preview-free diff --git a/plugins/pstack/skills/poteto-mode/scripts/runner/providers/fixtures/opencode/models-openrouter-no-key.stderr.txt b/plugins/pstack/skills/poteto-mode/scripts/runner/providers/fixtures/opencode/models-openrouter-no-key.stderr.txt new file mode 100644 index 00000000..80bb135d --- /dev/null +++ b/plugins/pstack/skills/poteto-mode/scripts/runner/providers/fixtures/opencode/models-openrouter-no-key.stderr.txt @@ -0,0 +1 @@ +Error: Provider not found: openrouter diff --git a/plugins/pstack/skills/poteto-mode/scripts/runner/providers/fixtures/opencode/provider-error-midrun.stdout.jsonl b/plugins/pstack/skills/poteto-mode/scripts/runner/providers/fixtures/opencode/provider-error-midrun.stdout.jsonl new file mode 100644 index 00000000..2e21373c --- /dev/null +++ b/plugins/pstack/skills/poteto-mode/scripts/runner/providers/fixtures/opencode/provider-error-midrun.stdout.jsonl @@ -0,0 +1 @@ +{"type":"error","timestamp":1792120479456,"sessionID":"ses_ebd4baeb6ffe0aISAK65siMOAS","error":{"name":"UnknownError","data":{"message":"CLOUDFLARE_ACCOUNT_ID and CLOUDFLARE_GATEWAY_ID missing. Set with: export CLOUDFLARE_ACCOUNT_ID=<value> && export CLOUDFLARE_GATEWAY_ID=<value>"}}} diff --git a/plugins/pstack/skills/poteto-mode/scripts/runner/providers/fixtures/opencode/stream-complete-openrouter-cost.jsonl b/plugins/pstack/skills/poteto-mode/scripts/runner/providers/fixtures/opencode/stream-complete-openrouter-cost.jsonl new file mode 100644 index 00000000..7f070fb9 --- /dev/null +++ b/plugins/pstack/skills/poteto-mode/scripts/runner/providers/fixtures/opencode/stream-complete-openrouter-cost.jsonl @@ -0,0 +1,6 @@ +{"type":"step_start","timestamp":1792120470937,"sessionID":"ses_ebd4bd01effepe6HiR7u0Cr9YY","part":{"id":"prt_142b43d95001vWEEdoP88wqqYu","messageID":"msg_142b43169001a7o2PHyGhJSqcA","sessionID":"ses_ebd4bd01effepe6HiR7u0Cr9YY","type":"step-start"}} +{"type":"tool_use","timestamp":1792120471325,"sessionID":"ses_ebd4bd01effepe6HiR7u0Cr9YY","part":{"type":"tool","tool":"read","callID":"call_5e0e5d45988b4b9fab9cfb3f","state":{"status":"completed","input":{"filePath":"<capture-root>/cwd/marker.txt"},"output":"<path><capture-root>/cwd/marker.txt</path>\n<type>file</type>\n<content>\n1: 930c99fe-5257-4a96-9f2e-5fcb8b0ca213\n\n(End of file - total 1 lines)\n</content>","metadata":{"preview":"930c99fe-5257-4a96-9f2e-5fcb8b0ca213","truncated":false,"loaded":[],"display":{"type":"file","path":"<capture-root>/cwd/marker.txt","text":"930c99fe-5257-4a96-9f2e-5fcb8b0ca213","lineStart":1,"lineEnd":1,"totalLines":1,"truncated":false}},"title":"<capture-root-rel>/cwd/marker.txt","time":{"start":1792120471311,"end":1792120471323}},"metadata":{"openrouter":{"reasoning_details":[{"type":"reasoning.text","text":"The user wants me to read marker.txt in the current directory and reply with exactly its contents.","format":"unknown","index":0}]}},"id":"prt_142b43e34001CSfYq42jJ6DNVS","sessionID":"ses_ebd4bd01effepe6HiR7u0Cr9YY","messageID":"msg_142b43169001a7o2PHyGhJSqcA"}} +{"type":"step_finish","timestamp":1792120471362,"sessionID":"ses_ebd4bd01effepe6HiR7u0Cr9YY","part":{"id":"prt_142b43f37001p2vth4uo6qUxL8","reason":"tool-calls","messageID":"msg_142b43169001a7o2PHyGhJSqcA","sessionID":"ses_ebd4bd01effepe6HiR7u0Cr9YY","type":"step-finish","tokens":{"total":23901,"input":3572,"output":71,"reasoning":19,"cache":{"write":0,"read":20239}},"cost":0.0007097818}} +{"type":"step_start","timestamp":1792120473915,"sessionID":"ses_ebd4bd01effepe6HiR7u0Cr9YY","part":{"id":"prt_142b449390015V46X19YGcb6sb","messageID":"msg_142b43f3c001TvK42JoIRg3nWB","sessionID":"ses_ebd4bd01effepe6HiR7u0Cr9YY","type":"step-start"}} +{"type":"text","timestamp":1792120473971,"sessionID":"ses_ebd4bd01effepe6HiR7u0Cr9YY","part":{"id":"prt_142b4493a001O3QYUeIyztzCXY","messageID":"msg_142b43f3c001TvK42JoIRg3nWB","sessionID":"ses_ebd4bd01effepe6HiR7u0Cr9YY","type":"text","text":"930c99fe-5257-4a96-9f2e-5fcb8b0ca213","time":{"start":1792120473914,"end":1792120473966}}} +{"type":"step_finish","timestamp":1792120473971,"sessionID":"ses_ebd4bd01effepe6HiR7u0Cr9YY","part":{"id":"prt_142b4496f001IQxfAMfLg4vzpo","reason":"stop","messageID":"msg_142b43f3c001TvK42JoIRg3nWB","sessionID":"ses_ebd4bd01effepe6HiR7u0Cr9YY","type":"step-finish","tokens":{"total":24021,"input":2640,"output":27,"reasoning":0,"cache":{"write":0,"read":21354}},"cost":0.0006337968}} diff --git a/plugins/pstack/skills/poteto-mode/scripts/runner/providers/fixtures/opencode/stream-complete-zen-free.jsonl b/plugins/pstack/skills/poteto-mode/scripts/runner/providers/fixtures/opencode/stream-complete-zen-free.jsonl new file mode 100644 index 00000000..9d9a26da --- /dev/null +++ b/plugins/pstack/skills/poteto-mode/scripts/runner/providers/fixtures/opencode/stream-complete-zen-free.jsonl @@ -0,0 +1,6 @@ +{"type":"step_start","timestamp":1792120868325,"sessionID":"ses_ebd45be83ffepdmN8mnSK2STa0","part":{"id":"prt_142ba4de3001mg11d88NTztGHD","messageID":"msg_142ba42c50016f4HkGW93GHQjD","sessionID":"ses_ebd45be83ffepdmN8mnSK2STa0","type":"step-start"}} +{"type":"tool_use","timestamp":1792120868586,"sessionID":"ses_ebd45be83ffepdmN8mnSK2STa0","part":{"type":"tool","tool":"read","callID":"call_00_wfen7my4fnxz1falp46l1yjc","state":{"status":"completed","input":{"filePath":"<capture-root>/cwd/marker.txt"},"output":"<path><capture-root>/cwd/marker.txt</path>\n<type>file</type>\n<content>\n1: c1889c77-4282-43bc-9962-8e2348de8475\n\n(End of file - total 1 lines)\n</content>","metadata":{"preview":"c1889c77-4282-43bc-9962-8e2348de8475","truncated":false,"loaded":[],"display":{"type":"file","path":"<capture-root>/cwd/marker.txt","text":"c1889c77-4282-43bc-9962-8e2348de8475","lineStart":1,"lineEnd":1,"totalLines":1,"truncated":false}},"title":"<capture-root-rel>/cwd/marker.txt","time":{"start":1792120868564,"end":1792120868575}},"id":"prt_142ba4eb5001D6MZLl1drps1va","sessionID":"ses_ebd45be83ffepdmN8mnSK2STa0","messageID":"msg_142ba42c50016f4HkGW93GHQjD"}} +{"type":"step_finish","timestamp":1792120868586,"sessionID":"ses_ebd45be83ffepdmN8mnSK2STa0","part":{"id":"prt_142ba4ee20015d9fxdMpoCkDBh","reason":"tool-calls","messageID":"msg_142ba42c50016f4HkGW93GHQjD","sessionID":"ses_ebd45be83ffepdmN8mnSK2STa0","type":"step-finish","tokens":{"total":3919,"input":2063,"output":64,"reasoning":0,"cache":{"write":0,"read":1792}},"cost":0}} +{"type":"step_start","timestamp":1792120869823,"sessionID":"ses_ebd45be83ffepdmN8mnSK2STa0","part":{"id":"prt_142ba53b9001Y0b4J0kJWCH2nI","messageID":"msg_142ba4ee6001DJwq00Ko87yk0T","sessionID":"ses_ebd45be83ffepdmN8mnSK2STa0","type":"step-start"}} +{"type":"text","timestamp":1792120869843,"sessionID":"ses_ebd45be83ffepdmN8mnSK2STa0","part":{"id":"prt_142ba53bd001QCQU2MHfBuiN9o","messageID":"msg_142ba4ee6001DJwq00Ko87yk0T","sessionID":"ses_ebd45be83ffepdmN8mnSK2STa0","type":"text","text":"c1889c77-4282-43bc-9962-8e2348de8475","time":{"start":1792120869821,"end":1792120869833}}} +{"type":"step_finish","timestamp":1792120869843,"sessionID":"ses_ebd45be83ffepdmN8mnSK2STa0","part":{"id":"prt_142ba53cb001ydorbS6G8XuOHd","reason":"stop","messageID":"msg_142ba4ee6001DJwq00Ko87yk0T","sessionID":"ses_ebd45be83ffepdmN8mnSK2STa0","type":"step-finish","tokens":{"total":4032,"input":169,"output":23,"reasoning":0,"cache":{"write":0,"read":3840}},"cost":0}} diff --git a/plugins/pstack/skills/poteto-mode/scripts/runner/providers/fixtures/opencode/tool-ask-auto-rejected.stderr.txt b/plugins/pstack/skills/poteto-mode/scripts/runner/providers/fixtures/opencode/tool-ask-auto-rejected.stderr.txt new file mode 100644 index 00000000..4de1e781 --- /dev/null +++ b/plugins/pstack/skills/poteto-mode/scripts/runner/providers/fixtures/opencode/tool-ask-auto-rejected.stderr.txt @@ -0,0 +1 @@ +! permission requested: edit (<capture-root-rel>/cwd/created.txt); auto-rejecting diff --git a/plugins/pstack/skills/poteto-mode/scripts/runner/providers/fixtures/opencode/tool-ask-auto-rejected.stdout.jsonl b/plugins/pstack/skills/poteto-mode/scripts/runner/providers/fixtures/opencode/tool-ask-auto-rejected.stdout.jsonl new file mode 100644 index 00000000..f42bb7fc --- /dev/null +++ b/plugins/pstack/skills/poteto-mode/scripts/runner/providers/fixtures/opencode/tool-ask-auto-rejected.stdout.jsonl @@ -0,0 +1,3 @@ +{"type":"step_start","timestamp":1792120720077,"sessionID":"ses_ebd47fa9bffeo3kRPNs4MIWejp","part":{"id":"prt_142b80acb001keeDECHLOVhueo","messageID":"msg_142b806a2001q6EHCS5geKNMzz","sessionID":"ses_ebd47fa9bffeo3kRPNs4MIWejp","type":"step-start"}} +{"type":"tool_use","timestamp":1792120721095,"sessionID":"ses_ebd47fa9bffeo3kRPNs4MIWejp","part":{"type":"tool","tool":"write","callID":"call_00_rWzmdPqwMnPPaek30yHL8708","state":{"status":"error","input":{"filePath":"<capture-root>/cwd/created.txt","content":"hi"},"error":"The user rejected permission to use this specific tool call.","time":{"start":1792120721083,"end":1792120721089}},"id":"prt_142b80e1b001eK8QjxU4xL0w3M","sessionID":"ses_ebd47fa9bffeo3kRPNs4MIWejp","messageID":"msg_142b806a2001q6EHCS5geKNMzz"}} +{"type":"step_finish","timestamp":1792120721118,"sessionID":"ses_ebd47fa9bffeo3kRPNs4MIWejp","part":{"id":"prt_142b80ed1001Uvot1QqiuPzwfk","reason":"tool-calls","messageID":"msg_142b806a2001q6EHCS5geKNMzz","sessionID":"ses_ebd47fa9bffeo3kRPNs4MIWejp","type":"step-finish","tokens":{"total":7998,"input":7908,"output":77,"reasoning":13,"cache":{"write":0,"read":0}},"cost":0}} diff --git a/plugins/pstack/skills/poteto-mode/scripts/runner/providers/fixtures/opencode/unknown-model.stderr-error-line.txt b/plugins/pstack/skills/poteto-mode/scripts/runner/providers/fixtures/opencode/unknown-model.stderr-error-line.txt new file mode 100644 index 00000000..e599dfa0 --- /dev/null +++ b/plugins/pstack/skills/poteto-mode/scripts/runner/providers/fixtures/opencode/unknown-model.stderr-error-line.txt @@ -0,0 +1 @@ +timestamp=2026-10-16T03:12:25.358Z level=ERROR run=a3eb9568 message=failed ref=err_3131d374 error="ProviderModelNotFoundError: Model not found: opencode/deepseek-nonexistent. Did you mean: deepseek-v4.1-flash-free? diff --git a/plugins/pstack/skills/poteto-mode/scripts/runner/providers/fixtures/opencode/unknown-model.stdout.jsonl b/plugins/pstack/skills/poteto-mode/scripts/runner/providers/fixtures/opencode/unknown-model.stdout.jsonl new file mode 100644 index 00000000..2d4d23d6 --- /dev/null +++ b/plugins/pstack/skills/poteto-mode/scripts/runner/providers/fixtures/opencode/unknown-model.stdout.jsonl @@ -0,0 +1 @@ +{"type":"error","timestamp":1792120345359,"sessionID":"ses_ebd4dae00ffeqp6Pm0SEzuY2c3","error":{"name":"UnknownError","data":{"message":"Unexpected server error. Check server logs for details.","ref":"err_3131d374"}}} diff --git a/plugins/pstack/skills/poteto-mode/scripts/runner/providers/grok.test.ts b/plugins/pstack/skills/poteto-mode/scripts/runner/providers/grok.test.ts index 191a2563..2095c089 100644 --- a/plugins/pstack/skills/poteto-mode/scripts/runner/providers/grok.test.ts +++ b/plugins/pstack/skills/poteto-mode/scripts/runner/providers/grok.test.ts @@ -7,8 +7,8 @@ adapterSuite(grok); const lane = (request: Partial<LaneRequest> = {}) => testLane({ provider: "grok", model: "grok-4.6", ...request }); -const argv = (request: Partial<LaneRequest> = {}) => grok.launch.argv(lane(request), { sharedGitDir: null }); -const parse = (stdout: string) => grok.parse({ exitCode: 0, stdout, stderr: "" }, lane()); +const argv = (request: Partial<LaneRequest> = {}) => grok.launch.argv(lane(request), { sharedGitDir: null, providerSidecar: null }); +const parse = (stdout: string) => grok.parse({ exitCode: 0, stdout, stderr: "", sidecar: null }, lane()); describe("grok launch", () => { it("limits Grok to the assigned cwd and disables recursive agents", () => { diff --git a/plugins/pstack/skills/poteto-mode/scripts/runner/providers/grok.ts b/plugins/pstack/skills/poteto-mode/scripts/runner/providers/grok.ts index dc3922dc..256cb292 100644 --- a/plugins/pstack/skills/poteto-mode/scripts/runner/providers/grok.ts +++ b/plugins/pstack/skills/poteto-mode/scripts/runner/providers/grok.ts @@ -1,6 +1,8 @@ import { adapter, type Lane, type ParseResult } from "../registry/contract.ts"; import type { AccessMode } from "../types.ts"; import { + BASE_ALLOWLIST, + BASE_PREFIXES, boundedOutput, effortArgs, exactOrSuffixed, @@ -14,33 +16,8 @@ import { type JsonObject, } from "./shared.ts"; -// A Grok lane's shell can read every variable it inherits in both access modes, so it gets -// only what grok needs to start, authenticate and reach the network. -const ENVIRONMENT = [ - "PATH", - "HOME", - "USER", - "LOGNAME", - "SHELL", - "TMPDIR", - "LANG", - "TERM", - "COLORTERM", - "TZ", - "HTTP_PROXY", - "HTTPS_PROXY", - "NO_PROXY", - "ALL_PROXY", - "http_proxy", - "https_proxy", - "no_proxy", - "all_proxy", - "SSL_CERT_FILE", - "SSL_CERT_DIR", - "NODE_EXTRA_CA_CERTS", -]; - -const ENVIRONMENT_PREFIXES = ["LC_", "XDG_", "GROK_", "XAI_"]; +// Grok's shell can read what it inherits in both access modes, so it adds only its own prefixes. +const ENVIRONMENT_PREFIXES = [...BASE_PREFIXES, "GROK_", "XAI_"]; const LOGIN = /\blogged in\b|\bYou are using XAI_API_KEY\.|\bYou are authenticated via deployment key\./i; @@ -109,6 +86,7 @@ function parse(stdout: string, lane: Lane): ParseResult { export const grok = adapter({ id: "grok", executable: "grok", + effortIn: "flag", preflight: { argv: () => ["models"], retryAfterUnauthenticatedMs: 5_000, @@ -126,6 +104,8 @@ export const grok = adapter({ }, launch: { prompt: "file", + sidecar: null, + supports: ["read-only", "isolated-write"], needsSharedGitDir: () => false, argv: (lane) => [ "--prompt-file", @@ -155,7 +135,7 @@ export const grok = adapter({ }, parse: (run, lane) => parse(run.stdout, lane), proof: { report: "required", matches: exactOrSuffixed }, - environment: () => ({ kind: "allowlist", names: ENVIRONMENT, prefixes: ENVIRONMENT_PREFIXES }), + environment: () => ({ kind: "allowlist", names: BASE_ALLOWLIST, prefixes: ENVIRONMENT_PREFIXES }), docs: { name: "Grok Build CLI", hostPrerequisites: ["Linux: Landlock and `bwrap`"], diff --git a/plugins/pstack/skills/poteto-mode/scripts/runner/providers/opencode.test.ts b/plugins/pstack/skills/poteto-mode/scripts/runner/providers/opencode.test.ts new file mode 100644 index 00000000..7765945c --- /dev/null +++ b/plugins/pstack/skills/poteto-mode/scripts/runner/providers/opencode.test.ts @@ -0,0 +1,169 @@ +import { describe, expect, it } from "bun:test"; +import { readFileSync } from "node:fs"; +import { join } from "node:path"; +import type { ModelRun } from "../registry/contract.ts"; +import type { LaneRequest } from "../registry/index.ts"; +import { adapterSuite, testFacts, testLane } from "./adapter-suite.ts"; +import { opencode } from "./opencode.ts"; + +adapterSuite(opencode); + +// Sanitized from OpenCode 1.19.1 captures: structure kept, paths replaced by placeholders. +const fixture = (name: string): string => readFileSync(join(import.meta.dir, "fixtures/opencode", name), "utf8"); + +const MODEL = "opencode/deepseek-v4.1-flash-free"; +const lane = (request: Partial<LaneRequest> = {}) => testLane({ provider: "opencode", model: MODEL, effort: "high", ...request }); +const run = (overrides: Partial<ModelRun>): ModelRun => ({ exitCode: 0, stdout: "", stderr: "", sidecar: null, ...overrides }); +const preflight = (exitCode: number, stdout: string, stderr = "", request: Partial<LaneRequest> = {}) => + opencode.preflight.classify({ exitCode, stdout, stderr }, lane(request)); +const event = (type: string, part: Record<string, unknown> = {}) => JSON.stringify({ type, timestamp: 1, sessionID: "ses_x", part }); + +describe("opencode launch", () => { + it("runs the pinned model and variant with a fixed title and the ERROR log on stderr", () => { + expect(opencode.launch.argv(lane(), testFacts(opencode))).toEqual([ + "run", "--pure", "--agent", "build", "--format", "json", "--title", "pstack-lane", "--print-logs", "--log-level", "ERROR", + "-m", MODEL, "--variant", "high", + ]); + expect(opencode.launch.argv(lane({ effort: "default", model: "openrouter/deepseek/deepseek-v4-pro" }), testFacts(opencode))) + .toEqual(["run", "--pure", "--agent", "build", "--format", "json", "--title", "pstack-lane", "--print-logs", "--log-level", "ERROR", "-m", "openrouter/deepseek/deepseek-v4-pro"]); + }); + + it("refuses an isolated-write lane and efforts the family does not list", () => { + expect(() => lane({ mode: "isolated-write" })) + .toThrow("provider opencode does not support isolated-write; it supports only: read-only"); + expect(() => lane({ effort: "medium" })) + .toThrow(`effort medium is not offered for opencode:${MODEL}; choose one of: low, high, max`); + }); + + it("writes a lockdown config that denies every tool except the read tools", () => { + const config = opencode.launch.sidecar!.config!; + expect([config.name, opencode.launch.sidecar!.output]).toEqual(["opencode.json", null]); + expect(JSON.parse(config.content(lane()))).toEqual({ + share: "disabled", + autoupdate: false, + snapshot: false, + instructions: [], + permission: { "*": "deny", read: "allow", glob: "allow", grep: "allow", list: "allow", todowrite: "allow" }, + }); + }); + + it("points OpenCode at the run-owned config and home, and passes no OPENCODE_ prefix or provider key", () => { + expect(opencode.environment(lane().access, "/tmp/r.provider")).toMatchObject({ + kind: "allowlist", + prefixes: ["LC_", "XDG_"], + set: { + XDG_CONFIG_HOME: "/tmp/r.provider", + OPENCODE_TEST_HOME: "/tmp/r.provider", + OPENCODE_CONFIG: "/tmp/r.provider/opencode.json", + OPENCODE_DISABLE_PROJECT_CONFIG: "1", + OPENCODE_DISABLE_CLAUDE_CODE: "1", + OPENCODE_DISABLE_EXTERNAL_SKILLS: "1", + OPENCODE_DISABLE_AUTOUPDATE: "1", + }, + }); + const policy = opencode.environment(lane().access, "/tmp/r.provider"); + if (policy.kind !== "allowlist") throw new Error("expected an allowlist"); + expect(policy.names.filter((name) => /KEY|TOKEN|OPENCODE/.test(name))).toEqual([]); + }); +}); + +describe("opencode preflight", () => { + it("asks the model's provider and passes only when the model is listed", () => { + expect(opencode.preflight.argv(lane(), null)).toEqual(["models", "opencode"]); + expect(opencode.preflight.argv(lane({ model: "openrouter/deepseek/deepseek-v4-pro" }), null)).toEqual(["models", "openrouter"]); + expect(preflight(0, fixture("models-opencode-gate-env.txt"))).toEqual({ kind: "passed", evidence: `model ${MODEL} listed` }); + expect(preflight(0, fixture("models-opencode-gate-env.txt"), "", { model: "opencode/deepseek-v4.1-flash" })) + .toEqual({ kind: "unavailable-model" }); + }); + + it("reads Provider not found as unauthenticated and a model with no provider as unavailable", () => { + expect(preflight(1, "", fixture("models-openrouter-no-key.stderr.txt"), { model: "openrouter/deepseek/deepseek-v4-pro" })) + .toEqual({ kind: "unauthenticated" }); + expect(preflight(0, fixture("models-opencode-gate-env.txt"), "", { model: "deepseek-v4.1-flash-free" })) + .toEqual({ kind: "unavailable-model" }); + }); + + it("reads any other failure as inconclusive", () => { + for (const [exitCode, stderr] of [[1, ""], [1, "Error: network unreachable"], [2, "Provider not found: opencode"]] as const) { + expect([exitCode, stderr, preflight(exitCode, MODEL, stderr)]).toEqual([exitCode, stderr, { kind: "inconclusive" }]); + } + }); +}); + +describe("opencode parse", () => { + it("takes the reply after the last step_start and sums usage and cost over every step", () => { + expect(opencode.parse(run({ stdout: fixture("stream-complete-zen-free.jsonl") }), lane())).toEqual({ + kind: "complete", + text: "c1889c77-4282-43bc-9962-8e2348de8475", + metadata: { + reportedModel: null, + sessionId: "ses_ebd45be83ffepdmN8mnSK2STa0", + usage: { inputTokens: 2232, cachedInputTokens: 5632, cacheCreationInputTokens: 0, outputTokens: 87, reasoningTokens: 0, totalTokens: 7951 }, + costUsd: 0, + }, + }); + const paid = opencode.parse(run({ stdout: fixture("stream-complete-openrouter-cost.jsonl") }), lane()); + expect(paid).toMatchObject({ + kind: "complete", + text: "930c99fe-5257-4a96-9f2e-5fcb8b0ca213", + metadata: { usage: { inputTokens: 6212, cachedInputTokens: 41593, outputTokens: 98, reasoningTokens: 19, totalTokens: 47922 } }, + }); + expect(paid.kind === "complete" ? paid.metadata.costUsd : null).toBeCloseTo(0.0013435786, 12); + expect(opencode.proof).toMatchObject({ report: "absent" }); + expect(opencode.proof.matches(lane(), MODEL)).toBe(false); + }); + + it("joins the last step's texts in order and ignores text from earlier steps", () => { + const stdout = [ + event("step_start"), event("text", { text: "earlier" }), event("step_finish", { reason: "tool-calls" }), + event("step_start"), event("text", { text: "one " }), event("text", { text: "two" }), event("step_finish", { reason: "stop" }), + ].join("\n"); + expect(opencode.parse(run({ stdout }), lane())).toMatchObject({ kind: "complete", text: "one two" }); + }); + + it("reads an auto-rejected tool call at exit 0 as a failure", () => { + expect(opencode.parse(run({ stdout: fixture("tool-ask-auto-rejected.stdout.jsonl"), stderr: fixture("tool-ask-auto-rejected.stderr.txt") }), lane())) + .toEqual({ + kind: "provider-failure", + status: "child-failed", + message: "opencode ended on a rejected tool call", + metadata: { + reportedModel: null, + sessionId: "ses_ebd47fa9bffeo3kRPNs4MIWejp", + usage: { inputTokens: 7908, cachedInputTokens: 0, cacheCreationInputTokens: 0, outputTokens: 77, reasoningTokens: 13, totalTokens: 7998 }, + costUsd: 0, + }, + }); + }); + + it("reads an error event as a failure with its name and message", () => { + expect(opencode.parse(run({ exitCode: 1, stdout: fixture("unknown-model.stdout.jsonl"), stderr: fixture("unknown-model.stderr-error-line.txt") }), lane())) + .toMatchObject({ kind: "provider-failure", status: "child-failed", message: "opencode reported UnknownError: Unexpected server error. Check server logs for details." }); + expect(opencode.parse(run({ exitCode: 1, stdout: fixture("provider-error-midrun.stdout.jsonl") }), lane())) + .toMatchObject({ kind: "provider-failure", status: "child-failed", message: expect.stringContaining("UnknownError: CLOUDFLARE_ACCOUNT_ID and CLOUDFLARE_GATEWAY_ID missing") }); + }); + + it("reads exit 130 and 143 as cancelled", () => { + for (const exitCode of [130, 143]) { + expect(opencode.parse(run({ exitCode, stdout: fixture("cancel-sigterm.stdout.jsonl") }), lane())).toMatchObject({ + kind: "provider-failure", + status: "cancelled", + message: `opencode exited with status ${exitCode} after a signal`, + metadata: { sessionId: "ses_ebd44eb9fffeJtwYYpMWCmalvr", usage: null }, + }); + } + }); + + it("never completes a stream that ends inside a step, on another reason or with no or empty text", () => { + expect(opencode.parse(run({ stdout: fixture("cancel-sigterm.stdout.jsonl") }), lane())) + .toEqual({ kind: "malformed", message: "opencode stream did not end with step_finish" }); + expect(opencode.parse(run({ stdout: "" }), lane())).toEqual({ kind: "malformed", message: "opencode stream did not end with step_finish" }); + expect(opencode.parse(run({ stdout: "not json\n" }), lane())).toEqual({ kind: "malformed", message: "opencode emitted a non-JSON event" }); + expect(opencode.parse(run({ stdout: [event("step_start"), event("step_finish", { reason: "stop" })].join("\n") }), lane())) + .toEqual({ kind: "malformed", message: "opencode's last step has no text" }); + expect(opencode.parse(run({ stdout: [event("step_start"), event("text", { text: "" }), event("step_finish", { reason: "stop" })].join("\n") }), lane())) + .toEqual({ kind: "malformed", message: "opencode's last step has no text" }); + expect(opencode.parse(run({ stdout: [event("step_start"), event("text", { text: "partial" }), event("step_finish", { reason: "length" })].join("\n") }), lane())) + .toMatchObject({ kind: "provider-failure", status: "child-failed", message: "opencode ended its last step with reason \"length\"" }); + }); +}); diff --git a/plugins/pstack/skills/poteto-mode/scripts/runner/providers/opencode.ts b/plugins/pstack/skills/poteto-mode/scripts/runner/providers/opencode.ts new file mode 100644 index 00000000..4b5888d4 --- /dev/null +++ b/plugins/pstack/skills/poteto-mode/scripts/runner/providers/opencode.ts @@ -0,0 +1,178 @@ +import { adapter, type ModelRun, type ParseResult, type ProviderMetadata } from "../registry/contract.ts"; +import { BASE_ALLOWLIST, BASE_PREFIXES, effortArgs, finiteNumber, nullableString, object, type JsonObject } from "./shared.ts"; + +// The run-owned global config. `"*": "deny"` removes every tool the allow-list does not name from the model's tool +// list, so bash, edit, write, task, webfetch, websearch, skill and question are gone. Measured with OpenCode 1.19.1. +const READS = ["read", "glob", "grep", "list", "todowrite"]; +const LOCKDOWN = { + share: "disabled", + autoupdate: false, + snapshot: false, + instructions: [], + permission: { "*": "deny", ...Object.fromEntries(READS.map((tool) => [tool, "allow"])) }, +}; +const CONFIG_NAME = "opencode.json"; + +// SIGINT exits 130 and SIGTERM 143, each after only a step_start event. +const SIGNAL_EXITS = [130, 143]; +const ANSI = /\u001b\[[0-9;]*m/g; + +function events(stdout: string): JsonObject[] | null { + const parsed: JsonObject[] = []; + for (const line of stdout.split("\n")) { + if (line.trim().length === 0) continue; + let event: JsonObject | null; + try { + event = object(JSON.parse(line)); + } catch { + return null; + } + if (event === null) return null; + parsed.push(event); + } + return parsed; +} + +function metadata(stream: readonly JsonObject[]): ProviderMetadata { + const finishes = stream.filter((event) => event.type === "step_finish").map((event) => object(event.part)); + const sum = (read: (part: JsonObject | null) => unknown) => + finishes.reduce((total, part) => total + (finiteNumber(read(part)) ?? 0), 0); + const tokens = (part: JsonObject | null) => object(part?.tokens); + const costs = finishes.map((part) => finiteNumber(part?.cost)).filter((cost) => cost !== undefined); + return { + reportedModel: null, + sessionId: stream.map((event) => nullableString(event.sessionID)).find((id) => id !== null) ?? null, + usage: finishes.length === 0 ? null : { + inputTokens: sum((part) => tokens(part)?.input), + cachedInputTokens: sum((part) => object(tokens(part)?.cache)?.read), + cacheCreationInputTokens: sum((part) => object(tokens(part)?.cache)?.write), + outputTokens: sum((part) => tokens(part)?.output), + reasoningTokens: sum((part) => tokens(part)?.reasoning), + totalTokens: sum((part) => tokens(part)?.total), + }, + costUsd: costs.length === 0 ? null : costs.reduce((total, cost) => total + cost, 0), + }; +} + +function parse(run: ModelRun): ParseResult { + const stream = events(run.stdout); + const found = metadata(stream ?? []); + if (run.exitCode !== null && SIGNAL_EXITS.includes(run.exitCode)) { + return { kind: "provider-failure", status: "cancelled", message: `opencode exited with status ${run.exitCode} after a signal`, metadata: found }; + } + if (stream === null) return { kind: "malformed", message: "opencode emitted a non-JSON event" }; + const error = stream.find((event) => event.type === "error"); + if (error !== undefined) { + // An unknown model or provider reports only UnknownError here; the ERROR log line on stderr names the cause. + const detail = object(error.error); + const named = [nullableString(detail?.name), nullableString(object(detail?.data)?.message)].filter((part) => part !== null); + return { kind: "provider-failure", status: "child-failed", message: `opencode reported ${named.join(": ") || "an error"}`, metadata: found }; + } + const lastStart = stream.findLastIndex((event) => event.type === "step_start"); + const lastFinish = stream.findLastIndex((event) => event.type === "step_finish"); + if (lastFinish < 0 || lastFinish < lastStart) return { kind: "malformed", message: "opencode stream did not end with step_finish" }; + const reason = object(stream[lastFinish]!.part)?.reason; + if (reason === "tool-calls") { + // Headless OpenCode auto-rejects a permission prompt and ends the turn with no reply, at exit 0. + return { kind: "provider-failure", status: "child-failed", message: "opencode ended on a rejected tool call", metadata: found }; + } + if (reason !== "stop") { + return { kind: "provider-failure", status: "child-failed", message: `opencode ended its last step with reason ${JSON.stringify(reason ?? null)}`, metadata: found }; + } + const text = stream.slice(lastStart + 1) + .filter((event) => event.type === "text") + .map((event) => object(event.part)?.text) + .filter((part): part is string => typeof part === "string") + .join(""); + if (text.length === 0) return { kind: "malformed", message: "opencode's last step has no text" }; + return { kind: "complete", text, metadata: found }; +} + +export const opencode = adapter({ + id: "opencode", + executable: "opencode", + effortIn: "flag", + preflight: { + // Lists only providers with a usable credential, plus the free OpenCode Zen models. + argv: (lane) => ["models", lane.model.split("/")[0]!], + retryAfterUnauthenticatedMs: null, + classify(run, lane) { + if (!lane.model.includes("/")) return { kind: "unavailable-model" }; + if (run.exitCode === 0) { + return run.stdout.split("\n").some((line) => line.trim() === lane.model) + ? { kind: "passed", evidence: `model ${lane.model} listed` } + : { kind: "unavailable-model" }; + } + return /^Error: Provider not found:/m.test(run.stderr.replace(ANSI, "")) ? { kind: "unauthenticated" } : { kind: "inconclusive" }; + }, + }, + launch: { + prompt: "stdin", + sidecar: { output: null, config: { name: CONFIG_NAME, content: () => `${JSON.stringify(LOCKDOWN, null, 2)}\n` } }, + // No OS sandbox: with external_directory denied, a shell write still escaped the cwd, so no mode is a bounded writer. + supports: ["read-only"], + needsSharedGitDir: () => false, + argv: (lane) => [ + "run", + "--pure", + // A config's default_agent would otherwise pick another agent. + "--agent", + "build", + "--format", + "json", + // Without a title, OpenCode calls a second model to name the session. + "--title", + "pstack-lane", + // An unknown model's stdout is a generic UnknownError; the ERROR log line names the cause. + "--print-logs", + "--log-level", + "ERROR", + "-m", + lane.cliModel, + ...effortArgs(lane, ["--variant", lane.effort]), + ], + }, + parse: (run) => parse(run), + proof: { report: "absent", matches: () => false }, + // OpenCode 1.19.1 reads the home .opencode/ (an opencode.json merged after OPENCODE_CONFIG, and plugins) under + // OPENCODE_TEST_HOME when set, so the sidecar keeps the host's ~/.opencode/ out; stored logins still come from the + // real home. No other OPENCODE_ prefix: OPENCODE_PERMISSION turned bash and edit back on over the lockdown, and + // others replace the config or the model catalog. No provider key names: a lane uses OpenCode's stored logins or + // a free Zen model. + environment: (_access, sidecarDir) => ({ + kind: "allowlist", + names: BASE_ALLOWLIST, + prefixes: BASE_PREFIXES, + set: { + XDG_CONFIG_HOME: `${sidecarDir}`, + OPENCODE_TEST_HOME: `${sidecarDir}`, + OPENCODE_CONFIG: `${sidecarDir}/${CONFIG_NAME}`, + OPENCODE_DISABLE_PROJECT_CONFIG: "1", + OPENCODE_DISABLE_CLAUDE_CODE: "1", + OPENCODE_DISABLE_EXTERNAL_SKILLS: "1", + OPENCODE_DISABLE_AUTOUPDATE: "1", + }, + }), + docs: { + name: "OpenCode", + hostPrerequisites: [], + statePaths: [ + "`~/.local/share/opencode/` (`auth.json`, the stored logins, which the CLI reads; the session database, logs and tool output it writes)", + "`~/.local/state/opencode/`, `$XDG_CACHE_HOME/opencode/` and `$TMPDIR/opencode/`, which it writes", + ], + notes: [ + "The model is `provider/model`, such as `opencode/deepseek-v4.1-flash-free`. The effort is `--variant <effort>`, omitted for `default`. OpenCode accepts an unknown variant silently, so only a registered family's efforts are checked.", + "Read-only only, measured on OpenCode 1.19.1. Each lane runs the `build` agent (`--agent build`, because a config's `default_agent` would otherwise pick another agent) with a run-owned config whose `\"*\": \"deny\"` permission removes every tool except `read`, `glob`, `grep`, `list` and `todowrite` from the model's tool list. OpenCode's permission policy enforces this, not an OS sandbox. With `external_directory` denied, a shell write still escaped the cwd, so no OpenCode mode is a bounded writer, and the runner refuses an `isolated-write` lane before launch.", + "The runner writes that config to `<receipt>.provider/opencode.json` (0600, in a 0700 directory), points `OPENCODE_CONFIG` at it, and sets `XDG_CONFIG_HOME` and `OPENCODE_TEST_HOME` to the directory, so OpenCode reads the run-owned directory in place of the host's `~/.config/opencode/` and `~/.opencode/`. OpenCode 1.19.1 merges `~/.opencode/opencode.json` after `OPENCODE_CONFIG` and imports `~/.opencode/plugins/*.js` even with `--pure`, and it finds that home `.opencode/` through `OPENCODE_TEST_HOME` when set. Measured with a fresh `HOME` whose `.opencode/opencode.json` allowed `bash`, `edit` and `write` and whose `.opencode/plugins/` held a canary plugin: without `OPENCODE_TEST_HOME`, `opencode debug agent build` listed those tools and the canary ran, and neither `OPENCODE_CONFIG_CONTENT` nor `OPENCODE_CONFIG_DIR` changed that; with it, `debug agent build` and `debug config` showed only the lockdown, the canary did not run, and a lane asked to write and edit files changed nothing. Stored logins still come from the real home. `OPENCODE_DISABLE_PROJECT_CONFIG=1` stops the repository's `opencode.json`, `opencode.jsonc`, `.opencode/` agents and commands, and `AGENTS.md` from loading. `OPENCODE_DISABLE_CLAUDE_CODE=1` stops `~/.claude/CLAUDE.md` and `~/.claude/skills`, and `OPENCODE_DISABLE_EXTERNAL_SKILLS=1` stops `~/.agents/skills`. `OPENCODE_DISABLE_AUTOUPDATE=1` and `--pure` turn off updates and external plugins. The built-in `customize-opencode` skill remains, but the `skill` tool is denied.", + "A managed config (`/Library/Application Support/opencode/` on macOS, `/etc/opencode/` on Linux) merges after the run-owned config and overrides it. Measured on OpenCode 1.19.1 with layers planted through the binary's `OPENCODE_TEST_MANAGED_CONFIG_DIR` redirect: a managed `mcp` server started in a real lane, a managed `lsp` command ran when the model called `read`, a managed `provider` `baseURL` rerouted the lane, and a managed `agent.build` prompt replaced the build agent's prompt. `--agent build` and `-m` took precedence over a managed `default_agent` and `agent.build.model`, and `--pure` kept a managed `plugin` from loading. On macOS, `/Library/Managed Preferences/<user>/ai.opencode.managed.plist` and `/Library/Managed Preferences/ai.opencode.managed.plist` merge last. The plist layer is from OpenCode's source and was not measured.", + "A stored `wellknown` login (from `opencode providers login <url>`) or an active console organization (from `opencode console login`) makes every OpenCode process, including each lane, fetch remote config and merge it into the lane's configuration. Measured with a fake well-known login and a local server, whose config added an MCP server and a `default_agent`. The console organization fetch is from OpenCode's source and was not measured.", + "OpenCode 1.19.1 imports a repository's `.opencode/plugins/*.js` from the cwd or a parent up to the git root, even with `--pure` and `OPENCODE_DISABLE_PROJECT_CONFIG=1`. The import runs each module's top-level code; only the plugin's initialization is skipped.", + "The child environment drops every other `OPENCODE_*` variable, because `OPENCODE_PERMISSION` turns `bash` and `edit` back on and others replace the config or the model catalog. It also drops provider keys such as `OPENROUTER_API_KEY`. A lane authenticates through OpenCode's stored logins (`opencode auth login`, kept in `~/.local/share/opencode/auth.json`), or uses a free OpenCode Zen model, which needs no login.", + "The preflight runs `opencode models <provider>`, where `<provider>` is the model's part before the first `/`. It passes when it exits 0 and one line equals the model. `Error: Provider not found:` with a nonzero exit is `unauthenticated`/77. Exit 0 without the model line is `unavailable-model`/69, and so is a model with no `/`. Any other output is `child-failed`/70.", + "The lane passes `--title pstack-lane`, because without a title OpenCode calls a second model to name the session. The prompt goes on stdin.", + "The reply, session ID, token usage and cost come from the `--format json` event stream on stdout. A lane is complete only when the last `step_finish` has reason `stop` and a `text` event follows the last `step_start`; the reply is those texts in order. Usage and cost are summed over every `step_finish`. An `error` event is `child-failed`/70. For an unknown model that event says only `UnknownError`, and `--print-logs --log-level ERROR` puts its `ProviderModelNotFoundError` cause on stderr, which the receipt keeps. A final `step_finish` with reason `tool-calls` is a permission prompt that headless OpenCode auto-rejected at exit 0, and is `child-failed`/70. Exit 130 or 143 after a signal is `cancelled`/130.", + "No event names the served model, so the model proof is the pinned argv (`pinned-argv`).", + "Sessions stay in the local session database. The config turns sharing off, and OpenCode 1.19.1 keeps no server-side session record.", + ], + }, +}); diff --git a/plugins/pstack/skills/poteto-mode/scripts/runner/providers/shared.ts b/plugins/pstack/skills/poteto-mode/scripts/runner/providers/shared.ts index cd8ea845..4c9fdf07 100644 --- a/plugins/pstack/skills/poteto-mode/scripts/runner/providers/shared.ts +++ b/plugins/pstack/skills/poteto-mode/scripts/runner/providers/shared.ts @@ -1,6 +1,33 @@ import type { ChildRun, Lane, PreflightVerdict } from "../registry/contract.ts"; import { evidence, modelFailureStatus, type NormalizedUsage } from "../types.ts"; +/** What an allowlisted CLI needs to start, authenticate and reach the network. A lane's shell can + * read every variable it inherits, so an allowlist adapter adds only its own prefixes (#141). */ +export const BASE_ALLOWLIST = [ + "PATH", + "HOME", + "USER", + "LOGNAME", + "SHELL", + "TMPDIR", + "LANG", + "TERM", + "COLORTERM", + "TZ", + "HTTP_PROXY", + "HTTPS_PROXY", + "NO_PROXY", + "ALL_PROXY", + "http_proxy", + "https_proxy", + "no_proxy", + "all_proxy", + "SSL_CERT_FILE", + "SSL_CERT_DIR", + "NODE_EXTRA_CA_CERTS", +]; +export const BASE_PREFIXES = ["LC_", "XDG_"]; + export type JsonObject = Record<string, unknown>; export function object(value: unknown): JsonObject | null { diff --git a/plugins/pstack/skills/poteto-mode/scripts/runner/registry.test.ts b/plugins/pstack/skills/poteto-mode/scripts/runner/registry.test.ts index 093a46c9..18f44c91 100644 --- a/plugins/pstack/skills/poteto-mode/scripts/runner/registry.test.ts +++ b/plugins/pstack/skills/poteto-mode/scripts/runner/registry.test.ts @@ -4,6 +4,7 @@ import { tmpdir } from "node:os"; import { dirname, join, relative, resolve } from "node:path"; import { defaultDescriptor, + effortsOf, type LaneRequest, REGISTRY, resolveLane, @@ -83,10 +84,27 @@ describe("registry data", () => { ["sol", "max"], ["grok", "xhigh"], ["opus", "max"], + ["luna", "max"], + ["sonnet", "max"], + ["opus-5-5", "max"], + ["swe-2", "max"], + ["swe-1.6", "default"], + ["deepseek-v4.1-flash-free", "max"], + ["deepseek-flash", "max"], + ["deepseek-v4-pro", "max"], + ["amp-deepseek-v4.1-flash", "default"], + ["gemini-3.8-flash", "high"], + ["composer-2.5", "default"], + ["gemini-3.7-flash", "high"], ]); expect(REGISTRY.families.filter((entry) => entry.provider === "claude").map((entry) => [entry.name, entry.model])) - .toEqual([["fable", "fable"], ["opus", "opus"]]); - expect(REGISTRY.families.filter((entry) => entry.efforts.includes("ultra")).map((entry) => entry.name)).toEqual(["sol"]); + .toEqual([["fable", "fable"], ["opus", "opus"], ["sonnet", "sonnet"], ["opus-5-5", "claude-opus-5-5"]]); + expect(REGISTRY.families.filter((entry) => entry.rolling !== undefined) + .map((entry) => [entry.name, entry.rolling!.revisionPrefix])) + .toEqual([["fable", "claude-fable-"], ["opus", "claude-opus-"], ["sonnet", "claude-sonnet-"]]); + expect(REGISTRY.families.filter((entry) => entry.provider === "codex").map((entry) => [entry.name, entry.model])) + .toEqual([["sol", "gpt-6.1-sol"], ["luna", "gpt-6-luna"]]); + expect(REGISTRY.families.filter((entry) => effortsOf(entry).includes("ultra")).map((entry) => entry.name)).toEqual(["sol"]); }); it("keeps family names and (provider, model) pairs unique, and efforts ordered", () => { @@ -97,11 +115,11 @@ describe("registry data", () => { .map((model) => `${entry.provider}:${model}`)); expect(new Set(pairs).size).toBe(pairs.length); for (const entry of REGISTRY.families) { - for (const efforts of [entry.efforts, ...(entry.predecessors ?? []).map((kept) => kept.efforts)]) { + for (const efforts of [effortsOf(entry), ...(entry.predecessors ?? []).map((kept) => kept.efforts)]) { expect(efforts.length).toBeGreaterThan(0); expect([...efforts]).toEqual(EFFORTS.filter((effort) => efforts.includes(effort))); } - expect(entry.efforts).toContain(entry.defaultEffort); + expect(effortsOf(entry)).toContain(entry.defaultEffort); } }); @@ -206,12 +224,15 @@ describe("hand-written dispatch and setup rules", () => { it("normalizes rolling-alias pins and keeps predecessors before any runtime route", () => { const normalization = section(dispatch, "## Read-time normalization", "## The parent owns the route"); + expect(normalization).toContain("an exact model-matrix row, then a kept predecessor, then a rolling-alias revision"); + expect(normalization).toContain("such as a registered exact pin, is never normalized"); expect(normalization).toContain("replace that model component in memory"); expect(normalization).toContain("Never pass the versioned predecessor to Claude."); expect(normalization).toContain("without writing user files"); expect(normalization).toContain("`/setup-pstack` will rewrite it"); expect(normalization).toContain("runner rejects a missed rolling-alias version pin"); - expect(normalization).toContain("A kept predecessor in the catalog matches its family. Dispatch it unchanged"); + expect(normalization).toContain("A kept predecessor in the catalog matches its family. Dispatch it with the sheet's model"); + expect(normalization).toContain("Do not rewrite its model in memory."); }); it("keeps setup's fail-closed reconfiguration order", () => { @@ -234,7 +255,9 @@ describe("hand-written dispatch and setup rules", () => { "A failed probe writes nothing:", "Run one probe per family", "normalized complete role map from step 2", - "starts with `claude-fable-` or `claude-opus-`", + "Match in this order: an exact model-matrix row, then a kept predecessor, then a rolling-alias revision.", + "such as a registered exact pin, is never migratable", + "starts with a rolling alias's versioned-predecessor prefix", "preserving the provider, effort, role, and lane order", "Show any rolling-alias migrations", "An effort-only rerun cannot change a role's family.", @@ -245,6 +268,28 @@ describe("hand-written dispatch and setup rules", () => { } }); + it("refuses a read-only-only provider for every role whose lanes write files", () => { + const writers = Object.entries(ROLE_POINTERS) + .filter(([file]) => file.includes("/playbooks/")) + .filter(([file]) => readFileSync(join(PLUGIN_ROOT, file), "utf8").includes("`isolated-write`")) + .flatMap(([, roles]) => roles); + expect(new Set(writers)).toEqual(new Set(["feature, refactoring", "bug-fix", "perf-issue", "hillclimb"])); + const rule = section(setup, "### 3. Parse per-family efforts", "### 4. Collect one requested effort per family"); + expect(rule).toContain("Lanes for these roles write files: `feature, refactoring`, `bug-fix`, `perf-issue`, and `hillclimb`, whose playbooks delegate with `isolated-write`; `hardest tasks`, which takes the hardest code changes; and `arena runners`, `architect runners` and `swarm workers`, which write to their own worktrees or output directories."); + // poteto-mode sends the hardest changes to `hardest tasks`; arena and architect candidates and swarm workers each write + // to their own location. + expect(readFileSync(join(PLUGIN_ROOT, "skills/poteto-mode/SKILL.md"), "utf8")).toContain("the hardest changes read `hardest tasks`"); + expect(readFileSync(join(PLUGIN_ROOT, "skills/arena/SKILL.md"), "utf8")).toContain("Give each candidate its own output location (a git worktree"); + expect(readFileSync(join(PLUGIN_ROOT, "skills/swarm/SKILL.md"), "utf8")).toContain("in the worker's own worktree"); + expect(readFileSync(join(PLUGIN_ROOT, "skills/architect/SKILL.md"), "utf8")).toContain("Take the runners from `architect runners` in the current harness's pstack model sheet, in place of Arena's `arena runners`."); + expect(rule).toContain("lacks `isolated-write`: name the role and the provider, and say why."); + expect(rule).toContain("The runner refuses that provider's writer lane before launch"); + expect(REGISTRY.adapters.filter((adapter) => !adapter.launch.supports.includes("isolated-write")).map((adapter) => adapter.id)) + .toEqual(["devin", "opencode", "dsh", "amp", "agy", "cursor"]); + const devinRoute = renderCatalog(REGISTRY).content.split("\n").find((line) => line.startsWith("| devin |")); + expect(devinRoute).toContain("| `read-only` |"); + }); + it("proposes a kept predecessor's current model at the same effort and keeps it on a decline", () => { expect(setup).toContain("model-catalog.md#kept-predecessors-and-normalization) is not inconsistent state."); expect(setup).toContain("propose replacing every occurrence with its family's current model at the same effort, and ask."); @@ -254,6 +299,23 @@ describe("hand-written dispatch and setup rules", () => { .toEqual(expect.arrayContaining(["codex:gpt-5.6-sol", "grok:grok-4.6"])); }); + it("passes full access only to an opted-in external Codex writer, and setup keeps the setting line", () => { + const rule = section(dispatch, "### Codex writer sandbox", "### Host and parent prerequisites"); + expect(rule).toContain("`codex writer sandbox: full-access` or `codex writer sandbox: bounded`"); + expect(rule).toContain("Append `--writer-sandbox full-access` only when that line says `full-access` and the lane is an external Codex `isolated-write` lane."); + expect(rule).toContain("the setting never moves a native lane to the runner"); + expect(rule).toContain("Never add the flag, or retry with it, after a sandbox failure"); + expect(rule).toContain("it confines nothing"); + expect(dispatch).not.toMatch(/Bash sandbox, Codex `workspace-write`, and/); + const load = section(setup, "### 2. Load current state", "### 3. Parse per-family efforts"); + expect(load.indexOf("Set it aside before the role overlay, the retired-role rule, and descriptor validation")) + .toBeLessThan(load.indexOf("is from a retired role. Drop it")); + expect(load).toContain("Record `codex writer sandbox: full-access` on a yes and `codex writer sandbox: bounded` on a decline"); + expect(load).toContain("When the line exists, keep it unless the operator names a change, and do not ask."); + expect(setup).toContain("Keep the setting line from step 2 where the loaded sheet had it."); + expect(setup).not.toContain("Every non-alias value must match"); + }); + }); const REPO_ROOT = join(PLUGIN_ROOT, "../.."); @@ -302,7 +364,7 @@ describe("role pointers and text scans", () => { .map((path) => join(PLUGIN_ROOT, path))); const files = [...walk(PLUGIN_ROOT, (name) => name.endsWith(".md")), join(REPO_ROOT, "README.md"), join(REPO_ROOT, "docs/reference.md")] .filter((path) => !generated.has(path)); - const descriptor = new RegExp(`\\b(?:${REGISTRY.adapters.map((adapter) => adapter.id).join("|")}):[A-Za-z0-9._-]+@[a-z]+`); + const descriptor = new RegExp(`\\b(?:${REGISTRY.adapters.map((adapter) => adapter.id).join("|")}):[A-Za-z0-9._/-]+@[a-z]+`); const versioned = REGISTRY.families .flatMap((entry) => [entry.model, ...(entry.predecessors ?? []).map((kept) => kept.model)]) .filter((model) => /[0-9]/.test(model)) @@ -327,7 +389,7 @@ describe("resolveLane", () => { const lane = resolveLane(request({ parent: "claude", provider: "grok", model: "grok-4.5", effort: "high" })); expect(lane.family).toBeNull(); expect(lane.model).toBe("grok-4.5"); - expect(lane.adapter.launch.argv(lane, { sharedGitDir: null })) + expect(lane.adapter.launch.argv(lane, { sharedGitDir: null, providerSidecar: null })) .toEqual(expect.arrayContaining(["--model", "grok-4.5", "--reasoning-effort", "high"])); expect(resolveLane(request({ parent: "claude", provider: "grok", model: "grok-4.5", effort: "default" })).effort) .toBe("default"); @@ -349,17 +411,89 @@ describe("resolveLane", () => { }); it("refuses an unnormalized Claude revision with today's message", () => { - for (const [model, alias] of [["claude-opus-5", "opus"], ["claude-fable-9-9", "fable"]]) { + for (const [model, alias] of [ + ["claude-opus-5", "opus"], + ["claude-fable-9-9", "fable"], + ["claude-sonnet-5-5", "sonnet"], + ]) { expect(() => resolveLane(request({ model }))) .toThrow(`Claude model ${model} is a version pin; normalize it to ${alias} before invoking the runner`); } expect(resolveLane(request({ model: "claude-opus-beta" })).family).toBeNull(); }); + it("resolves the registered Opus 5.5 pin before the opus revision pattern", () => { + expect(resolveModel("claude", "claude-opus-5-5")).toEqual({ + kind: "current", + family: REGISTRY.families.find((entry) => entry.name === "opus-5-5")!, + }); + for (const model of ["claude-opus-5", "claude-opus-5-6", "claude-opus-5-5-1"]) { + const match = resolveModel("claude", model); + expect([model, match.kind, match.kind === "stale-revision" ? match.family.name : null]) + .toEqual([model, "stale-revision", "opus"]); + } + const lane = resolveLane(request({ model: "claude-opus-5-5" })); + expect([lane.family?.name, lane.family?.rolling, lane.model]).toEqual(["opus-5-5", null, "claude-opus-5-5"]); + expect(lane.adapter.launch.argv(lane, { sharedGitDir: null, providerSidecar: null })).toEqual(expect.arrayContaining(["claude-opus-5-5"])); + }); + + it("resolves Sonnet and Luna, and rejects ultra for both", () => { + expect(resolveLane(request({ model: "sonnet", effort: "low" })).family?.name).toBe("sonnet"); + const luna = resolveLane(request({ parent: "claude", provider: "codex", model: "gpt-6-luna" })); + expect([luna.family?.name, luna.effort]).toEqual(["luna", "max"]); + expect(() => resolveLane(request({ parent: "claude", provider: "codex", model: "gpt-6-luna", effort: "ultra" }))) + .toThrow("effort ultra is not offered for codex:gpt-6-luna; choose one of: low, medium, high, xhigh, max"); + expect(() => resolveLane(request({ model: "sonnet", effort: "ultra" }))) + .toThrow("effort ultra is not offered for claude:sonnet"); + }); + + it("maps each Devin effort to its CLI model ID and keeps the descriptor's model for the receipt", () => { + const devin = (model: string, effort: string, mode = "read-only") => + resolveLane(request({ parent: "claude", provider: "devin", model, effort, mode })); + for (const [model, effort, cliModel] of [ + ["swe-2", "medium", "swe-2-medium"], ["swe-2", "high", "swe-2-high"], ["swe-2", "max", "swe-2-max"], + ["swe-1.6", "default", "swe-1-6"], + ]) { + const lane = devin(model!, effort!); + expect([lane.family?.name, lane.model, lane.cliModel]).toEqual([model, model, cliModel]); + } + expect(() => devin("swe-2", "default")).toThrow("effort default is not offered for devin:swe-2; choose one of: medium, high, max"); + // An unregistered Devin model is already a CLI model ID, so only the default effort runs it. + expect([devin("swe-1-7-medium", "default").cliModel, devin("swe-1-7-medium", "default").family]).toEqual(["swe-1-7-medium", null]); + expect(() => devin("swe-1-7-medium", "max")).toThrow("effort max is not offered for devin:swe-1-7-medium; choose one of: default"); + expect(() => devin("swe-2", "max", "isolated-write")) + .toThrow("provider devin does not support isolated-write; it supports only: read-only"); + for (const [provider, model] of [["codex", "gpt-6.1-sol"], ["grok", "grok-4.7"], ["grok", "grok-4.5"]]) { + expect(resolveLane(request({ parent: "claude", provider, model })).cliModel).toBe(model); + } + }); + + it("builds a bounded writer unless full access is requested, and refuses full access off a Codex writer", () => { + const codex = (overrides: Partial<LaneRequest>) => + resolveLane(request({ parent: "claude", provider: "codex", model: "gpt-6.1-sol", ...overrides })); + expect(codex({}).access).toEqual({ mode: "read-only" }); + expect(codex({ writerSandbox: "bounded" }).access).toEqual({ mode: "read-only" }); + expect(codex({ mode: "isolated-write" }).access).toEqual({ mode: "isolated-write", sandbox: "bounded" }); + expect(codex({ mode: "isolated-write", writerSandbox: "bounded" }).access).toEqual({ mode: "isolated-write", sandbox: "bounded" }); + expect(codex({ mode: "isolated-write", writerSandbox: "full-access" }).access) + .toEqual({ mode: "isolated-write", sandbox: "full-access" }); + expect(() => codex({ writerSandbox: "full-access" })).toThrow("writer sandbox full-access requires mode isolated-write"); + expect(() => codex({ mode: "isolated-write", writerSandbox: "danger-full-access" })) + .toThrow("writer-sandbox must be one of: bounded, full-access"); + expect(REGISTRY.adapters.filter((adapter) => adapter.launch.supports.includes("isolated-write-full-access")) + .map((adapter) => adapter.id)).toEqual(["codex"]); + for (const [parent, provider, model] of [["codex", "claude", "opus"], ["claude", "grok", "grok-4.7"]]) { + expect(() => resolveLane(request({ parent, provider, model, mode: "isolated-write", writerSandbox: "full-access" }))) + .toThrow(`provider ${provider} does not support isolated-write-full-access; it supports only: read-only, isolated-write`); + } + expect(() => resolveLane(request({ parent: "claude", provider: "devin", model: "swe-2", mode: "isolated-write", writerSandbox: "full-access" }))) + .toThrow("provider devin does not support isolated-write-full-access; it supports only: read-only"); + }); + it("rejects same-provider recursion and unknown names", () => { expect(() => resolveLane(request({ parent: "claude" }))) .toThrow("provider claude is native to parent claude; use the parent subagent primitive"); - expect(() => resolveLane(request({ provider: "devin" }))).toThrow("provider must be one of: claude, codex, grok"); + expect(() => resolveLane(request({ provider: "kiro" }))).toThrow("provider must be one of: claude, codex, grok, devin, opencode, dsh, amp, agy, cursor"); expect(() => resolveLane(request({ parent: "grok" }))).toThrow("parent must be one of: claude, codex"); expect(() => resolveLane(request({ mode: "full" }))).toThrow("mode must be one of: read-only, isolated-write"); expect(() => resolveLane(request({ model: " " }))).toThrow("model is required"); diff --git a/plugins/pstack/skills/poteto-mode/scripts/runner/registry/contract.ts b/plugins/pstack/skills/poteto-mode/scripts/runner/registry/contract.ts index b4ab77cb..3d27c190 100644 --- a/plugins/pstack/skills/poteto-mode/scripts/runner/registry/contract.ts +++ b/plugins/pstack/skills/poteto-mode/scripts/runner/registry/contract.ts @@ -1,9 +1,30 @@ -import type { AccessMode, Effort, NormalizedUsage, Parent } from "../types.ts"; +import { + type AccessKind, + type AccessMode, + type Effort, + type NormalizedUsage, + type Parent, + type ReceiptStatus, + UsageError, + type WriterSandbox, +} from "../types.ts"; declare const laneBrand: unique symbol; -export interface Access { - readonly mode: AccessMode; +/** Full access exists only on a writer, so "read-only with full access" cannot be built. */ +export type Access = + | { readonly mode: "read-only" } + | { readonly mode: "isolated-write"; readonly sandbox: WriterSandbox }; + +/** The only Access constructor. A writer is bounded unless the caller passes `full-access`. */ +export function access(mode: AccessMode, sandbox: WriterSandbox = "bounded"): Access { + if (mode === "isolated-write") return { mode, sandbox }; + if (sandbox === "full-access") throw new UsageError("writer sandbox full-access requires mode isolated-write"); + return { mode }; +} + +export function accessKind(value: Access): AccessKind { + return value.mode === "isolated-write" && value.sandbox === "full-access" ? "isolated-write-full-access" : value.mode; } /** A validated lane. resolveLane is the only constructor, so nothing downstream re-parses strings. */ @@ -18,8 +39,10 @@ export interface Lane { readonly model: string; readonly rolling: RollingAlias | null; } | null; - /** The descriptor's model component, passed to the CLI unchanged. */ + /** The descriptor's model component; the receipt's `model`. */ readonly model: string; + /** What argv passes. It differs from `model` only for a family whose effort is part of the model ID. */ + readonly cliModel: string; readonly effort: Effort; readonly access: Access; readonly promptPath: string; @@ -47,11 +70,18 @@ export interface ChildRun { readonly stderr: string; } -/** Only a preflight can say "unauthenticated" (#142). run.ts records the raw output of a failure. */ +/** A model run, with the bytes the CLI wrote to the provider sidecar's output, or null when it wrote none. */ +export interface ModelRun extends ChildRun { + readonly sidecar: string | null; +} + +/** Only a preflight can say "unauthenticated" (#142). run.ts records the raw output of a failure. + * "inconclusive" is status output the adapter cannot read; run.ts records it as child-failed. */ export type PreflightVerdict = | { readonly kind: "passed"; readonly evidence: string } | { readonly kind: "unauthenticated" } - | { readonly kind: "unavailable-model" }; + | { readonly kind: "unavailable-model" } + | { readonly kind: "inconclusive" }; export interface ProviderMetadata { readonly reportedModel: string | null; @@ -72,12 +102,32 @@ export type ParseResult = | { readonly kind: "malformed"; readonly message: string }; export type EnvPolicy = - | { readonly kind: "allowlist"; readonly names: readonly string[]; readonly prefixes: readonly string[] } + | { + readonly kind: "allowlist"; + readonly names: readonly string[]; + readonly prefixes: readonly string[]; + /** Run-owned values, applied after filtering. One replaces an inherited value of the same name. */ + readonly set?: Readonly<Record<string, string>>; + } | { readonly kind: "inherit"; readonly dropPrefixes: readonly string[] }; /** Facts run.ts establishes before argv. Each is null unless the adapter asked for it. */ export interface LaunchFacts { readonly sharedGitDir: string | null; + /** The lane's private provider directory, the absent output path in it and the written config. */ + readonly providerSidecar: { + readonly directory: string; + readonly output: string | null; + readonly config: string | null; + } | null; +} + +/** A private directory run.ts creates next to the receipt (0700). It writes `config` there (0600), creating any + * subdirectories in `name` (0700), before the preflight. It passes the absent `output` path and reads that file + * after the child exits; a null `output` means the CLI writes no file there. */ +export interface ProviderSidecar { + readonly output: string | null; + readonly config: { readonly name: string; content(lane: Lane): string } | null; } export interface ProviderAdapter { @@ -85,27 +135,44 @@ export interface ProviderAdapter { readonly id: string; /** Looked up on the child's PATH. Missing → unavailable-cli. */ readonly executable: string; + /** "flag": argv carries the effort, and omits the flag for `default`. + * "model-id": this provider's family entries map each effort to a CLI model ID. + * "config": the sidecar config carries the effort, and omits it for `default`. */ + readonly effortIn: "flag" | "model-id" | "config"; readonly preflight: { - argv(lane: Lane): readonly string[]; + /** sidecar is the lane's provider sidecar, already written, or null when the adapter declares none. */ + argv(lane: Lane, sidecar: LaunchFacts["providerSidecar"]): readonly string[]; /** One more preflight after an "unauthenticated" verdict, after this delay. null means none. */ readonly retryAfterUnauthenticatedMs: number | null; classify(run: ChildRun, lane: Lane): PreflightVerdict; + /** run.ts runs `<executable> <argv>` with the preflight's environment and cwd before the authentication preflight, + * and refuses the lane when `check` returns a refusal. The preflight then stays not-run. */ + readonly guard?: { + readonly argv: readonly string[]; + check(run: ChildRun): { readonly status: ReceiptStatus; readonly message: string } | null; + }; }; readonly launch: { /** "stdin": run.ts writes the prompt to stdin. "file": argv carries lane.promptPath. */ readonly prompt: "stdin" | "file"; + readonly sidecar: ProviderSidecar | null; + /** resolveLane refuses any other access kind before launch. */ + readonly supports: readonly AccessKind[]; + /** With "stdin" prompts, what run.ts writes instead of the bare prompt. */ + stdinEnvelope?(prompt: string): string; needsSharedGitDir(access: Access): boolean; /** Omits the effort flag for the `default` effort. */ argv(lane: Lane, facts: LaunchFacts): readonly string[]; }; /** run.ts calls parse for every child that exits by itself. After a nonzero exit it honors only provider-failure. */ - parse(run: ChildRun, lane: Lane): ParseResult; + parse(run: ModelRun, lane: Lane): ParseResult; readonly proof: { /** "absent": the CLI never reports the served model, so the exact argv is the evidence (pinned-argv). */ readonly report: "required" | "absent"; matches(lane: Lane, reported: string): boolean; }; - environment(access: Access): EnvPolicy; + /** sidecarDir is the lane's provider directory, or null when the adapter declares no sidecar. */ + environment(access: Access, sidecarDir: string | null): EnvPolicy; /** Markdown that render.ts puts into the model catalog's routes and provider notes. */ readonly docs: { readonly name: string; @@ -115,9 +182,9 @@ export interface ProviderAdapter { }; } -/** Keeps the literal id, from which the provider list derives ProviderId. */ -export function adapter<const Id extends string>( - value: ProviderAdapter & { readonly id: Id } -): ProviderAdapter & { readonly id: Id } { +/** Keeps the literal id and effort shape, from which the provider list derives ProviderId and family shapes. */ +export function adapter<const Id extends string, const E extends ProviderAdapter["effortIn"]>( + value: ProviderAdapter & { readonly id: Id; readonly effortIn: E } +): ProviderAdapter & { readonly id: Id; readonly effortIn: E } { return value; } diff --git a/plugins/pstack/skills/poteto-mode/scripts/runner/registry/family.ts b/plugins/pstack/skills/poteto-mode/scripts/runner/registry/family.ts index 2cd6d51e..4356d3f9 100644 --- a/plugins/pstack/skills/poteto-mode/scripts/runner/registry/family.ts +++ b/plugins/pstack/skills/poteto-mode/scripts/runner/registry/family.ts @@ -1,8 +1,9 @@ -import type { Effort } from "../types.ts"; +import { EFFORTS, type Effort } from "../types.ts"; import type { RollingAlias } from "./contract.ts"; import type { ADAPTERS } from "./providers.ts"; export type ProviderId = (typeof ADAPTERS)[number]["id"]; +type EffortInOf<P extends ProviderId> = Extract<(typeof ADAPTERS)[number], { readonly id: P }>["effortIn"]; export interface KeptPredecessor { readonly model: string; @@ -10,21 +11,40 @@ export interface KeptPredecessor { readonly efforts: readonly Effort[]; } -export interface FamilyEntry { +interface FamilyCommon<P extends ProviderId> { /** The key role defaults name. A Claude family's native agents are pstack-<name>-<effort>. */ readonly name: string; /** The matrix label. */ readonly label: string; - readonly provider: ProviderId; + readonly provider: P; /** The descriptor's model component. */ readonly model: string; - readonly efforts: readonly Effort[]; readonly defaultEffort: Effort; /** Cursor's selector for the same choice, kept for sync review. */ readonly upstream: string | null; /** Present for Claude rolling aliases. Drives report matching and read-time normalization. */ readonly rolling?: RollingAlias; - readonly predecessors?: readonly KeptPredecessor[]; +} + +type ModelIds = { readonly [E in Effort]?: string }; + +/** The provider picks the effort shape, so a Sol-shaped entry for a model-ID provider fails `tsc`. + * A model-ID family keeps no predecessor yet: no released pair of one has changed its model. */ +export type FamilyEntry<P extends ProviderId = ProviderId> = P extends ProviderId + ? FamilyCommon<P> & (EffortInOf<P> extends "model-id" + ? { readonly efforts: ModelIds; readonly predecessors?: never } + : { readonly efforts: readonly Effort[]; readonly predecessors?: readonly KeptPredecessor[] }) + : never; + +/** A model-ID family's map from effort to CLI model ID, or null for a family whose argv carries the effort. */ +export function modelIds(entry: FamilyEntry): ModelIds | null { + return Array.isArray(entry.efforts) ? null : entry.efforts as ModelIds; +} + +/** A family's selectable efforts, in EFFORTS order for a model-ID family. */ +export function effortsOf(entry: FamilyEntry): readonly Effort[] { + const ids = modelIds(entry); + return ids === null ? entry.efforts as readonly Effort[] : EFFORTS.filter((effort) => ids[effort] !== undefined); } export type LaneDefault<F extends string> = F | "inherit-parent" | "auto"; diff --git a/plugins/pstack/skills/poteto-mode/scripts/runner/registry/index.ts b/plugins/pstack/skills/poteto-mode/scripts/runner/registry/index.ts index 9d61f781..e1f6bd3d 100644 --- a/plugins/pstack/skills/poteto-mode/scripts/runner/registry/index.ts +++ b/plugins/pstack/skills/poteto-mode/scripts/runner/registry/index.ts @@ -1,10 +1,11 @@ -import { ACCESS_MODES, EFFORTS, PARENTS, type Parent, UsageError } from "../types.ts"; -import { isRevision, type Lane, type ProviderAdapter } from "./contract.ts"; -import type { FamilyEntry, KeptPredecessor, ReleasedPair, RoleDefault } from "./family.ts"; +import { ACCESS_MODES, EFFORTS, PARENTS, type Parent, UsageError, WRITER_SANDBOXES } from "../types.ts"; +import { access, accessKind, isRevision, type Lane, type ProviderAdapter } from "./contract.ts"; +import { effortsOf, type FamilyEntry, type KeptPredecessor, modelIds, type ReleasedPair, type RoleDefault } from "./family.ts"; import { FAMILIES, type FamilyName, RELEASED_PAIRS, ROLE_DEFAULTS } from "./models.ts"; import { ADAPTERS } from "./providers.ts"; export type { Lane, LaunchFacts, ParseResult } from "./contract.ts"; +export { effortsOf, modelIds } from "./family.ts"; /** The only registry module code outside registry/ imports. */ export interface Registry { @@ -27,6 +28,8 @@ export interface LaneRequest { readonly model: string; readonly effort: string; readonly mode: string; + /** Absent means bounded. */ + readonly writerSandbox?: string; readonly promptPath: string; readonly cwd: string; readonly outputPath: string; @@ -63,7 +66,10 @@ function oneOf<T extends string>(name: string, value: string, choices: readonly /** Raw request → Lane, or a UsageError. An unregistered pair runs unchanged. */ export function resolveLane(request: LaneRequest): Lane { - const mode = oneOf("mode", request.mode, ACCESS_MODES); + const laneAccess = access( + oneOf("mode", request.mode, ACCESS_MODES), + request.writerSandbox === undefined ? undefined : oneOf("writer-sandbox", request.writerSandbox, WRITER_SANDBOXES) + ); const parent = oneOf("parent", request.parent, Object.keys(PARENTS) as Parent[]); const adapter = REGISTRY.adapters.find((entry) => entry.id === request.provider); if (adapter === undefined) { @@ -83,12 +89,23 @@ export function resolveLane(request: LaneRequest): Lane { ); } const family = match.kind === "unregistered" ? null : match.family; - const offered = match.kind === "kept" ? match.predecessor.efforts : family?.efforts ?? EFFORTS; + // An unregistered model-ID pair names the CLI model ID itself, so no effort can be mapped onto it. + const offered = match.kind === "kept" + ? match.predecessor.efforts + : family === null + ? adapter.effortIn === "model-id" ? ["default" as const] : EFFORTS + : effortsOf(family); if (!offered.includes(effort)) { throw new UsageError( `effort ${effort} is not offered for ${adapter.id}:${request.model}; choose one of: ${offered.join(", ")}` ); } + const kind = accessKind(laneAccess); + if (!adapter.launch.supports.includes(kind)) { + throw new UsageError( + `provider ${adapter.id} does not support ${kind}; it supports only: ${adapter.launch.supports.join(", ")}` + ); + } return { parent, adapter, @@ -99,8 +116,9 @@ export function resolveLane(request: LaneRequest): Lane { rolling: family.rolling ?? null, }, model: request.model, + cliModel: (family === null ? undefined : modelIds(family)?.[effort]) ?? request.model, effort, - access: { mode }, + access: laneAccess, promptPath: request.promptPath, cwd: request.cwd, outputPath: request.outputPath, diff --git a/plugins/pstack/skills/poteto-mode/scripts/runner/registry/models.ts b/plugins/pstack/skills/poteto-mode/scripts/runner/registry/models.ts index 5434aada..742378d5 100644 --- a/plugins/pstack/skills/poteto-mode/scripts/runner/registry/models.ts +++ b/plugins/pstack/skills/poteto-mode/scripts/runner/registry/models.ts @@ -44,6 +44,118 @@ export const FAMILIES = [ upstream: "opus", rolling: { revisionPrefix: "claude-opus-" }, }), + family({ + name: "luna", + label: "GPT-6 Luna", + provider: "codex", + model: "gpt-6-luna", + efforts: UP_TO_MAX, // no ultra in `codex debug models`, Codex CLI 0.159.0 + defaultEffort: "max", + upstream: null, + }), + family({ + name: "sonnet", + label: "Sonnet", + provider: "claude", + model: "sonnet", + efforts: UP_TO_MAX, + defaultEffort: "max", + upstream: null, + rolling: { revisionPrefix: "claude-sonnet-" }, + }), + family({ + name: "opus-5-5", + label: "Opus 5.5", + provider: "claude", + model: "claude-opus-5-5", // an exact pin: resolveModel matches it before the opus revision pattern + efforts: UP_TO_MAX, + defaultEffort: "max", + upstream: "claude-opus-5-5-xhigh", + }), + family({ + name: "swe-2", + label: "SWE-2", + provider: "devin", + model: "swe-2", + // `devin models list --format json`, Devin CLI 3000.10.21: no low variant. The bare slug runs swe-2-max. + efforts: { medium: "swe-2-medium", high: "swe-2-high", max: "swe-2-max" }, + defaultEffort: "max", + upstream: null, + }), + family({ + name: "swe-1.6", + label: "SWE-1.6", + provider: "devin", + model: "swe-1.6", + efforts: { default: "swe-1-6" }, // the family's single variant + defaultEffort: "default", + upstream: null, + }), + family({ + name: "deepseek-v4.1-flash-free", + label: "DeepSeek V4.1 Flash (OpenCode Zen)", + provider: "opencode", + model: "opencode/deepseek-v4.1-flash-free", + efforts: ["low", "high", "max"], // `opencode models --verbose` variants, OpenCode 1.19.1 + defaultEffort: "max", + upstream: null, + }), + family({ + name: "deepseek-flash", + label: "DeepSeek Flash (dsh)", + provider: "dsh", + model: "deepseek-flash", + efforts: ["low", "high", "max"], // dsh 0.2.0-rc.2 fails `medium` locally + defaultEffort: "max", + upstream: null, + }), + family({ + name: "deepseek-v4-pro", + label: "DeepSeek V4 Pro (dsh)", + provider: "dsh", + model: "deepseek-v4-pro", + efforts: ["low", "high", "max"], + defaultEffort: "max", + upstream: null, + }), + family({ + name: "amp-deepseek-v4.1-flash", + label: "DeepSeek V4.1 Flash (Amp)", + provider: "amp", + model: "deepseek-v4.1-flash", + efforts: { default: "deepseek-v4.1-flash" }, // an official Amp mode with no effort variants + defaultEffort: "default", + upstream: null, + }), + family({ + name: "gemini-3.8-flash", + label: "Gemini 3.8 Flash (Antigravity)", + provider: "agy", + model: "gemini-3.8-flash", + // `agy models`, Antigravity CLI 1.3.6. Gemini 3.7 Flash is not listed. The bare slug runs gemini-3.8-flash-high. + efforts: { low: "gemini-3.8-flash-low", medium: "gemini-3.8-flash-medium", high: "gemini-3.8-flash-high" }, + defaultEffort: "high", + upstream: null, + }), + family({ + name: "composer-2.5", + label: "Composer 2.5 (Cursor)", + provider: "cursor", + model: "composer-2.5", + efforts: { default: "composer-2.5" }, // `cursor-agent models`, Cursor CLI 2026.10.14-0f9edde + defaultEffort: "default", + upstream: null, + }), + family({ + name: "gemini-3.7-flash", + label: "Gemini 3.7 Flash (Cursor)", + provider: "cursor", + model: "gemini-3.7-flash", + // `cursor-agent models`, Cursor CLI 2026.10.14-0f9edde. There is no bare `gemini-3.7-flash` ID. + efforts: { low: "gemini-3.7-flash-low", medium: "gemini-3.7-flash-medium", high: "gemini-3.7-flash-high" }, + defaultEffort: "high", + upstream: null, + }), ] as const; export type FamilyName = (typeof FAMILIES)[number]["name"]; @@ -76,4 +188,16 @@ export const RELEASED_PAIRS = [ { pair: "codex:gpt-6.1-sol", family: "sol" }, { pair: "grok:grok-4.6", family: "grok" }, { pair: "grok:grok-4.7", family: "grok" }, + { pair: "codex:gpt-6-luna", family: "luna" }, + { pair: "claude:sonnet", family: "sonnet" }, + { pair: "claude:claude-opus-5-5", family: "opus-5-5" }, + { pair: "devin:swe-2", family: "swe-2" }, + { pair: "devin:swe-1.6", family: "swe-1.6" }, + { pair: "opencode:opencode/deepseek-v4.1-flash-free", family: "deepseek-v4.1-flash-free" }, + { pair: "dsh:deepseek-flash", family: "deepseek-flash" }, + { pair: "dsh:deepseek-v4-pro", family: "deepseek-v4-pro" }, + { pair: "amp:deepseek-v4.1-flash", family: "amp-deepseek-v4.1-flash" }, + { pair: "agy:gemini-3.8-flash", family: "gemini-3.8-flash" }, + { pair: "cursor:composer-2.5", family: "composer-2.5" }, + { pair: "cursor:gemini-3.7-flash", family: "gemini-3.7-flash" }, ] as const satisfies readonly ReleasedPair<FamilyName>[]; diff --git a/plugins/pstack/skills/poteto-mode/scripts/runner/registry/providers.ts b/plugins/pstack/skills/poteto-mode/scripts/runner/registry/providers.ts index 980dc253..6b9dd32a 100644 --- a/plugins/pstack/skills/poteto-mode/scripts/runner/registry/providers.ts +++ b/plugins/pstack/skills/poteto-mode/scripts/runner/registry/providers.ts @@ -1,6 +1,12 @@ +import { agy } from "../providers/agy.ts"; +import { amp } from "../providers/amp.ts"; import { claude } from "../providers/claude.ts"; import { codex } from "../providers/codex.ts"; +import { cursor } from "../providers/cursor.ts"; +import { devin } from "../providers/devin.ts"; +import { dsh } from "../providers/dsh.ts"; import { grok } from "../providers/grok.ts"; +import { opencode } from "../providers/opencode.ts"; /** The only list of providers. */ -export const ADAPTERS = [claude, codex, grok] as const; +export const ADAPTERS = [claude, codex, grok, devin, opencode, dsh, amp, agy, cursor] as const; diff --git a/plugins/pstack/skills/poteto-mode/scripts/runner/registry/render.ts b/plugins/pstack/skills/poteto-mode/scripts/runner/registry/render.ts index 2bbc0b47..74fe2488 100644 --- a/plugins/pstack/skills/poteto-mode/scripts/runner/registry/render.ts +++ b/plugins/pstack/skills/poteto-mode/scripts/runner/registry/render.ts @@ -1,8 +1,8 @@ import { existsSync, mkdirSync, readdirSync, readFileSync, rmSync, writeFileSync } from "node:fs"; import { dirname, join } from "node:path"; import { ACCESS_MODES, PARENTS, type Parent } from "../types.ts"; -import type { EnvPolicy, ProviderAdapter } from "./contract.ts"; -import { defaultDescriptor, REGISTRY, type Registry } from "./index.ts"; +import { access, type EnvPolicy, type ProviderAdapter } from "./contract.ts"; +import { defaultDescriptor, effortsOf, modelIds, REGISTRY, type Registry } from "./index.ts"; export interface GeneratedFile { /** Relative to the plugin root. */ @@ -41,11 +41,14 @@ function policy(value: EnvPolicy, long: boolean): string { : `inherited without ${value.dropPrefixes.map((prefix) => code(`${prefix}*`)).join(", ")}`; } if (!long) return "allowlist"; - return `only ${value.names.map(code).join(" ")}, and variables starting with ${value.prefixes.map(code).join(", ")}`; + const set = Object.entries(value.set ?? {}).map(([name, setting]) => code(`${name}=${setting}`)); + return `only ${value.names.map(code).join(" ")}, and variables starting with ${value.prefixes.map(code).join(", ")}` + + (set.length === 0 ? "" : `; sets ${set.join(", ")}`); } function environment(adapter: ProviderAdapter, long: boolean): string { - const [read, write] = ACCESS_MODES.map((mode) => policy(adapter.environment({ mode }), long)); + const sidecarDir = adapter.launch.sidecar === null ? null : "<receipt>.provider"; + const [read, write] = ACCESS_MODES.map((mode) => policy(adapter.environment(access(mode), sidecarDir), long)); return read === write ? read! : `read-only: ${read}; isolated-write: ${write}`; } @@ -72,7 +75,7 @@ export function renderCatalog(registry: Registry): GeneratedFile { family.provider, family.model, family.defaultEffort, - family.efforts.join(" "), + effortsOf(family).join(" "), family.provider === "claude" ? code(`pstack-${family.name}-<effort>`) : "-", named.has(family.name) ? "default" : "optional", family.upstream ?? "-", @@ -81,6 +84,16 @@ export function renderCatalog(registry: Registry): GeneratedFile { "", "A family is `default` when a role default names it and `optional` otherwise. A Claude-native agent exists for each selectable effort of a Claude family.", "", + "Effort in the model ID: for these families the effort selects the CLI model, and argv carries no effort flag.", + "", + ...table( + ["Family", "Effort", "CLI model"], + registry.families.flatMap((family) => effortsOf(family).flatMap((effort) => { + const id = modelIds(family)?.[effort]; + return id === undefined ? [] : [[family.name, effort, code(id)]]; + })) + ), + "", "## Kept predecessors and normalization", "", ...table( @@ -108,10 +121,11 @@ export function renderCatalog(registry: Registry): GeneratedFile { "## Routes", "", ...table( - ["Provider", ...parents.map((parent) => `${PARENTS[parent].harness} parent`), "Prompt", "Model proof", "Child environment", "Host prerequisites"], + ["Provider", ...parents.map((parent) => `${PARENTS[parent].harness} parent`), "Access modes", "Prompt", "Model proof", "Child environment", "Host prerequisites"], registry.adapters.map((adapter) => [ adapter.id, ...parents.map((parent) => route(adapter, parent)), + adapter.launch.supports.map(code).join(", "), adapter.launch.prompt, adapter.proof.report === "required" ? "provider report" : "pinned argv", environment(adapter, false), @@ -119,6 +133,8 @@ export function renderCatalog(registry: Registry): GeneratedFile { ]) ), "", + "The runner refuses an access mode outside a provider's Access modes cell before launch. `isolated-write-full-access` is a writer capability, not a `--mode` value: the runner takes it as `--mode isolated-write --writer-sandbox full-access`, which a parent passes only when the model sheet opts in, as [`provider-dispatch.md`](provider-dispatch.md#codex-writer-sandbox) says.", + "", "## Provider notes", ...registry.adapters.flatMap((adapter) => [ "", @@ -151,7 +167,7 @@ export function renderCatalog(registry: Registry): GeneratedFile { /** pstack-<family>-<effort>.md for each Claude family and each selectable effort. */ export function renderNativeAgents(registry: Registry): readonly GeneratedFile[] { return registry.families.filter((family) => family.provider === "claude").flatMap((family) => - family.efforts.map((effort) => { + effortsOf(family).map((effort) => { const name = `pstack-${family.name}-${effort}`; return { path: `agents/${name}.md`, diff --git a/plugins/pstack/skills/poteto-mode/scripts/runner/run.test.ts b/plugins/pstack/skills/poteto-mode/scripts/runner/run.test.ts index 963503a7..5e1a407b 100644 --- a/plugins/pstack/skills/poteto-mode/scripts/runner/run.test.ts +++ b/plugins/pstack/skills/poteto-mode/scripts/runner/run.test.ts @@ -13,11 +13,15 @@ import { symlinkSync, writeFileSync, } from "node:fs"; -import { dirname, join } from "node:path"; +import { dirname, join, resolve } from "node:path"; import { tmpdir } from "node:os"; import { childEnvironment, discoverSharedGitDir, runLane } from "./run.ts"; import { main } from "./cli.ts"; import { type LaneRequest, resolveLane } from "./registry/index.ts"; +import type { Lane, ProviderAdapter } from "./registry/contract.ts"; +import { devin } from "./providers/devin.ts"; +import { opencode } from "./providers/opencode.ts"; +import { BASE_ALLOWLIST, BASE_PREFIXES } from "./providers/shared.ts"; import type { ProviderId } from "./registry/family.ts"; import type { RunnerReceipt } from "./types.ts"; @@ -830,6 +834,28 @@ printf '%s\\n' '{"type":"thread.started","thread_id":"isolated"}' '{"type":"item }); } + it("records the writer sandbox the lane ran with: none for a reader, bounded by default, and full access on opt-in", async () => { + const lanes = [ + [options("codex", "sandbox-reader"), null, "read-only"], + [{ ...options("codex", "sandbox-bounded"), mode: "isolated-write" }, "bounded", "workspace-write"], + [{ ...options("codex", "sandbox-full"), mode: "isolated-write", writerSandbox: "full-access" }, "full-access", "danger-full-access"], + ] as const; + for (const [input, writerSandbox, sandbox] of lanes) { + expect((await runLane(laneOf(input))).exitCode).toBe(0); + const saved = receipt(input.receiptPath); + expect([saved.status, saved.writerSandbox]).toEqual(["complete", writerSandbox]); + expect(saved.argv.slice(saved.argv.indexOf("--sandbox"), saved.argv.indexOf("--sandbox") + 2)).toEqual(["--sandbox", sandbox]); + } + }); + + it("records the opted-in writer sandbox on a failure receipt", async () => { + process.env.PATH = join(scratch, "empty-bin"); + mkdirSync(process.env.PATH); + const input = { ...options("codex", "sandbox-missing-cli"), mode: "isolated-write", writerSandbox: "full-access" }; + expect((await runLane(laneOf(input))).exitCode).toBe(69); + expect(receipt(input.receiptPath)).toMatchObject({ status: "unavailable-cli", writerSandbox: "full-access" }); + }); + it("records Codex's exact argv without fabricating a reported model", async () => { const input = options("codex"); const result = await runLane(laneOf(input)); @@ -1661,7 +1687,282 @@ const startedPath =`)); }); }); +describe("devin lanes", () => { + const LOGGED_IN = "Logged in (via Devin).\n\nUser: <redacted>"; + const DEVIN_VARIABLES = ["DEVIN_PERMISSION_MODE", "DEVIN_SANDBOX", "DEVIN_MODEL", "DEVIN_REFUSAL_FALLBACK"]; + + // A Devin stand-in: `auth status` prints `status`; the model call runs `turn`, which sees `args`, `at`, + // `exportPath` and `config`. The child environment keeps no test variable, so each case bakes its own. + function fakeDevin(status: string, turn: string): void { + writeFileSync(join(bin, "devin"), `#!/usr/bin/env bun +import { existsSync, readFileSync, statSync, writeFileSync } from "node:fs"; +import { dirname } from "node:path"; +const args = process.argv.slice(2); +if (args[0] === "auth") { + console.log(${JSON.stringify(status)}); + process.exit(0); +} +const at = (flag) => args[args.indexOf(flag) + 1]; +const exportPath = at("--export"), config = at("--config"); +${turn} +`); + chmodSync(join(bin, "devin"), 0o755); + } + + // Reports what the child saw as the export's final message, and prints a decoy on stdout. + const OBSERVE = `const observed = { + model: at("--model"), + exportAbsent: !existsSync(exportPath), + dirMode: (statSync(dirname(config)).mode & 0o777).toString(8), + configMode: (statSync(config).mode & 0o777).toString(8), + subagents: JSON.parse(readFileSync(config, "utf8")).subagents_enabled, + devinVariables: Object.keys(process.env).filter((key) => key.startsWith("DEVIN_")), +}; +console.log("Welcome to Devin CLI!\\nSTDOUT_IS_NOT_THE_REPLY"); +writeFileSync(exportPath, JSON.stringify({ + session_id: "d1", + steps: [{ source: "agent", message: JSON.stringify(observed), extra: { generation_model: at("--model") } }], + final_metrics: { total_prompt_tokens: 5, total_completion_tokens: 2 }, +}));`; + + function devinOptions(suffix: string, overrides: Partial<LaneRequest> = {}): LaneRequest { + return { ...options("grok", suffix), provider: "devin", model: "swe-2", effort: "max", ...overrides }; + } + + beforeEach(() => { + for (const name of DEVIN_VARIABLES) process.env[name] = `${name}-value`; + }); + afterEach(() => { + for (const name of DEVIN_VARIABLES) delete process.env[name]; + }); + + for (const [model, effort, cliModel] of [["swe-2", "max", "swe-2-max"], ["swe-1.6", "default", "swe-1-6"]] as const) { + it(`runs devin:${model}@${effort} as ${cliModel}, with the reply and proof from the export`, async () => { + fakeDevin(LOGGED_IN, OBSERVE); + const input = devinOptions(`devin-${model}`, { model, effort }); + const result = await runLane(laneOf(input)); + expect(result.exitCode).toBe(0); + const saved = receipt(input.receiptPath); + expect(saved).toMatchObject({ + status: "complete", + provider: "devin", + model, + effort, + mode: "read-only", + reportedModel: cliModel, + modelVerified: true, + modelEvidence: "provider-report", + sessionId: "d1", + usage: { inputTokens: 5, outputTokens: 2 }, + preflight: { status: "passed", evidence: "authenticated" }, + }); + const dir = `${input.receiptPath}.provider`; + expect(saved.argv.slice(1)).toEqual([ + "-p", "--prompt-file", input.promptPath, "--model", cliModel, "--permission-mode", "auto", + "--respect-workspace-trust", "false", "--export", join(dir, "export.json"), "--config", join(dir, "lockdown.json"), + ]); + expect(JSON.parse(readFileSync(input.outputPath, "utf8"))).toEqual({ + model: cliModel, + exportAbsent: true, + dirMode: "700", + configMode: "600", + subagents: false, + devinVariables: [], + }); + expect(readFileSync(streamPath(saved.stdoutPath), "utf8")).toContain("STDOUT_IS_NOT_THE_REPLY"); + expect(statSync(dir).mode & 0o777).toBe(0o700); + }); + } + + it("refuses an export whose generation model is not the CLI model ID", async () => { + fakeDevin(LOGGED_IN, `writeFileSync(exportPath, JSON.stringify({ steps: [{ source: "agent", message: "OK", extra: { generation_model: "swe-2" } }] }));`); + const input = devinOptions("devin-mismatch"); + expect((await runLane(laneOf(input))).exitCode).toBe(65); + expect(receipt(input.receiptPath)).toMatchObject({ + status: "malformed-output", + error: { message: "requested model swe-2 was not reported by devin" }, + }); + expect(existsSync(input.outputPath)).toBe(false); + }); + + for (const [name, status, exitCode, message] of [ + ["logged out at exit 0", "Not logged in.\n Credentials path: /home/fixture/.local/share/devin/credentials.toml", 77, "authentication or model preflight failed"], + ["an unknown status", "Welcome to Devin CLI!", 70, "authentication preflight output was inconclusive"], + ] as const) { + it(`classifies ${name} before any model call`, async () => { + const started = join(scratch, "devin-model.started"); + fakeDevin(status, `writeFileSync(${JSON.stringify(started)}, "started");`); + const input = devinOptions(`devin-preflight-${exitCode}`); + const result = await runLane(laneOf(input)); + expect(result.exitCode).toBe(exitCode); + expect(result.receipt).toMatchObject({ + status: exitCode === 77 ? "unauthenticated" : "child-failed", + preflight: { status: "failed" }, + error: { message }, + }); + expect(existsSync(started)).toBe(false); + }); + } + + it("classifies an unknown model, a rejected tool and SIGTERM from the captured shapes", async () => { + fakeDevin(LOGGED_IN, `console.error("Error: Unknown model: 'swe-2-max'\\nAvailable: adaptive, swe-2, swe-1.6"); +process.exit(1);`); + const unknown = devinOptions("devin-unknown-model"); + expect((await runLane(laneOf(unknown))).receipt).toMatchObject({ status: "unavailable-model", exitCode: 1 }); + + fakeDevin(LOGGED_IN, `console.error("warning: rejected a tool call that requires confirmation."); +writeFileSync(exportPath, JSON.stringify({ steps: [{ source: "agent", message: "", extra: { generation_model: at("--model") } }] }));`); + const rejected = devinOptions("devin-rejected"); + const rejectedResult = await runLane(laneOf(rejected)); + expect(rejectedResult.exitCode).toBe(70); + expect(rejectedResult.receipt).toMatchObject({ + status: "child-failed", + exitCode: 0, + error: { message: "devin ended with an empty final message" }, + }); + expect(rejectedResult.receipt.error?.evidence).toContain("rejected a tool call"); + expect(existsSync(rejected.outputPath)).toBe(false); + + fakeDevin(LOGGED_IN, `process.stdout.write("partial reply"); +process.kill(process.pid, "SIGTERM"); +await Bun.sleep(5_000);`); + const terminated = devinOptions("devin-sigterm"); + const terminatedResult = await runLane(laneOf(terminated)); + expect(terminatedResult.exitCode).toBe(130); + expect(terminatedResult.receipt).toMatchObject({ status: "cancelled", exitCode: 143, signal: null }); + expect(existsSync(terminated.outputPath)).toBe(false); + }); + + it("refuses a lane whose provider directory already exists and rolls back its reservations", async () => { + fakeDevin(LOGGED_IN, OBSERVE); + const input = devinOptions("devin-taken"); + mkdirSync(`${input.receiptPath}.provider`); + await expect(runLane(laneOf(input))).rejects.toThrow(); + for (const path of [input.outputPath, input.receiptPath, `${input.receiptPath}.stdout`, `${input.receiptPath}.stderr`]) { + expect([path, existsSync(path)]).toEqual([path, false]); + } + }); + + it("runs a Devin lane in a repository that holds .devin/", async () => { + fakeDevin(LOGGED_IN, OBSERVE); + const repo = join(scratch, "repo"); + fs.mkdirSync(join(repo, ".git"), { recursive: true }); + fs.mkdirSync(join(repo, ".devin")); + const input = devinOptions("devin-repo-config", { cwd: repo }); + expect((await runLane(laneOf(input))).receipt).toMatchObject({ status: "complete", exitCode: 0 }); + }); + + it("refuses an isolated-write Devin lane before launch", () => { + const input = devinOptions("devin-writer", { mode: "isolated-write" }); + expect(() => laneOf(input)).toThrow("provider devin does not support isolated-write"); + expect(existsSync(input.receiptPath)).toBe(false); + }); +}); + +// A provider that uses every optional part of the contract: run-owned environment values, a nested sidecar config +// with no output file and a stdin envelope. +const STAND_IN_CONFIG = ".agents/agents/lane.md"; +const STAND_IN: ProviderAdapter = { + ...devin, + executable: "standin", + effortIn: "config", + preflight: { + argv: () => ["preflight"], + retryAfterUnauthenticatedMs: null, + classify: (run) => run.exitCode === 0 ? { kind: "passed", evidence: run.stdout.trim() } : { kind: "inconclusive" }, + }, + launch: { + prompt: "stdin", + sidecar: { output: null, config: { name: STAND_IN_CONFIG, content: (lane) => `effort: ${lane.effort}\n` } }, + supports: ["read-only"], + stdinEnvelope: (prompt) => `${JSON.stringify({ event: "user", message: { role: "user", content: prompt } })}\n`, + needsSharedGitDir: () => false, + argv: () => ["run"], + }, + parse: (run) => ({ + kind: "complete", + text: run.stdout, + metadata: { reportedModel: null, sessionId: null, usage: null, costUsd: null }, + }), + proof: { report: "absent", matches: () => false }, + environment: (_access, sidecarDir) => ({ + kind: "allowlist", + names: BASE_ALLOWLIST, + prefixes: BASE_PREFIXES, + set: { XDG_CONFIG_HOME: `${sidecarDir}`, STAND_IN_CONFIG: `${sidecarDir}/${STAND_IN_CONFIG}`, CODEX_THREAD_ID: "restored" }, + }), +}; + +function standInLane(suffix: string, overrides: Partial<LaneRequest> = {}): Lane { + return { ...laneOf({ ...options("grok", suffix), provider: "devin", model: "swe-2", effort: "max", ...overrides }), adapter: STAND_IN }; +} + +describe("stand-in provider contract", () => { + let inherited: string | undefined; + beforeEach(() => { + // The preflight reports what it sees; the model call echoes stdin. + writeFileSync(join(bin, "standin"), `#!/usr/bin/env bun +import { readFileSync, statSync } from "node:fs"; +import { dirname } from "node:path"; +const mode = (path) => (statSync(path).mode & 0o777).toString(8); +if (process.argv[2] === "preflight") { + const config = process.env.STAND_IN_CONFIG; + console.log(JSON.stringify({ + xdg: process.env.XDG_CONFIG_HOME, + codexThread: process.env.CODEX_THREAD_ID ?? null, + config: readFileSync(config, "utf8"), + modes: [mode(dirname(dirname(config))), mode(dirname(config)), mode(config)], + })); + process.exit(0); +} +process.stdout.write(await Bun.stdin.text()); +`); + chmodSync(join(bin, "standin"), 0o755); + inherited = process.env.XDG_CONFIG_HOME; + process.env.XDG_CONFIG_HOME = "/inherited/config"; + }); + afterEach(() => { + if (inherited === undefined) delete process.env.XDG_CONFIG_HOME; + else process.env.XDG_CONFIG_HOME = inherited; + }); + + it("creates the nested config before the preflight, which sees the set environment, and sends the envelope", async () => { + const lane = standInLane("stand-in"); + writeFileSync(lane.promptPath, "Return \"the\" marker.\n"); + const result = await runLane(lane); + expect(result.exitCode).toBe(0); + const dir = `${lane.receiptPath}.provider`; + expect(result.receipt).toMatchObject({ + status: "complete", + reportedModel: null, + modelVerified: false, + modelEvidence: "pinned-argv", + preflight: { status: "passed" }, + }); + expect(JSON.parse(result.receipt.preflight.evidence)).toEqual({ + xdg: dir, + codexThread: null, + config: "effort: max\n", + modes: ["700", "700", "600"], + }); + expect(readFileSync(lane.outputPath, "utf8")) + .toBe(`${JSON.stringify({ event: "user", message: { role: "user", content: "Return \"the\" marker.\n" } })}\n`); + }); +}); + describe("childEnvironment", () => { + it("applies an adapter's set values after filtering and drops the parent identity markers last", () => { + const lane = standInLane("stand-in-env"); + const source = { PATH: "/bin", XDG_CONFIG_HOME: "/inherited", XDG_DATA_HOME: "/data", CODEX_THREAD_ID: "parent", GITHUB_TOKEN: "t" }; + const dir = `${lane.receiptPath}.provider`; + expect(childEnvironment(lane, source)).toEqual({ + PATH: "/bin", + XDG_CONFIG_HOME: dir, + XDG_DATA_HOME: "/data", + STAND_IN_CONFIG: `${dir}/${STAND_IN_CONFIG}`, + }); + }); + it("removes only inherited runtime identity needed to avoid nested detection", () => { const source = { PATH: "/bin", @@ -1718,6 +2019,34 @@ describe("childEnvironment", () => { } }); + it("passes a Devin lane the shared base allowlist without any DEVIN_* or Grok variable", () => { + const cases: [name: string, kept: boolean][] = [ + ["PATH", true], + ["HOME", true], + ["TZ", true], + ["LC_ALL", true], + ["XDG_DATA_HOME", true], + ["HTTPS_PROXY", true], + ["NODE_EXTRA_CA_CERTS", true], + ["DEVIN_PERMISSION_MODE", false], + ["DEVIN_SANDBOX", false], + ["DEVIN_MODEL", false], + ["DEVIN_REFUSAL_FALLBACK", false], + ["GROK_HOME", false], + ["XAI_API_KEY", false], + ["GITHUB_TOKEN", false], + ["CLAUDECODE", false], + ["CODEX_THREAD_ID", false], + ]; + const source = Object.fromEntries(cases.map(([name]) => [name, `${name}-value`])); + const kept = Object.fromEntries(cases.filter(([, keep]) => keep).map(([name]) => [name, `${name}-value`])); + const devin = laneOf({ ...options("grok"), provider: "devin", model: "swe-2", effort: "max" }); + expect(childEnvironment(devin, source)).toEqual(kept); + const grokOnly = ["GROK_HOME", "XAI_API_KEY"]; + expect(Object.keys(childEnvironment(laneOf(options("grok")), source)).sort()) + .toEqual([...Object.keys(kept), ...grokOnly].sort()); + }); + it("drops GIT_* only for a Claude writer", () => { const source = { PATH: "/bin", GIT_DIR: "/elsewhere/.git", GIT_WORK_TREE: "/elsewhere" }; expect(childEnvironment(laneOf({ ...options("claude"), mode: "isolated-write" }), source)).toEqual({ PATH: "/bin" }); @@ -1784,3 +2113,860 @@ describe("discoverSharedGitDir", () => { } }); }); + +describe("opencode lanes", () => { + const MODEL = "opencode/deepseek-v4.1-flash-free"; + const LISTED = `opencode/big-pickle\n${MODEL}\nopencode/nemotron-3-ultra-free`; + const INHERITED = ["OPENCODE_PERMISSION", "OPENCODE_CONFIG_CONTENT", "OPENROUTER_API_KEY"]; + const event = (type: string, part: Record<string, unknown> = {}) => ({ type, timestamp: 1, sessionID: "ses_fixture", part }); + + // An OpenCode stand-in: `models` runs `models`; the model call runs `turn`, which + // sees `args` and `prompt`. Each call appends its kind to `opencode.calls` next to the stand-in, and its cwd, + // environment and config file to `opencode.contexts`. The child environment keeps no test variable, so each case + // bakes its own. + function fakeOpencode(models: string, turn: string): void { + writeFileSync(join(bin, "opencode"), `#!/usr/bin/env bun +import { appendFileSync, readFileSync, statSync, writeFileSync } from "node:fs"; +import { dirname, join } from "node:path"; +const args = process.argv.slice(2); +const call = (kind) => { + appendFileSync(join(dirname(process.argv[1]), "opencode.calls"), kind + "\\n"); + const context = { cwd: process.cwd(), env: process.env, config: readFileSync(process.env.OPENCODE_CONFIG, "utf8") }; + appendFileSync(join(dirname(process.argv[1]), "opencode.contexts"), JSON.stringify(context) + "\\n"); +}; +if (args[0] === "models") { +call("preflight"); +${models} +} +call("model"); +const prompt = await Bun.stdin.text(); +const emit = (value) => console.log(JSON.stringify(value)); +${turn} +`); + chmodSync(join(bin, "opencode"), 0o755); + } + + const LIST = `console.log(${JSON.stringify(LISTED)}); process.exit(0);`; + // Reports what the child saw as the reply, over two steps, the first ending on an allowed tool call. + const OBSERVE = `const config = process.env.OPENCODE_CONFIG; +const observed = { + args, + prompt, + xdg: process.env.XDG_CONFIG_HOME === dirname(config), + modes: [(statSync(dirname(config)).mode & 0o777).toString(8), (statSync(config).mode & 0o777).toString(8)], + permission: JSON.parse(readFileSync(config, "utf8")).permission["*"], + opencode: Object.keys(process.env).filter((key) => key.startsWith("OPENCODE_") || key.endsWith("_API_KEY")).sort(), +}; +emit(${JSON.stringify(event("step_start"))}); +emit(${JSON.stringify(event("step_finish", { reason: "tool-calls", tokens: { total: 10, input: 6, output: 3, reasoning: 1, cache: { read: 2, write: 0 } }, cost: 0.5 }))}); +emit(${JSON.stringify(event("step_start"))}); +emit({ type: "text", timestamp: 2, sessionID: "ses_fixture", part: { type: "text", text: JSON.stringify(observed) } }); +emit(${JSON.stringify(event("step_finish", { reason: "stop", tokens: { total: 5, input: 4, output: 1, reasoning: 0, cache: { read: 0, write: 0 } }, cost: 0.25 }))});`; + + function opencodeOptions(suffix: string, overrides: Partial<LaneRequest> = {}): LaneRequest { + return { ...options("grok", suffix), provider: "opencode", model: MODEL, effort: "max", ...overrides }; + } + + beforeEach(() => { + for (const name of INHERITED) process.env[name] = `${name}-value`; + }); + afterEach(() => { + for (const name of INHERITED) delete process.env[name]; + }); + + it("runs a read-only lane with the run-owned config, the prompt on stdin and the reply from the last step", async () => { + fakeOpencode(LIST, OBSERVE); + const input = opencodeOptions("opencode-complete"); + const result = await runLane(laneOf(input)); + expect(result.exitCode).toBe(0); + const saved = receipt(input.receiptPath); + expect(saved).toMatchObject({ + status: "complete", + provider: "opencode", + model: MODEL, + effort: "max", + reportedModel: null, + modelVerified: false, + modelEvidence: "pinned-argv", + sessionId: "ses_fixture", + usage: { inputTokens: 10, cachedInputTokens: 2, cacheCreationInputTokens: 0, outputTokens: 4, reasoningTokens: 1, totalTokens: 15 }, + costUsd: 0.75, + preflight: { status: "passed", evidence: `model ${MODEL} listed` }, + }); + const args = ["run", "--pure", "--agent", "build", "--format", "json", "--title", "pstack-lane", "--print-logs", "--log-level", "ERROR", "-m", MODEL, "--variant", "max"]; + expect(saved.argv.slice(1)).toEqual(args); + expect(saved.preflight.argv.slice(1)).toEqual(["models", "opencode"]); + expect(JSON.parse(readFileSync(input.outputPath, "utf8"))).toEqual({ + args, + prompt: "Return the marker.", + xdg: true, + modes: ["700", "600"], + permission: "deny", + opencode: [ + "OPENCODE_CONFIG", + "OPENCODE_DISABLE_AUTOUPDATE", + "OPENCODE_DISABLE_CLAUDE_CODE", + "OPENCODE_DISABLE_EXTERNAL_SKILLS", + "OPENCODE_DISABLE_PROJECT_CONFIG", + "OPENCODE_TEST_HOME", + ], + }); + }); + + it("runs the preflight and the model with one config, environment and cwd", async () => { + fakeOpencode(LIST, OBSERVE); + const input = opencodeOptions("opencode-one-context"); + expect((await runLane(laneOf(input))).receipt.status).toBe("complete"); + expect(readFileSync(join(bin, "opencode.calls"), "utf8")).toBe("preflight\nmodel\n"); + const contexts = readFileSync(join(bin, "opencode.contexts"), "utf8").trim().split("\n").map((line) => JSON.parse(line)); + expect(contexts).toHaveLength(2); + expect(JSON.parse(contexts[0].config).permission["*"]).toBe("deny"); + expect(contexts[1]).toEqual(contexts[0]); + }); + + for (const [name, model, models, status, exitCode] of [ + ["a provider with no login", "openrouter/deepseek/deepseek-v4-pro", `console.error("\\u001b[91m\\u001b[1mError: \\u001b[0mProvider not found: openrouter"); process.exit(1);`, "unauthenticated", 77], + ["an unlisted model", "opencode/deepseek-v4.1-flash", LIST, "unavailable-model", 69], + ] as const) { + it(`classifies ${name} before any model call`, async () => { + const started = join(scratch, "opencode-model.started"); + fakeOpencode(models, `writeFileSync(${JSON.stringify(started)}, "started");`); + const input = opencodeOptions(`opencode-preflight-${exitCode}`, { model, effort: "default" }); + const result = await runLane(laneOf(input)); + expect(result.exitCode).toBe(exitCode); + expect(result.receipt).toMatchObject({ status, preflight: { status: "failed" }, error: { message: "authentication or model preflight failed" } }); + expect(existsSync(started)).toBe(false); + }); + } + + it("classifies a rejected tool call, an error event and SIGTERM from the captured shapes", async () => { + fakeOpencode(LIST, `console.error("! permission requested: edit (created.txt); auto-rejecting"); +emit(${JSON.stringify(event("step_start"))}); +emit(${JSON.stringify(event("step_finish", { reason: "tool-calls" }))});`); + const rejected = opencodeOptions("opencode-rejected"); + const rejectedResult = await runLane(laneOf(rejected)); + expect(rejectedResult.exitCode).toBe(70); + expect(rejectedResult.receipt).toMatchObject({ status: "child-failed", exitCode: 0, error: { message: "opencode ended on a rejected tool call" } }); + expect(rejectedResult.receipt.error?.evidence).toContain("auto-rejecting"); + expect(existsSync(rejected.outputPath)).toBe(false); + + fakeOpencode(LIST, `console.error('level=ERROR message=failed error="ProviderModelNotFoundError: Model not found"'); +emit({ type: "error", timestamp: 1, sessionID: "ses_fixture", error: { name: "UnknownError", data: { message: "Unexpected server error. Check server logs for details." } } }); +process.exit(1);`); + const failed = opencodeOptions("opencode-error"); + const failedResult = await runLane(laneOf(failed)); + expect(failedResult.exitCode).toBe(70); + expect(failedResult.receipt).toMatchObject({ status: "child-failed", exitCode: 1, error: { message: "opencode reported UnknownError: Unexpected server error. Check server logs for details." } }); + expect(failedResult.receipt.error?.evidence).toContain("ProviderModelNotFoundError"); + + fakeOpencode(LIST, `emit(${JSON.stringify(event("step_start"))}); +process.kill(process.pid, "SIGTERM"); +await Bun.sleep(5_000);`); + const terminated = opencodeOptions("opencode-sigterm"); + const terminatedResult = await runLane(laneOf(terminated)); + expect(terminatedResult.exitCode).toBe(130); + expect(terminatedResult.receipt).toMatchObject({ status: "cancelled", exitCode: 143, signal: null }); + expect(existsSync(terminated.outputPath)).toBe(false); + }); + + it("runs in a repository holding .opencode/ and an opencode.json, which the environment turns off", async () => { + fakeOpencode(LIST, OBSERVE); + const repo = join(scratch, "repo"), nested = join(repo, "src"); + mkdirSync(join(repo, ".opencode", "plugins"), { recursive: true }); + mkdirSync(nested); + writeFileSync(join(repo, "opencode.json"), "{\"permission\":{\"bash\":\"allow\"}}\n"); + const input = opencodeOptions("opencode-project-config", { cwd: nested }); + const result = await runLane(laneOf(input)); + expect(result.exitCode).toBe(0); + expect(result.receipt).toMatchObject({ status: "complete", preflight: { status: "passed" } }); + expect(JSON.parse(readFileSync(input.outputPath, "utf8")).permission).toBe("deny"); + }); + + it("refuses an isolated-write OpenCode lane before launch", () => { + const input = opencodeOptions("opencode-writer", { mode: "isolated-write" }); + expect(() => laneOf(input)).toThrow("provider opencode does not support isolated-write"); + expect(existsSync(input.receiptPath)).toBe(false); + }); +}); + +describe("dsh lanes", () => { + const DSH_VARIABLES = ["DSH_PERMISSION_MODE", "DSH_HOME", "DEEPSEEK_BASE_URL", "DEEPSEEK_API_KEY"]; + const TRANSPORT = `{"type":"status","phase":"turn_end","turn":1,"reason":{"kind":"error","error":{"message":"DeepSeek Messages transport failed","code":"TRANSPORT"}}}`; + const MISSING = `{"type":"status","phase":"turn_end","turn":1,"reason":{"kind":"error","error":{"message":"llm-deepseek: no API key","code":"MISSING_CREDENTIAL"}}}`; + + // A dsh stand-in: `--version` prints `version`; the preflight (a positional `preflight` task) prints + // `status`; the model call reads stdin and runs `turn`, which sees `args`, `patch`, `prompt` and `emit`. + // Each call appends its kind to `dsh.calls` next to the stand-in. The child environment keeps no test + // variable, so each case bakes its own. + function fakeDsh(status: string, turn: string, version = "0.2.0-rc.2"): void { + writeFileSync(join(bin, "dsh"), `#!/usr/bin/env bun +import { appendFileSync, readFileSync, statSync, writeFileSync } from "node:fs"; +import { dirname, join } from "node:path"; +const args = process.argv.slice(2); +const call = (kind) => appendFileSync(join(dirname(process.argv[1]), "dsh.calls"), kind + "\\n"); +if (args[0] === "--version") { + call("version"); + console.log(${JSON.stringify(version)}); + process.exit(0); +} +const patch = args[args.indexOf("--patch") + 1]; +const emit = (event) => console.log(JSON.stringify(event)); +if (args.at(-1) === "preflight") { + call("preflight"); + console.log(${JSON.stringify(status)}); + process.exit(1); +} +call("model"); +const prompt = await Bun.stdin.text(); +emit({ type: "session", sessionId: "session-fixture", cwd: process.cwd() }); +${turn} +`); + chmodSync(join(bin, "dsh"), 0o755); + } + + // Reports what the child saw as the final text. + const OBSERVE = `const observed = { + prompt, + patch: [["- id: sandbox-policy", 4], ["- id: credentials", 3], ["- id: agent-default-model", 5]].flatMap(([row, count]) => { + const lines = readFileSync(patch, "utf8").split("\\n"); + return lines.slice(lines.indexOf(row), lines.indexOf(row) + count); + }), + home: process.env.DSH_HOME, + dirMode: (statSync(dirname(patch)).mode & 0o777).toString(8), + patchMode: (statSync(patch).mode & 0o777).toString(8), + variables: Object.keys(process.env).filter((key) => /^(DSH_|DEEPSEEK_)/.test(key)).sort(), +}; +emit({ type: "status", phase: "step_end", turn: 1, step: 1, usage: { inputTokens: 5, outputTokens: 2, cacheReadTokens: 3, totalTokens: 10 } }); +emit({ type: "status", phase: "turn_end", turn: 1, reason: { kind: "completed" } }); +emit({ type: "final", text: JSON.stringify(observed) });`; + + function dshOptions(suffix: string, overrides: Partial<LaneRequest> = {}): LaneRequest { + return { ...options("grok", suffix), provider: "dsh", model: "deepseek-flash", effort: "high", ...overrides }; + } + + beforeEach(() => { + for (const name of DSH_VARIABLES) process.env[name] = `${name}-value`; + }); + afterEach(() => { + for (const name of DSH_VARIABLES) delete process.env[name]; + }); + + it("runs a read-only lane with the run-owned patch, the prompt on stdin and the pinned-argv proof", async () => { + fakeDsh(TRANSPORT, OBSERVE); + const input = dshOptions("dsh-complete"); + const result = await runLane(laneOf(input)); + expect(result.exitCode).toBe(0); + const saved = receipt(input.receiptPath); + expect(saved).toMatchObject({ + status: "complete", + provider: "dsh", + model: "deepseek-flash", + effort: "high", + mode: "read-only", + reportedModel: null, + modelVerified: false, + modelEvidence: "pinned-argv", + sessionId: "session-fixture", + usage: { inputTokens: 5, outputTokens: 2, cachedInputTokens: 3, totalTokens: 10 }, + preflight: { status: "passed", evidence: "credential present; not validated" }, + }); + const dir = `${input.receiptPath}.provider`; + expect(saved.argv.slice(1)).toEqual(["--profile", "headless", "--patch", join(dir, "patch.yml"), "--json", "-"]); + expect(saved.preflight.argv.slice(1, 5)).toEqual(["--profile", "headless", "--patch", join(dir, "patch.yml")]); + expect(saved.preflight.argv.at(-1)).toBe("preflight"); + expect(JSON.parse(readFileSync(input.outputPath, "utf8"))).toEqual({ + prompt: "Return the marker.", + patch: [ + "- id: sandbox-policy", + " config:", + " mode: read-only", + ` workspaceRoot: ${JSON.stringify(scratch)}`, + "- id: credentials", + " config:", + ` path: ${JSON.stringify(join(resolve("DSH_HOME-value"), ".credentials.yaml"))}`, + "- id: agent-default-model", + " config:", + " provider: deepseek-official", + " model: \"deepseek-flash\"", + " reasoningEffort: \"high\"", + ], + dirMode: "700", + patchMode: "600", + home: dir, + variables: ["DEEPSEEK_API_KEY", "DSH_HOME"], + }); + expect(statSync(dir).mode & 0o777).toBe(0o700); + }); + + for (const [name, status, exitCode, message] of [ + ["a missing credential", MISSING, 77, "authentication or model preflight failed"], + ["an unknown status", "Welcome to dsh", 70, "authentication preflight output was inconclusive"], + ] as const) { + it(`classifies ${name} before any model call`, async () => { + const started = join(scratch, "dsh-model.started"); + fakeDsh(status, `writeFileSync(${JSON.stringify(started)}, "started");`); + const input = dshOptions(`dsh-preflight-${exitCode}`); + const result = await runLane(laneOf(input)); + expect(result.exitCode).toBe(exitCode); + expect(result.receipt).toMatchObject({ + status: exitCode === 77 ? "unauthenticated" : "child-failed", + preflight: { status: "failed" }, + error: { message }, + }); + expect(existsSync(started)).toBe(false); + }); + } + + it("never completes SIGTERM's exit 0 with a partial final", async () => { + fakeDsh(TRANSPORT, `emit({ type: "status", phase: "turn_start", turn: 1 }); +emit({ type: "final", text: "partial reply" });`); + const input = dshOptions("dsh-sigterm-shape"); + const result = await runLane(laneOf(input)); + expect(result.exitCode).toBe(65); + expect(result.receipt).toMatchObject({ status: "malformed-output", exitCode: 0, error: { message: "dsh ended without a completed turn" } }); + expect(existsSync(input.outputPath)).toBe(false); + }); + + it("classifies a turn error, an unknown model and exit 130 from the captured shapes", async () => { + fakeDsh(TRANSPORT, `emit({ type: "status", phase: "turn_end", turn: 1, reason: { kind: "error", error: { message: "rate limited", code: "RATE_LIMIT" } } }); +emit({ type: "final", text: "" }); +console.error("dsh: RATE_LIMIT: rate limited"); +process.exit(1);`); + const failed = dshOptions("dsh-turn-error"); + expect((await runLane(laneOf(failed))).receipt).toMatchObject({ + status: "child-failed", + exitCode: 1, + sessionId: "session-fixture", + error: { message: "dsh turn failed: RATE_LIMIT: rate limited" }, + }); + + fakeDsh(TRANSPORT, `emit({ type: "status", phase: "turn_end", turn: 1, reason: { kind: "error", error: { message: "The supported API model names are deepseek-flash, deepseek-v4-pro, but you passed deepseek-next.", code: "INVALID_REQUEST", status: 400 } } }); +console.error("dsh: INVALID_REQUEST: The supported API model names are deepseek-flash, deepseek-v4-pro, but you passed deepseek-next."); +process.exit(1);`); + const unknown = dshOptions("dsh-unknown-model", { model: "deepseek-next" }); + const unknownResult = await runLane(laneOf(unknown)); + expect(unknownResult.exitCode).toBe(69); + expect(unknownResult.receipt).toMatchObject({ status: "unavailable-model", exitCode: 1 }); + + fakeDsh(TRANSPORT, `emit({ type: "final", text: "partial reply" }); +process.exit(130);`); + const interrupted = dshOptions("dsh-sigint"); + expect((await runLane(laneOf(interrupted))).receipt).toMatchObject({ status: "cancelled", exitCode: 130, signal: null }); + expect(existsSync(interrupted.outputPath)).toBe(false); + }); + + it("refuses a dsh other than the measured version before the preflight", async () => { + fakeDsh(TRANSPORT, OBSERVE, "0.3.0"); + const input = dshOptions("dsh-version"); + const result = await runLane(laneOf(input)); + expect(result.exitCode).toBe(69); + expect(result.receipt).toMatchObject({ + status: "unavailable-cli", + preflight: { status: "not-run" }, + error: { + message: "dsh 0.3.0 is not the measured 0.2.0-rc.2; the adapter must be re-measured before it runs this version", + evidence: "0.3.0", + }, + }); + expect(readFileSync(join(bin, "dsh.calls"), "utf8")).toBe("version\n"); + expect(existsSync(input.outputPath)).toBe(false); + }); + + it("checks the measured version, then runs the preflight and the model", async () => { + fakeDsh(TRANSPORT, OBSERVE); + const input = dshOptions("dsh-version-match"); + expect((await runLane(laneOf(input))).receipt.status).toBe("complete"); + expect(readFileSync(join(bin, "dsh.calls"), "utf8")).toBe("version\npreflight\nmodel\n"); + }); + + it("runs a dsh lane whose cwd holds .env", async () => { + fakeDsh(TRANSPORT, OBSERVE); + const cwd = join(scratch, "env-repo"); + fs.mkdirSync(join(cwd, ".git"), { recursive: true }); + writeFileSync(join(cwd, ".env"), "PSTACK_FIXTURE=1\n"); + const input = dshOptions("dsh-cwd-env", { cwd }); + expect((await runLane(laneOf(input))).receipt).toMatchObject({ status: "complete", exitCode: 0 }); + }); + + it("refuses an isolated-write dsh lane before launch", () => { + const input = dshOptions("dsh-writer", { mode: "isolated-write" }); + expect(() => laneOf(input)).toThrow("provider dsh does not support isolated-write"); + expect(existsSync(input.receiptPath)).toBe(false); + }); +}); + +describe("amp lanes", () => { + const SIGNED_IN = "Signed in as <EMAIL>\n**Individual credits:** $<CREDITS> remaining"; + const MISSING_KEY = "Error: Invalid or missing API key. Run 'amp login' to authenticate."; + const AMP_VARIABLES = ["AMP_API_KEY", "AMP_SETTINGS_FILE", "AMP_EXECUTOR", "AMP_INITIAL_AGENT_MODE_KEY", "AMP_URL"]; + + // An Amp stand-in: `usage` exits `usageExit` and prints `usage` (to stderr when it fails); the model call runs + // `turn`, which sees `args`, `at`, `settings` and the stdin `prompt`. The child environment keeps no test + // variable, so each case bakes its own. + function fakeAmp(usage: string, turn: string, usageExit = 0): void { + writeFileSync(join(bin, "amp"), `#!/usr/bin/env bun +import { existsSync, readFileSync, statSync, writeFileSync } from "node:fs"; +import { dirname } from "node:path"; +const args = process.argv.slice(2); +if (args[0] === "usage") { + (${usageExit} === 0 ? console.log : console.error)(${JSON.stringify(usage)}); + process.exit(${usageExit}); +} +const at = (flag) => args[args.indexOf(flag) + 1]; +const settings = at("--settings-file"); +const prompt = await Bun.stdin.text(); +const emit = (event) => console.log(JSON.stringify({ ...event, session_id: "T-fixture" })); +${turn} +`); + chmodSync(join(bin, "amp"), 0o755); + } + + // Reports what the child saw as the result text. + const OBSERVE = `const observed = { + prompt, + mode: at("-m"), + dirMode: (statSync(dirname(settings)).mode & 0o777).toString(8), + settingsMode: (statSync(settings).mode & 0o777).toString(8), + tools: JSON.parse(readFileSync(settings, "utf8"))["amp.tools.enable"], + ampVariables: Object.fromEntries(Object.entries(process.env).filter(([key]) => key.startsWith("AMP_"))), +}; +emit({ type: "system", subtype: "init", tools: ["Read"], agent_mode: "medium" }); +emit({ type: "assistant", message: { role: "assistant", content: [{ type: "text", text: "x" }], usage: { input_tokens: 3, cache_read_input_tokens: 4, output_tokens: 1 } } }); +emit({ type: "assistant", message: { role: "assistant", content: [{ type: "text", text: "y" }], usage: { input_tokens: 2, output_tokens: 5 } } }); +emit({ type: "result", subtype: "success", is_error: false, num_turns: 2, result: JSON.stringify(observed) });`; + + function ampOptions(suffix: string, overrides: Partial<LaneRequest> = {}): LaneRequest { + return { ...options("grok", suffix), provider: "amp", model: "deepseek-v4.1-flash", effort: "default", ...overrides }; + } + + beforeEach(() => { + for (const name of AMP_VARIABLES) process.env[name] = `${name}-value`; + }); + afterEach(() => { + for (const name of AMP_VARIABLES) delete process.env[name]; + }); + + for (const parent of ["claude", "codex"] as const) { + it(`runs amp:deepseek-v4.1-flash from a ${parent} parent with the pinned mode, lockdown settings and prompt on stdin`, async () => { + fakeAmp(SIGNED_IN, OBSERVE); + const input = ampOptions(`amp-${parent}`, { parent }); + const result = await runLane(laneOf(input)); + expect(result.exitCode).toBe(0); + const saved = receipt(input.receiptPath); + expect(saved).toMatchObject({ + status: "complete", + parent, + provider: "amp", + model: "deepseek-v4.1-flash", + effort: "default", + mode: "read-only", + reportedModel: null, + modelVerified: false, + modelEvidence: "pinned-argv", + sessionId: "T-fixture", + usage: { inputTokens: 5, cachedInputTokens: 4, outputTokens: 6 }, + costUsd: null, + preflight: { status: "passed", evidence: "authenticated" }, + }); + const dir = `${input.receiptPath}.provider`; + expect(saved.argv.slice(1)).toEqual([ + "-x", "--stream-json", "-m", "deepseek-v4.1-flash", "--settings-file", join(dir, "settings.json"), + "--visibility", "private", "--no-ide", "--no-notifications", "-l", "pstack-runner", + ]); + expect(saved.preflight.argv.slice(1)).toEqual(["usage"]); + expect(JSON.parse(readFileSync(input.outputPath, "utf8"))).toEqual({ + prompt: "Return the marker.", + mode: "deepseek-v4.1-flash", + dirMode: "700", + settingsMode: "600", + tools: ["Read"], + ampVariables: { AMP_API_KEY: "AMP_API_KEY-value", AMP_IGNORE_GUIDANCE_FILES: "*" }, + }); + expect(statSync(dir).mode & 0o777).toBe(0o700); + }); + } + + for (const [name, usage, usageExit, exitCode, message] of [ + ["a missing key", MISSING_KEY, 1, 77, "authentication or model preflight failed"], + ["a login prompt", "Would you like to log in to Amp? [(y)es, (n)o]", 0, 70, "authentication preflight output was inconclusive"], + ] as const) { + it(`classifies ${name} before any model call`, async () => { + const started = join(scratch, "amp-model.started"); + fakeAmp(usage, `writeFileSync(${JSON.stringify(started)}, "started");`, usageExit); + const input = ampOptions(`amp-preflight-${exitCode}`); + const result = await runLane(laneOf(input)); + expect(result.exitCode).toBe(exitCode); + expect(result.receipt).toMatchObject({ + status: exitCode === 77 ? "unauthenticated" : "child-failed", + preflight: { status: "failed" }, + error: { message }, + }); + expect(existsSync(started)).toBe(false); + }); + } + + it("reads exit-0 error and cancellation results, and an unknown mode's exit 1, as failures", async () => { + fakeAmp(SIGNED_IN, `emit({ type: "result", subtype: "error_during_execution", is_error: true, num_turns: 0, error: "Claude Usage is set to Subscription Only." });`); + const subscription = ampOptions("amp-subscription"); + const subscriptionResult = await runLane(laneOf(subscription)); + expect(subscriptionResult.exitCode).toBe(70); + expect(subscriptionResult.receipt).toMatchObject({ + status: "child-failed", + exitCode: 0, + sessionId: "T-fixture", + error: { message: "Claude Usage is set to Subscription Only." }, + }); + expect(existsSync(subscription.outputPath)).toBe(false); + + fakeAmp(SIGNED_IN, `emit({ type: "result", subtype: "error_during_execution", is_error: true, num_turns: 0, error: "User cancelled (SIGINT/SIGTERM)" });`); + const cancelled = ampOptions("amp-cancelled"); + const cancelledResult = await runLane(laneOf(cancelled)); + expect(cancelledResult.exitCode).toBe(130); + expect(cancelledResult.receipt).toMatchObject({ status: "cancelled", exitCode: 0, signal: null }); + expect(existsSync(cancelled.outputPath)).toBe(false); + + fakeAmp(SIGNED_IN, `console.error("Error: Unexpected error inside Amp CLI. Run with AMP_DEBUG=1 to see detailed logs."); +process.exit(1);`); + const unknown = ampOptions("amp-unknown-mode", { model: "nonexistent-mode-xyz" }); + expect((await runLane(laneOf(unknown))).receipt).toMatchObject({ status: "child-failed", exitCode: 1 }); + }); + + it("runs an Amp lane whose workspace holds .amp", async () => { + fakeAmp(SIGNED_IN, OBSERVE); + const repo = join(scratch, "repo"), nested = join(repo, "src"); + fs.mkdirSync(join(repo, ".amp/plugins"), { recursive: true }); + fs.writeFileSync(join(repo, ".amp/settings.json"), "{}\n"); + fs.mkdirSync(nested); + const input = ampOptions("amp-project-config", { cwd: nested }); + expect((await runLane(laneOf(input))).exitCode).toBe(0); + expect(receipt(input.receiptPath)).toMatchObject({ status: "complete", preflight: { status: "passed" } }); + }); + + it("refuses an isolated-write Amp lane before launch", () => { + const input = ampOptions("amp-writer", { mode: "isolated-write" }); + expect(() => laneOf(input)).toThrow("provider amp does not support isolated-write"); + expect(existsSync(input.receiptPath)).toBe(false); + }); +}); + + +describe("agy lanes", () => { + const MODELS = "gemini-3.8-flash-high\tGemini 3.8 Flash (High)\ngemini-3.8-flash-medium\tGemini 3.8 Flash (Medium)\ngemini-3.8-flash-low\tGemini 3.8 Flash (Low)\n"; + const SIGN_IN = "Fetching available models...\nError: Please sign in to view available models. Launch the CLI without arguments to sign in."; + + // An agy stand-in: `models` prints `models` to stdout, or `signIn` to stderr with exit 1; the model call runs + // `turn`, which sees `args`, `at`, `stdin` and `result(body)`. The child environment keeps no test variable. + function fakeAgy(turn: string, models = MODELS, signIn = false): void { + writeFileSync(join(bin, "agy"), `#!/usr/bin/env bun +import { existsSync, readFileSync, statSync, writeFileSync } from "node:fs"; +import { join } from "node:path"; +const args = process.argv.slice(2); +if (args[0] === "models") { + if (${signIn}) { + console.error(${JSON.stringify(SIGN_IN)}); + process.exit(1); + } + process.stdout.write(${JSON.stringify(models)}); + process.exit(0); +} +const at = (flag) => args[args.indexOf(flag) + 1]; +const stdin = await Bun.stdin.text(); +const result = (body) => console.log(JSON.stringify({ event: "result", result: { conversation_id: "a1", status: "SUCCESS", response: "", usage: { input_tokens: 5, output_tokens: 2, thinking_tokens: 1, cache_read_tokens: 0, total_tokens: 7 }, ...body } })); +${turn} +`); + chmodSync(join(bin, "agy"), 0o755); + } + + // Reports what the child saw: the agent file in --add-dir, its modes, the stdin line and the environment. + const OBSERVE = `const agent = join(at("--add-dir"), ".agents/agents", at("--agent") + ".md"); +if (!existsSync(agent)) { + console.error('error: unknown agent "' + at("--agent") + '"'); + process.exit(1); +} +const mode = (path) => (statSync(path).mode & 0o777).toString(8); +const observed = { + model: at("--model"), + stdin: JSON.parse(stdin), + stdinLines: stdin.split("\\n").length - 1, + tools: readFileSync(agent, "utf8").split("tools:\\n")[1].split("\\n")[0], + modes: [mode(at("--add-dir")), mode(agent)], + geminiVariables: Object.keys(process.env).filter((key) => key.startsWith("GEMINI_")), +}; +console.log(JSON.stringify({ event: "init", init: { model: at("--model") } })); +result({ response: JSON.stringify(observed) + "\\n" });`; + + function agyOptions(suffix: string, overrides: Partial<LaneRequest> = {}): LaneRequest { + return { ...options("grok", suffix), provider: "agy", model: "gemini-3.8-flash", effort: "high", ...overrides }; + } + + beforeEach(() => { + process.env.GEMINI_API_KEY = "GEMINI_API_KEY-value"; + }); + afterEach(() => { + delete process.env.GEMINI_API_KEY; + }); + + for (const effort of ["low", "medium", "high"] as const) { + it(`runs agy:gemini-3.8-flash@${effort} with the envelope on stdin and the run-owned agent, on pinned argv`, async () => { + fakeAgy(OBSERVE); + const input = agyOptions(`agy-${effort}`, { effort }); + writeFileSync(input.promptPath, "Read \"marker.txt\".\nReply with its contents.\n"); + const result = await runLane(laneOf(input)); + expect(result.exitCode).toBe(0); + const saved = receipt(input.receiptPath); + const cliModel = `gemini-3.8-flash-${effort}`; + expect(saved).toMatchObject({ + status: "complete", + provider: "agy", + model: "gemini-3.8-flash", + effort, + mode: "read-only", + reportedModel: null, + modelVerified: false, + modelEvidence: "pinned-argv", + sessionId: "a1", + usage: { inputTokens: 5, cachedInputTokens: 0, outputTokens: 2, reasoningTokens: 1, totalTokens: 7 }, + preflight: { argv: [saved.executable, "models"], status: "passed", evidence: `model ${cliModel} listed` }, + }); + const dir = `${input.receiptPath}.provider`; + expect(saved.argv.slice(1)).toEqual([ + "--input-format", "stream-json", "--output-format", "stream-json", "--model", cliModel, + "--add-dir", dir, "--agent", "pstack-read-only", + ]); + expect(JSON.parse(readFileSync(input.outputPath, "utf8"))).toEqual({ + model: cliModel, + stdin: { event: "user", message: { role: "user", content: "Read \"marker.txt\".\nReply with its contents.\n" } }, + stdinLines: 1, + tools: " - view_file", + modes: ["700", "600"], + geminiVariables: [], + }); + }); + } + + for (const [name, signIn, models, status, exitCode] of [ + ["a sign-in failure", true, MODELS, "unauthenticated", 77], + ["an unlisted variant", false, "gemini-3.8-flash-low\tGemini 3.8 Flash (Low)\n", "unavailable-model", 69], + ] as const) { + it(`classifies ${name} before any model call`, async () => { + const started = join(scratch, "agy-model.started"); + fakeAgy(`writeFileSync(${JSON.stringify(started)}, "started");`, models, signIn); + const input = agyOptions(`agy-preflight-${exitCode}`); + const result = await runLane(laneOf(input)); + expect(result.exitCode).toBe(exitCode); + expect(result.receipt).toMatchObject({ status, preflight: { status: "failed" } }); + expect(existsSync(started)).toBe(false); + }); + } + + it("classifies an exit-0 denial, an unknown model, an interruption and a missing agent from the captured shapes", async () => { + fakeAgy(`console.error('jetski: no output produced — a tool required the "command" permission that headless mode cannot prompt for, so it was auto-denied.'); +result({ denied_actions: [{ action: "command", display_name: "RunCommand" }] });`); + const denied = agyOptions("agy-denied"); + const deniedResult = await runLane(laneOf(denied)); + expect(deniedResult.exitCode).toBe(70); + expect(deniedResult.receipt).toMatchObject({ + status: "child-failed", + exitCode: 0, + sessionId: "a1", + error: { message: "agy denied a tool that needed approval" }, + }); + expect(deniedResult.receipt.error?.evidence).toContain("auto-denied"); + expect(existsSync(denied.outputPath)).toBe(false); + + fakeAgy(`console.error('error: invalid model selection (--model "' + at("--model") + '" --effort ""): model is not recognized as a known model'); +result({ conversation_id: "", status: "ERROR", error: 'invalid model selection (--model "' + at("--model") + '")' }); +process.exit(1);`); + const unknown = agyOptions("agy-unknown-model"); + expect((await runLane(laneOf(unknown))).receipt).toMatchObject({ status: "unavailable-model", exitCode: 1, sessionId: null }); + + fakeAgy(`console.error("error: interrupted"); +result({ status: "ERROR", error: "interrupted" }); +process.exit(1);`); + const interrupted = agyOptions("agy-interrupted"); + const interruptedResult = await runLane(laneOf(interrupted)); + expect(interruptedResult.exitCode).toBe(130); + expect(interruptedResult.receipt).toMatchObject({ status: "cancelled", exitCode: 1, error: { message: "agy reported the turn was interrupted" } }); + + // The stand-in fails closed like agy when the run-owned agent file is gone. + fakeAgy(`import { rmSync } from "node:fs"; +rmSync(join(at("--add-dir"), ".agents"), { recursive: true }); +${OBSERVE}`); + const missing = agyOptions("agy-missing-agent"); + expect((await runLane(laneOf(missing))).receipt).toMatchObject({ status: "child-failed", exitCode: 1 }); + expect(existsSync(missing.outputPath)).toBe(false); + }); + + it("runs an agy lane in a repository that holds .agents/", async () => { + fakeAgy(OBSERVE); + const repo = join(scratch, "repo"); + fs.mkdirSync(join(repo, ".git"), { recursive: true }); + fs.mkdirSync(join(repo, ".agents")); + const input = agyOptions("agy-repo-config", { cwd: repo }); + expect((await runLane(laneOf(input))).receipt).toMatchObject({ status: "complete", exitCode: 0 }); + }); + + it("refuses an isolated-write agy lane before launch", () => { + const input = agyOptions("agy-writer", { mode: "isolated-write" }); + expect(() => laneOf(input)).toThrow("provider agy does not support isolated-write"); + expect(existsSync(input.receiptPath)).toBe(false); + }); +}); + +describe("cursor lanes", () => { + const MODELS = "Available models\n\nauto - Auto (current, default)\ncomposer-2.5 - Composer 2.5\ngemini-3.7-flash-high - Gemini 3.7 Flash\n"; + const AUTH_REQUIRED = "Error: Authentication required. Run 'agent login', pass --api-key/--auth-token, or set CURSOR_API_KEY/CURSOR_AUTH_TOKEN."; + const CURSOR_VARIABLES = ["CURSOR_API_KEY", "CURSOR_AUTH_TOKEN", "CURSOR_API_ENDPOINT", "CURSOR_SANDBOX_POLICY_DIR", "CURSOR_CONFIG_DIR"]; + + // A cursor-agent stand-in: `models` prints `models` to stdout, or the authentication error to stderr with exit 1; + // the model call runs `turn`, which sees `args`, `at`, the stdin `prompt` and `emit(event)`. The child environment + // keeps no test variable, so each case bakes its own. + function fakeCursor(turn: string, models = MODELS, loggedOut = false): void { + writeFileSync(join(bin, "cursor-agent"), `#!/usr/bin/env bun +import { readFileSync, statSync, writeFileSync } from "node:fs"; +import { join } from "node:path"; +const args = process.argv.slice(2); +if (args[0] === "models") { + if (${loggedOut}) { + console.error(${JSON.stringify(AUTH_REQUIRED)}); + process.exit(1); + } + process.stdout.write(${JSON.stringify(models)}); + process.exit(0); +} +const at = (flag) => args[args.indexOf(flag) + 1]; +const prompt = await Bun.stdin.text(); +const emit = (event) => console.log(JSON.stringify({ ...event, session_id: "c1" })); +${turn} +`); + chmodSync(join(bin, "cursor-agent"), 0o755); + } + + // Reports what the child saw as the last assistant text, after an earlier segment the result joins. + const OBSERVE = `const dir = process.env.CURSOR_CONFIG_DIR; +const config = join(dir, "cli-config.json"); +const observed = { + prompt, + model: at("--model"), + modes: [(statSync(dir).mode & 0o777).toString(8), (statSync(config).mode & 0o777).toString(8)], + deny: JSON.parse(readFileSync(config, "utf8")).permissions.deny, + cursorVariables: Object.keys(process.env).filter((key) => key.startsWith("CURSOR_")).sort(), +}; +emit({ type: "system", subtype: "init", model: "Composer 2.5" }); +emit({ type: "assistant", message: { role: "assistant", content: [{ type: "text", text: "Reading the file.\\n" }] } }); +emit({ type: "assistant", message: { role: "assistant", content: [{ type: "text", text: JSON.stringify(observed) }] } }); +emit({ type: "result", subtype: "success", is_error: false, result: "Reading the file.\\n" + JSON.stringify(observed), usage: { inputTokens: 6, outputTokens: 2, cacheReadTokens: 4, cacheWriteTokens: 3 } });`; + + function cursorOptions(suffix: string, overrides: Partial<LaneRequest> = {}): LaneRequest { + return { ...options("grok", suffix), provider: "cursor", model: "composer-2.5", effort: "default", ...overrides }; + } + + beforeEach(() => { + for (const name of CURSOR_VARIABLES) process.env[name] = `${name}-value`; + }); + afterEach(() => { + for (const name of CURSOR_VARIABLES) delete process.env[name]; + }); + + for (const [parent, model, effort, cliModel] of [ + ["claude", "composer-2.5", "default", "composer-2.5"], + ["codex", "gemini-3.7-flash", "high", "gemini-3.7-flash-high"], + ] as const) { + it(`runs cursor:${model}@${effort} from a ${parent} parent in the run-owned config directory, on pinned argv`, async () => { + fakeCursor(OBSERVE); + const input = cursorOptions(`cursor-${parent}`, { parent, model, effort }); + const result = await runLane(laneOf(input)); + expect(result.exitCode).toBe(0); + const saved = receipt(input.receiptPath); + expect(saved).toMatchObject({ + status: "complete", + parent, + provider: "cursor", + model, + effort, + mode: "read-only", + reportedModel: null, + modelVerified: false, + modelEvidence: "pinned-argv", + sessionId: "c1", + usage: { inputTokens: 6, cachedInputTokens: 4, cacheCreationInputTokens: 3, outputTokens: 2 }, + costUsd: null, + preflight: { argv: [saved.executable, "models"], status: "passed", evidence: `model ${cliModel} listed` }, + }); + expect(saved.argv.slice(1)).toEqual([ + "-p", "--output-format", "stream-json", "--model", cliModel, "--mode", "ask", "--sandbox", "enabled", "--trust", + ]); + expect(JSON.parse(readFileSync(input.outputPath, "utf8"))).toEqual({ + prompt: "Return the marker.", + model: cliModel, + modes: ["700", "600"], + deny: ["Write(**)", "Shell(*)", "Mcp(*:*)"], + cursorVariables: ["CURSOR_API_KEY", "CURSOR_AUTH_TOKEN", "CURSOR_CONFIG_DIR"], + }); + }); + } + + it("points CURSOR_CONFIG_DIR at the lane's provider directory, replacing an inherited value", async () => { + fakeCursor(`emit({ type: "assistant", message: { content: [{ type: "text", text: process.env.CURSOR_CONFIG_DIR }] } }); +emit({ type: "result", subtype: "success", is_error: false });`); + const input = cursorOptions("cursor-config-dir"); + await runLane(laneOf(input)); + expect(readFileSync(input.outputPath, "utf8")).toBe(`${input.receiptPath}.provider`); + }); + + for (const [name, loggedOut, models, status, exitCode] of [ + ["an authentication failure", true, MODELS, "unauthenticated", 77], + ["an unlisted model", false, "Available models\n\ncomposer-2.5-fast - Composer 2.5 Fast\n", "unavailable-model", 69], + ["empty model-list output", false, "", "child-failed", 70], + ] as const) { + it(`classifies ${name} before any model call`, async () => { + const started = join(scratch, "cursor-model.started"); + fakeCursor(`writeFileSync(${JSON.stringify(started)}, "started");`, models, loggedOut); + const input = cursorOptions(`cursor-preflight-${exitCode}`); + const result = await runLane(laneOf(input)); + expect(result.exitCode).toBe(exitCode); + expect(result.receipt).toMatchObject({ status, preflight: { status: "failed" } }); + expect(existsSync(started)).toBe(false); + }); + } + + it("reads an exit-0 error result, an unknown model and a signal exit as failures", async () => { + fakeCursor(`emit({ type: "result", subtype: "error", is_error: true, result: "Usage limit reached." });`); + const failed = cursorOptions("cursor-error-result"); + const failedResult = await runLane(laneOf(failed)); + expect(failedResult.exitCode).toBe(70); + expect(failedResult.receipt).toMatchObject({ status: "child-failed", exitCode: 0, sessionId: "c1", error: { message: "Usage limit reached." } }); + expect(existsSync(failed.outputPath)).toBe(false); + + fakeCursor(`console.error("Cannot use this model: " + at("--model") + ". Available models: auto, composer-2.5"); +process.exit(1);`, `${MODELS}claude-opus-9-9-max - Listed, then refused by the turn\n`); + const unknown = cursorOptions("cursor-unknown-model", { model: "claude-opus-9-9-max" }); + expect((await runLane(laneOf(unknown))).receipt).toMatchObject({ status: "unavailable-model", exitCode: 1 }); + + fakeCursor(`emit({ type: "system", subtype: "init", model: "Composer 2.5" }); +console.error("\\nAborting operation..."); +process.exit(130);`); + const interrupted = cursorOptions("cursor-sigint"); + expect((await runLane(laneOf(interrupted))).receipt).toMatchObject({ status: "child-failed", exitCode: 130 }); + expect(existsSync(interrupted.outputPath)).toBe(false); + }); + + it("runs a Cursor lane in a workspace holding .cursor/ and .claude settings files", async () => { + const repo = join(scratch, "repo"), nested = join(repo, "src"); + mkdirSync(join(repo, ".cursor"), { recursive: true }); + mkdirSync(join(repo, ".claude")); + writeFileSync(join(repo, ".claude/settings.json"), "{}"); + writeFileSync(join(repo, ".claude/settings.local.json"), "{}"); + mkdirSync(nested); + fakeCursor(OBSERVE); + const input = cursorOptions("cursor-project-config", { cwd: nested }); + expect((await runLane(laneOf(input))).exitCode).toBe(0); + expect(receipt(input.receiptPath)).toMatchObject({ status: "complete", preflight: { status: "passed" } }); + }); + + it("refuses an isolated-write Cursor lane before launch", () => { + const input = cursorOptions("cursor-writer", { mode: "isolated-write" }); + expect(() => laneOf(input)).toThrow("provider cursor does not support isolated-write"); + expect(existsSync(input.receiptPath)).toBe(false); + }); +}); diff --git a/plugins/pstack/skills/poteto-mode/scripts/runner/run.ts b/plugins/pstack/skills/poteto-mode/scripts/runner/run.ts index 8a2cc460..ce3486d7 100644 --- a/plugins/pstack/skills/poteto-mode/scripts/runner/run.ts +++ b/plugins/pstack/skills/poteto-mode/scripts/runner/run.ts @@ -5,12 +5,13 @@ import { openSync, readFileSync, realpathSync, + rmSync, statSync, unlinkSync, writeFileSync, writeSync, } from "node:fs"; -import { dirname, isAbsolute, relative, resolve, sep } from "node:path"; +import { dirname, isAbsolute, join, relative, resolve, sep } from "node:path"; import type { Lane, LaunchFacts, ParseResult } from "./registry/index.ts"; import { ERROR_EVIDENCE_LIMIT, @@ -72,15 +73,37 @@ function modelStreamPaths(receiptPath: string): { stdoutPath: string; stderrPath return { stdoutPath: `${receiptPath}.stdout`, stderrPath: `${receiptPath}.stderr` }; } +interface ReservedSidecar { + readonly facts: NonNullable<LaunchFacts["providerSidecar"]>; + readonly configContent: string | null; +} + +// The provider sidecar directory sits next to the receipt, like the stream files. +function providerSidecar(lane: Lane): ReservedSidecar | null { + const declared = lane.adapter.launch.sidecar; + if (declared === null) return null; + const directory = `${lane.receiptPath}.provider`; + return { + facts: { + directory, + output: declared.output === null ? null : join(directory, declared.output), + config: declared.config === null ? null : join(directory, declared.config.name), + }, + configContent: declared.config?.content(lane) ?? null, + }; +} + function reserveOutputs(lane: Lane): ModelStreams { const { stdoutPath, stderrPath } = modelStreamPaths(lane.receiptPath); - const paths = [lane.outputPath, lane.receiptPath, stdoutPath, stderrPath]; + const sidecar = providerSidecar(lane); + const paths = [lane.outputPath, lane.receiptPath, stdoutPath, stderrPath, ...(sidecar === null ? [] : [sidecar.facts.directory])]; const resolved = paths.map((path) => resolve(path)); if (new Set(resolved).size !== paths.length || resolved.includes(resolve(lane.promptPath))) { throw new UsageError("prompt, output, receipt, and stream paths must be distinct"); } const created: string[] = []; let stdout: number | null = null; + let stderr: number | null = null; try { for (const path of paths.slice(0, 2)) { reserve(path); @@ -88,15 +111,30 @@ function reserveOutputs(lane: Lane): ModelStreams { } stdout = openSync(stdoutPath, "wx", 0o600); created.push(stdoutPath); - const stderr = openSync(stderrPath, "wx", 0o600); + stderr = openSync(stderrPath, "wx", 0o600); + created.push(stderrPath); + if (sidecar !== null) { + // Not recursive, so an existing directory refuses the lane like any other reserved path. + mkdirSync(sidecar.facts.directory, { mode: 0o700 }); + created.push(sidecar.facts.directory); + if (sidecar.facts.config !== null && sidecar.configContent !== null) { + mkdirSync(dirname(sidecar.facts.config), { recursive: true, mode: 0o700 }); + writeFileSync(sidecar.facts.config, sidecar.configContent, { encoding: "utf8", mode: 0o600, flag: "wx" }); + } + } return { stdout, stderr }; } catch (error) { if (stdout !== null) closeSync(stdout); - for (const path of created) removeIfExists(path); + if (stderr !== null) closeSync(stderr); + for (const path of created) rmSync(path, { recursive: true, force: true }); throw error; } } +function readIfPresent(path: string | null | undefined): string | null { + return path != null && existsSync(path) ? readFileSync(path, "utf8") : null; +} + function writeReceipt(path: string, receipt: RunnerReceipt): void { writeFileSync(path, `${JSON.stringify(receipt, null, 2)}\n`, { encoding: "utf8", @@ -134,17 +172,20 @@ function installRunCancellation(): RunCancellation { }; } -// The adapter's policy applies first; the parent's identity markers are always dropped, so the -// child cannot detect a nested harness. +// The adapter's policy applies first, then its run-owned values; the parent's identity markers are +// always dropped last, so the child cannot detect a nested harness. export function childEnvironment( lane: Lane, source: NodeJS.ProcessEnv = process.env ): NodeJS.ProcessEnv { - const policy = lane.adapter.environment(lane.access); + const policy = lane.adapter.environment(lane.access, providerSidecar(lane)?.facts.directory ?? null); const keep = policy.kind === "allowlist" ? (key: string) => policy.names.includes(key) || policy.prefixes.some((prefix) => key.startsWith(prefix)) : (key: string) => !policy.dropPrefixes.some((prefix) => key.startsWith(prefix)); - const result = Object.fromEntries(Object.entries(source).filter(([key]) => keep(key))); + const result = { + ...Object.fromEntries(Object.entries(source).filter(([key]) => keep(key))), + ...(policy.kind === "allowlist" ? policy.set : undefined), + }; for (const key of PARENTS[lane.parent].identity) delete result[key]; return result; } @@ -541,7 +582,7 @@ function modelProof( function completeReceipt( lane: Lane, - partial: Omit<RunnerReceipt, "schemaVersion" | "parent" | "provider" | "model" | "effort" | "mode" | "cwd" | "promptPath" | "outputPath" | "stdoutPath" | "stderrPath"> + partial: Omit<RunnerReceipt, "schemaVersion" | "parent" | "provider" | "model" | "effort" | "mode" | "writerSandbox" | "cwd" | "promptPath" | "outputPath" | "stdoutPath" | "stderrPath"> ): RunnerReceipt { return { schemaVersion: 1, @@ -550,6 +591,7 @@ function completeReceipt( model: lane.model, effort: lane.effort, mode: lane.access.mode, + writerSandbox: lane.access.mode === "read-only" ? null : lane.access.sandbox, cwd: lane.cwd, promptPath: lane.promptPath, outputPath: lane.outputPath, @@ -710,13 +752,46 @@ async function executeLane( writeReceipt(lane.receiptPath, receipt); return { exitCode: statusExitCode(discovery.status), receipt }; } - const facts: LaunchFacts = { sharedGitDir: discovery.sharedGitDir }; + const facts: LaunchFacts = { sharedGitDir: discovery.sharedGitDir, providerSidecar: providerSidecar(lane)?.facts ?? null }; launchArgs = adapter.launch.argv(lane, facts); progress.argv = [executable, ...launchArgs]; } const preflight = adapter.preflight; - const preflightArgs = preflight.argv(lane); + if (preflight.guard !== undefined) { + const run = await runProcess(executable, preflight.guard.argv, lane.cwd, env, null, deadlineAt, cancellation); + if (run.cancelledBy !== null) return finishWithoutChild("cancelled", "during the pre-launch guard"); + if (run.timedOut) return finishWithoutChild("timed-out", "during the pre-launch guard"); + const refusal = preflight.guard.check(run); + if (refusal !== null) { + const completed = Date.now(); + receipt = completeReceipt(lane, { + status: refusal.status, + startedAt, + completedAt: new Date(completed).toISOString(), + elapsedMs: completed - started, + executable, + preflight: preflightState, + argv: [executable, ...launchArgs], + exitCode: run.exitCode, + signal: run.signal, + reportedModel: null, + modelVerified: false, + modelEvidence: null, + sessionId: null, + usage: null, + costUsd: null, + error: { + message: refusal.message, + evidence: evidence(`${run.stdout}\n${run.stderr}`), + }, + }); + removeIfExists(lane.outputPath); + writeReceipt(lane.receiptPath, receipt); + return { exitCode: statusExitCode(receipt.status), receipt }; + } + } + const preflightArgs = preflight.argv(lane, providerSidecar(lane)?.facts ?? null); const runPreflight = async () => { const run = await runProcess(executable, preflightArgs, lane.cwd, env, null, deadlineAt, cancellation); const verdict = preflight.classify(run, lane); @@ -785,7 +860,9 @@ async function executeLane( ? "timed-out" : attempt.verdict.kind === "unavailable-model" ? "unavailable-model" - : "unauthenticated"; + : attempt.verdict.kind === "inconclusive" + ? "child-failed" + : "unauthenticated"; receipt = completeReceipt(lane, { status, startedAt, @@ -807,7 +884,9 @@ async function executeLane( ? `launcher received ${preflightResult.cancelledBy} during preflight` : preflightResult.timedOut ? "authentication preflight timed out" - : "authentication or model preflight failed", + : attempt.verdict.kind === "inconclusive" + ? "authentication preflight output was inconclusive" + : "authentication or model preflight failed", evidence: preflightEvidence, }, }); @@ -828,7 +907,7 @@ async function executeLane( launchArgs, lane.cwd, env, - adapter.launch.prompt === "stdin" ? prompt : null, + adapter.launch.prompt === "stdin" ? adapter.launch.stdinEnvelope?.(prompt) ?? prompt : null, deadlineAt, cancellation, streamFiles @@ -846,7 +925,7 @@ async function executeLane( } as const; const parsed: ParseResult | null = result.cancelledBy === null && !result.timedOut - ? adapter.parse(result, lane) + ? adapter.parse({ ...result, sidecar: readIfPresent(providerSidecar(lane)?.facts.output) }, lane) : null; const providerFailure = parsed?.kind === "provider-failure" ? parsed : null; const metadata = providerFailure?.metadata @@ -941,11 +1020,11 @@ export async function runLane( ): Promise<RunResult> { validateLane(lane); const deadlineAt = lane.timeoutMs === null ? null : started + lane.timeoutMs; - const launchArgs = lane.adapter.launch.argv(lane, { sharedGitDir: null }); + const launchArgs = lane.adapter.launch.argv(lane, { sharedGitDir: null, providerSidecar: providerSidecar(lane)?.facts ?? null }); const progress: LaneProgress = { executable: null, preflight: { - argv: [lane.adapter.executable, ...lane.adapter.preflight.argv(lane)], + argv: [lane.adapter.executable, ...lane.adapter.preflight.argv(lane, providerSidecar(lane)?.facts ?? null)], status: "not-run", evidence: "", }, diff --git a/plugins/pstack/skills/poteto-mode/scripts/runner/types.ts b/plugins/pstack/skills/poteto-mode/scripts/runner/types.ts index 18d37dbd..0c838992 100644 --- a/plugins/pstack/skills/poteto-mode/scripts/runner/types.ts +++ b/plugins/pstack/skills/poteto-mode/scripts/runner/types.ts @@ -27,10 +27,16 @@ export const PARENTS = { /** `default` passes no effort, so the CLI or model uses its own setting. */ export const EFFORTS = ["low", "medium", "high", "xhigh", "max", "ultra", "default"] as const; export const ACCESS_MODES = ["read-only", "isolated-write"] as const; +/** A writer's sandbox. `full-access` is an explicit opt-in from the model sheet, never a recovery step. */ +export const WRITER_SANDBOXES = ["bounded", "full-access"] as const; +/** What an adapter's `launch.supports` lists: each access mode, and the full-access writer as its own capability. */ +export const ACCESS_KINDS = [...ACCESS_MODES, "isolated-write-full-access"] as const; export type Parent = keyof typeof PARENTS; export type Effort = (typeof EFFORTS)[number]; export type AccessMode = (typeof ACCESS_MODES)[number]; +export type WriterSandbox = (typeof WRITER_SANDBOXES)[number]; +export type AccessKind = (typeof ACCESS_KINDS)[number]; export type ReceiptStatus = | "complete" @@ -59,6 +65,8 @@ export interface RunnerReceipt { readonly model: string; readonly effort: Effort; readonly mode: AccessMode; + /** Null for a read-only lane. */ + readonly writerSandbox: WriterSandbox | null; readonly cwd: string; readonly promptPath: string; readonly outputPath: string; @@ -98,6 +106,9 @@ export function evidence(value: string): string { export function modelFailureStatus(value: string): "unavailable-model" | "child-failed" { return /model.{0,40}(not found|unknown|unavailable|unsupported|not supported|invalid)|invalid.{0,20}model/i.test(value) + || /^Error: Unknown model: /m.test(value) // Devin CLI 3000.10.21 + || /^Cannot use this model: /m.test(value) // Cursor CLI 2026.10.14-0f9edde + || /^dsh: INVALID_REQUEST: The supported API model names are .+, but you passed /m.test(value) // dsh 0.2.0-rc.2 ? "unavailable-model" : "child-failed"; } diff --git a/plugins/pstack/skills/poteto-mode/scripts/watch-pr/github.test.ts b/plugins/pstack/skills/poteto-mode/scripts/watch-pr/github.test.ts index 4ef39d2a..54722a94 100644 --- a/plugins/pstack/skills/poteto-mode/scripts/watch-pr/github.test.ts +++ b/plugins/pstack/skills/poteto-mode/scripts/watch-pr/github.test.ts @@ -5,6 +5,7 @@ import { buildReviewThreads, mapRollupNode, orderStack, + parseFastCheck, parsePullRequest, parseReviewThreadPage, resolveChecks, @@ -121,6 +122,37 @@ describe("rollup node mapping", () => { ).toMatchObject({ kind: "code-review-gate" }); }); + it("classifies a Code Review Gate waiting on an owner approval as the gate", () => { + expect( + mapRollupNode({ + __typename: "CheckRun", + name: "Code Review Gate", + status: "COMPLETED", + conclusion: "ACTION_REQUIRED", + description: "1 of 2 policies", + }) + ).toMatchObject({ + kind: "code-review-gate", + reportedState: "ACTION_REQUIRED", + }); + }); + + it("classifies a fast-path Code Review Gate waiting on an owner approval as the gate", () => { + expect( + parseFastCheck({ + name: "Code Review Gate", + state: "ACTION_REQUIRED", + bucket: "fail", + }) + ).toMatchObject({ + kind: "code-review-gate", + reportedState: "ACTION_REQUIRED", + }); + expect( + parseFastCheck({ name: "ci", state: "ACTION_REQUIRED", bucket: "fail" }) + ).toMatchObject({ kind: "failed", reportedState: "ACTION_REQUIRED" }); + }); + it("maps StatusContext states and drops unknown typenames", () => { expect( mapRollupNode({ diff --git a/plugins/pstack/skills/poteto-mode/scripts/watch-pr/github.ts b/plugins/pstack/skills/poteto-mode/scripts/watch-pr/github.ts index cfc3764a..4828b029 100644 --- a/plugins/pstack/skills/poteto-mode/scripts/watch-pr/github.ts +++ b/plugins/pstack/skills/poteto-mode/scripts/watch-pr/github.ts @@ -237,22 +237,21 @@ export function parseFastCheck(value: unknown): T.Check { const details = checkDetails(object, "name"); const state = string(object.state, "check.state").toUpperCase(); const bucket = string(object.bucket, "check.bucket"); - if ( - bucket === "fail" || - ["FAILURE", "ERROR", "ACTION_REQUIRED"].includes(state) - ) + if (state === "ACTION_REQUIRED") return gateOr("failed", details, state); + if (bucket === "fail" || ["FAILURE", "ERROR"].includes(state)) return { ...details, kind: "failed", reportedState: state }; - if (bucket === "pending") return pendingOrGate(details, state); + if (bucket === "pending") return gateOr("pending", details, state); if (bucket === "pass") return { ...details, kind: "passed", reportedState: state }; if (bucket === "skipping") return { ...details, kind: "skipped", reportedState: state }; return { ...details, kind: "failed", reportedState: state }; } -// The owner-approval gate is excluded from pending everywhere, so the rule has -// one home. Classifying it as pending on either read path makes the watcher -// wait on a human, which is the behaviour #172004 removed from the Python. -function pendingOrGate( +// Every read path classifies the owner-approval gate here, so the rule has one +// home. A gate that is pending or waits on an owner approval must not count as +// pending or failed, or the watcher waits on a human. +function gateOr( + kind: "pending" | "failed", details: { readonly name: string; readonly description: string; @@ -268,7 +267,7 @@ function pendingOrGate( name: "Code Review Gate", reportedState, } - : { ...details, kind: "pending", reportedState }; + : { ...details, kind, reportedState }; } export function mapRollupNode(value: unknown): T.Check | null { const object = record(value, "rollup node"); @@ -288,22 +287,19 @@ export function mapRollupNode(value: unknown): T.Check | null { ? object.conclusion.toUpperCase() : ""; if (status !== "COMPLETED") - return pendingOrGate({ ...details, link }, "PENDING"); + return gateOr("pending", { ...details, link }, "PENDING"); if (conclusion === "SUCCESS") return { ...details, link, kind: "passed", reportedState: "SUCCESS" }; if (conclusion === "NEUTRAL" || conclusion === "SKIPPED") return { ...details, link, kind: "skipped", reportedState: conclusion }; - return { - ...details, - link, - kind: "failed", - reportedState: conclusion === "ACTION_REQUIRED" ? conclusion : "FAILURE", - }; + if (conclusion === "ACTION_REQUIRED") + return gateOr("failed", { ...details, link }, conclusion); + return { ...details, link, kind: "failed", reportedState: "FAILURE" }; } const state = typeof object.state === "string" ? object.state.toUpperCase() : ""; if (state === "PENDING" || state === "EXPECTED") - return pendingOrGate({ ...details, link }, "PENDING"); + return gateOr("pending", { ...details, link }, "PENDING"); return state === "SUCCESS" ? { ...details, link, kind: "passed", reportedState: state } : { ...details, link, kind: "failed", reportedState: state || "FAILURE" }; diff --git a/plugins/pstack/skills/poteto-mode/scripts/worktree-audit.sh b/plugins/pstack/skills/poteto-mode/scripts/worktree-audit.sh index 2cb8bced..be41ad28 100755 --- a/plugins/pstack/skills/poteto-mode/scripts/worktree-audit.sh +++ b/plugins/pstack/skills/poteto-mode/scripts/worktree-audit.sh @@ -47,10 +47,19 @@ origin/*) git fetch origin "+refs/heads/${base#origin/}:refs/remotes/$base" --qu esac git rev-parse --verify --quiet "$base^{commit}" >/dev/null 2>&1 || { echo "error: base ref not found: $base" >&2; exit 1; } -# PR state by branch, fetched once. Empty if gh is unavailable. +# PR state by branch, fetched once as [{number, state, head}] with a lowercase +# state. Origin when its CLI can read this repo's PRs, otherwise gh. Empty when +# neither can. prs=$(mktemp) -gh pr list --author "@me" --state all --limit 1000 \ - --json number,state,headRefName 2>/dev/null > "$prs" || echo "[]" > "$prs" +if command -v origin >/dev/null 2>&1 \ + && raw=$(origin pr list --author "@me" --state all --limit 1000 \ + --json number,status,headRef 2>/dev/null); then + printf '%s' "$raw" | jq '[.[] | {number, state: .status, head: .headRef}]' > "$prs" 2>/dev/null +elif raw=$(gh pr list --author "@me" --state all --limit 1000 \ + --json number,state,headRefName 2>/dev/null); then + printf '%s' "$raw" | jq '[.[] | {number, state: (.state | ascii_downcase), head: .headRefName}]' > "$prs" 2>/dev/null +fi +[ -s "$prs" ] || echo "[]" > "$prs" # jq reads that JSON and rg finds the transcripts. Without either, the PR and # LAST_CHAT columns blank out and no row can reach verify-recent-chat, so an @@ -95,7 +104,7 @@ git worktree list --porcelain | awk '/^worktree /{print $2}' | while read -r wt; else remote=no-remote; fi pr=$([ -n "$branch" ] && jq -r --arg b "$branch" \ - '.[] | select(.headRefName==$b) | "#\(.number)/\(.state)"' "$prs" 2>/dev/null | head -1) + '.[] | select(.head==$b) | "#\(.number)/\(.state)"' "$prs" 2>/dev/null | head -1) [ -z "$pr" ] && pr="-" # Most recent chat whose transcript operated in this worktree. Match path @@ -110,7 +119,7 @@ git worktree list --porcelain | awk '/^worktree /{print $2}' | while read -r wt; recent=$([ "$last_ts" -gt 0 ] 2>/dev/null && [ $(( (now - last_ts) / 86400 )) -le 4 ] && echo yes || echo no) case "$dirty" in wip:*) bucket=hold-wip ;; *) - case "$pr" in *OPEN*) bucket=hold-open-pr ;; *) + case "$pr" in */open|*/draft) bucket=hold-open-pr ;; *) if [ "$recent" = yes ]; then bucket=verify-recent-chat elif [ "$merged" = YES ] || [ "$pr" != "-" ]; then bucket=safe else bucket=review; fi ;; diff --git a/plugins/pstack/skills/principle-attack-the-premise/SKILL.md b/plugins/pstack/skills/principle-attack-the-premise/SKILL.md index 0a85cfb4..5d0625f4 100644 --- a/plugins/pstack/skills/principle-attack-the-premise/SKILL.md +++ b/plugins/pstack/skills/principle-attack-the-premise/SKILL.md @@ -1,6 +1,6 @@ --- name: principle-attack-the-premise -description: "Apply when two or more fixes that share one premise have failed the same gate. Question the shared premise before the next fix. For imbalance problems, take a census per actor and test the asymmetry hypothesis." +description: "Apply when two or more fixes that share one premise have failed the same gate. Write the premise down and test it directly before you write another fix that assumes it." user-invocable: false --- @@ -8,16 +8,14 @@ user-invocable: false When two or more fixes that share one premise have failed the same gate, suspect the premise, not the fixes. -**Why:** Each failure under a shared premise is evidence about the premise. +**Why:** Each failed fix is evidence against the belief it assumed. A third fix on the same premise spends effort where the evidence already says the cause is not. **Pattern:** - **Write the premise down.** The premise is the one sentence that every failed fix assumed. -- **For imbalance problems, take a census before the next fix.** Count the imbalance per actor to test which actors hold it. Write the census as a rerunnable script per [Build the Lever](../principle-build-the-lever/SKILL.md). -- **If the census shows skew, investigate it.** If the same few actors hold most of the imbalance on every run, something assigns them that role. Find what assigns the role. That assignment is the next "why" per [Fix Root Causes](../principle-fix-root-causes/SKILL.md). -- **When role assignment causes the imbalance, remove that asymmetry instead of compensating for it**, per the [Laziness Protocol](../principle-laziness-protocol/SKILL.md). Rotate the role between actors, randomize the assignment, or move the role, so that no actor holds it on every run. A return path, a shared pool, a batched hand-off, or a periodic rebalance leaves the assignment in place and adds work on every run. +- **Test the premise before the next fix.** Find an observation that would come out differently if the premise were false, and make it. Look at how the failures are distributed, not only at how large they are. Make the check rerunnable per [Build the Lever](../principle-build-the-lever/SKILL.md). +- **Follow the evidence.** If it contradicts the premise, the cause is elsewhere. Look for what the failures have in common that the premise did not predict. If it confirms the premise, ask the next "why" per [Fix Root Causes](../principle-fix-root-causes/SKILL.md). The earlier fixes may have treated a consequence instead of the cause. +- **Remove the condition instead of compensating for it**, per the [Laziness Protocol](../principle-laziness-protocol/SKILL.md). If the evidence shows that one part always lands on the losing side, change what puts it there. A mechanism that cleans up after it leaves the cause in place and adds work on every run. -**Stop:** -- Do not start the next fix before the premise is written down and tested. For an imbalance hypothesis, include the census. -- If the census is even across actors, that is evidence against the asymmetry hypothesis, not proof that the shared premise is correct. Keep questioning the premise and retain the census as evidence. +**Stop:** Do not start the next fix before the premise is written down and tested. This principle is distinct from [Redesign from First Principles](../principle-redesign-from-first-principles/SKILL.md), which rebuilds a design around a new requirement. It questions a fact the current design assumes. diff --git a/plugins/pstack/skills/principle-fix-root-causes/SKILL.md b/plugins/pstack/skills/principle-fix-root-causes/SKILL.md index 0597b15c..daf19fdb 100644 --- a/plugins/pstack/skills/principle-fix-root-causes/SKILL.md +++ b/plugins/pstack/skills/principle-fix-root-causes/SKILL.md @@ -1,23 +1,21 @@ --- name: principle-fix-root-causes -description: "Apply when debugging. Trace each symptom to its root cause and fix it there; reproduce first, ask why until you reach it, resist nil-check guards that silence crashes." +description: "Apply when debugging. Trace each symptom back to the first place the system went wrong and fix it there. A guard is a fix only when the state it handles is legitimate." user-invocable: false --- # Fix Root Causes -When debugging, do not fix symptoms. Trace every problem to its root cause and fix it there. +When debugging, fix the fault, not the place where it surfaced. The symptom is where a wrong state became visible. The root cause is the first place the system did something it should not do, or failed to handle a state it should handle. -**Why:** Symptom fixes accumulate. Each workaround makes the system harder to reason about, and the real bug remains. Root-cause fixes are slower upfront but reduce total debugging time. +**Why:** A symptom fix hides the evidence and leaves the fault free to surface somewhere else. Each one makes the system harder to reason about, because the code now carries a workaround for a bug that nobody named. **Pattern:** -- Reproduce first -- Ask "why" until you hit the root cause -- Do not add guards (adding a nil check to silence a crash is a symptom fix) -- If a workaround needs a paragraph-long comment to justify it, the code is wrong (fix the code, not the comment) -- Check for the pattern, not just the instance (grep for the same pattern, fix all instances) -- When stuck, instrument. Don't guess (add logging, read the actual error) - -**Restart bugs: suspect state before code** - -When something "fails after restart," suspect stale persistent state first: config files, caches, lock files, serialized state. If clearing a state file restores behavior, prioritize state validation as the fix. +- Reproduce first. +- Ask "why" at each step until the answer is a wrong decision in the code, not a consequence of an earlier one. +- When stuck, instrument. Read the actual error and the actual state. Do not guess. +- Ask whether the state that triggered the symptom is legitimate. If the system may be in that state, the code that broke on it is the root cause, and handling the state there is the fix. If the state should never exist, a check where it broke hides the bug. Find what produced the state. Guards, retries, swallowed errors, clamped values, and discarded state are the usual ways to hide a bug. +- When something fails only after a restart, suspect persisted state first: config files, caches, lock files, serialized state. +- Check for the pattern, not just the instance. Find every place with the same fault and fix them all. +- If a workaround needs a paragraph-long comment to justify it, the code is wrong. Fix the code, not the comment. +- If the deciding fact took a deep dig to find, propose the cheap, always-on signal that would have shown it at first look. diff --git a/plugins/pstack/skills/principle-make-operations-idempotent/SKILL.md b/plugins/pstack/skills/principle-make-operations-idempotent/SKILL.md index 3caa9188..22aea3f2 100644 --- a/plugins/pstack/skills/principle-make-operations-idempotent/SKILL.md +++ b/plugins/pstack/skills/principle-make-operations-idempotent/SKILL.md @@ -6,15 +6,13 @@ user-invocable: false # Make Operations Idempotent -Design operations so they converge to the correct state regardless of how many times they run or where they start from. Every state-mutating operation should answer: "What happens if this runs twice? What happens if the previous run crashed halfway?" +Design operations so they converge to the correct state regardless of how many times they run or where the previous run stopped. Every state-mutating operation should answer: "What happens if this runs twice? What happens if the previous run crashed halfway?" **Why:** Commands, lifecycle operations, and processing loops run where crashes, restarts, and retries are normal. If partial state changes the next run's outcome, every restart becomes a debugging session. -**The pattern:** -- Convergent startup: scan for existing state, clean stale artifacts, adopt live sessions -- Content-based cleanup: compare by content equivalence, not creation order -- Self-healing locks: use PID-based stale lock detection -- Idempotent scheduling: failed work respawns cleanly, fresh input regenerated after each cycle +**The idea:** derive each step from the state that exists, not from what the previous run meant to do. Read the current state, compare it to the goal, and apply only the difference. A step written this way is safe to repeat, because a repeat finds nothing left to do. + +Anything a run leaves behind, such as a lock, a temporary file, a half-written record, or a claimed job, must be recognizable as stale by the next run. Judge staleness by evidence that stays true wherever the previous run could have stopped, not only in the common case. A marker that the next run cannot judge turns every crash into manual cleanup. For example, write to a temporary path and rename it into place, or compare artifacts by content instead of by creation order. **The test:** 1. What happens if this runs twice in a row? diff --git a/plugins/pstack/skills/principle-migrate-callers-then-delete-legacy-apis/SKILL.md b/plugins/pstack/skills/principle-migrate-callers-then-delete-legacy-apis/SKILL.md index ca19666a..4fc3dc0b 100644 --- a/plugins/pstack/skills/principle-migrate-callers-then-delete-legacy-apis/SKILL.md +++ b/plugins/pstack/skills/principle-migrate-callers-then-delete-legacy-apis/SKILL.md @@ -6,17 +6,9 @@ user-invocable: false # Migrate Callers Then Delete Legacy APIs -When we decide a new API is the right design, migrate callers and remove the old API in the same refactor wave instead of preserving compatibility layers. - -**Rule:** -- Do not keep legacy API paths only because internal callers still exist -- Inventory callers, migrate them, and delete the old API immediately -- Treat temporary adapters as exceptional and time-boxed, not default architecture - Update tests to assert the new contract, and delete tests that only protect pre-refactor implementation details **When this applies:** - No external users depend on backward compatibility - The project can absorb coordinated breaking changes - The new API is part of a simplification or refactor initiative - -Keeping both old and new APIs creates dual-path complexity, slows cleanup, and makes the codebase feel append-only. diff --git a/plugins/pstack/skills/principle-test-behavior-not-implementation/SKILL.md b/plugins/pstack/skills/principle-test-behavior-not-implementation/SKILL.md index 179e1a14..079f8e3d 100644 --- a/plugins/pstack/skills/principle-test-behavior-not-implementation/SKILL.md +++ b/plugins/pstack/skills/principle-test-behavior-not-implementation/SKILL.md @@ -1,25 +1,20 @@ --- name: principle-test-behavior-not-implementation -description: "Apply when you write, change, or keep a test. Call the code the way its users do and assert the observable result or contract. Check whether a relevant defect makes the test fail. Keep useful negative-path and relational contract tests." +description: "Apply when you write, change, or keep a test. Assert what the code's users can observe, so that the test fails when that behavior breaks and keeps passing when only the internals change." user-invocable: false --- # Test Behavior, Not Implementation -A test calls the code the way its users do and asserts the result or contract they observe. An assertion about internal calls or a copied constant can miss that behavior. +A test is a claim about what the code does for its users: a caller, an end user, or another service. Behavior is what those users can observe through the interface they use. That includes return values, errors, state they can read back, and effects on systems outside the code, such as a request sent, a row written, or a message emitted. Implementation is what they cannot observe: private helpers, internal call order, intermediate data, and which internal collaborator did the work. -The check: before you keep a test, introduce a relevant defect and check that the test fails. Returning `undefined` can be a useful probe when a defined result is required. It does not invalidate a negative-path test whose correct result is absence or a relational contract check. +Judge each test against two imagined edits: -**Why:** A test that cannot fail for a defect costs CI time and review attention and catches nothing. A constant pin can obstruct harmless edits when the pinned value is not part of the contract. +- **Break the behavior.** Make the code return a wrong value, skip an effect, or do the effect twice. If the test still passes, it proves nothing. The usual causes are an assertion looser than the contract, an expected value computed by the code under test, a restated constant or prompt string that never runs the code that reads it, and an assertion on data that the test built itself. +- **Change the internals.** Keep every observable result the same and restructure the code underneath. If the test now fails, it is pinned to the implementation. It will need a rewrite on every refactor and still catches no bug. -**Five shapes that can miss the relevant behavior:** +Keep a test that catches the first edit and survives the second. Rewrite or delete the rest. To write a test that does both, drive the code through its users' interface with a concrete input. Derive the expected value from the requirement, not from a run of the code. Assert what the contract promises, no less and no more. -- **Weak or no assertion.** No `expect`, or an assertion that checks too little of the contract. `toBeDefined`, `toBeTruthy`, `toBeInstanceOf`, and `toBeGreaterThan(0)` fail on `undefined`, but can accept a wrong defined result. -- **Mock or absence only.** Only checking that a mock was called can miss a wrong payload. Absence assertions such as `not.toThrow`, `not.toHaveBeenCalled`, `toBeUndefined`, `toEqual([])`, or `toHaveLength(0)` are useful when absence is the required behavior. Check that an incorrect presence or side effect fails the test. -- **Self-referential.** An expected value computed by the same faulty path can hide a defect: `expect(f(a)).toBe(f(a))`. Comparing two implementations can still test a required relation if the comparison detects relevant disagreement. -- **Constant pin.** Restating an incidental constant can block refactoring without testing behavior: `expect(LIMITS.maxTools).toBe(8)`. Keep prompt, configuration, and table checks that enforce a real contract. -- **Fixture asserts fixture.** The assertion reads data the test built, and the subject never runs. A subject called in `beforeEach` can still be tested if the assertion observes its result. +The contract decides, not the shape of the assertion. When "an empty cart never charges" is the promise, an assertion that the payment client got no call is behavior. Pair it with a case where the charge happens, so that the test shows it can see a call. The payload that a mocked external service receives is behavior, because that service observes it. A call to an internal helper is implementation, because no user can see it. An empty list is the exact answer when no record matches, and a hollow check when the test only needs some result. An exact error message pins wording that no caller reads. The error type that callers branch on is the contract. -**The fix:** call the subject inside the test body with one concrete input and assert the literal output or the observable effect, `expect(slugify("Hello, World!")).toBe("hello-world")`. For an absence, exercise the negative path and prove that an incorrect presence or side effect fails. A separate positive-path test can provide contrast. For an incidental constant, test the mechanism that reads it. For a mock, assert the payload or observable state when those are the contract. Rewrite or delete a test only when it checks no relevant behavior or contract. - -**Keep** a test of a relation across a table's rows (a key present in two tables, a parent that exists), and a compile-time check in a `*.test-d.ts` file. +A test can pin an invariant instead of a call, such as a key that must exist in two tables or a compile-time type check. The invariant is the behavior. diff --git a/plugins/pstack/skills/principle-type-system-discipline/SKILL.md b/plugins/pstack/skills/principle-type-system-discipline/SKILL.md index e4f77e61..2e918856 100644 --- a/plugins/pstack/skills/principle-type-system-discipline/SKILL.md +++ b/plugins/pstack/skills/principle-type-system-discipline/SKILL.md @@ -1,6 +1,6 @@ --- name: principle-type-system-discipline -description: "Apply when designing types, reviewing a function signature, or writing code in any statically-typed language. Make illegal states unrepresentable, brand semantic primitives, parse external data at boundaries, refuse to lie to the compiler, exhaust variants, derive from authoritative schemas." +description: "Apply when designing types, reviewing a function signature, or writing code in any statically-typed language. Make illegal states unrepresentable, brand semantic primitives, parse external data at boundaries, exhaust variants, derive from authoritative schemas." user-invocable: false --- @@ -8,11 +8,6 @@ user-invocable: false Make the compiler reject impossible states, mismatched primitives, and unhandled variants. Prefer defining errors and special cases out of existence over proliferating handlers. For TypeScript syntax, see the `typescript-best-practices` skill. -- **Make illegal states unrepresentable.** Model variants as sum types (discriminated unions, enums with payloads, sealed classes), not as a bag of optional fields where contradictory combinations compile. `{ completed: boolean; completedAt?: Date }` admits `completed: true` with no date. Derive the boolean from `completedAt`, or model `{ kind: 'open' } | { kind: 'done'; at: Date }`. If you can write a comment explaining when a combination of fields is valid, split it into a sum type. -- **Types are constructions, not restrictions.** Build the type up from the values you want instead of carving them out of a looser type with checks. A non-empty list is a head plus a rest. A valid time range is a start plus a duration. Choose the shape that cannot build the illegal value and expose the interface callers need on top. - **Brand semantic primitives.** `UserId` and `OrderId` are strings underneath but should not be interchangeable. Give each its own type, such as a branded intersection in TypeScript or a newtype elsewhere. Validate once at creation, trust the type downstream. If two arguments share a primitive type but mean different things, brand them. - **External data is untyped until parsed.** RPC payloads, JSON, IPC messages, CLI args, config files, environment variables, database rows. Have a parse function at every boundary that turns unstructured input into the typed model. The **boundary-discipline** principle skill says where to put validation. -- **Don't lie to the type system.** Casts, unsafe coercions, and assertion functions that bypass the compiler are latent runtime crashes. If the compiler can't prove a fact, prove it (validate, narrow, refine the model) or accept that the cast is a hazard. Trace each `any`, `as`, or `assertNotNull` to the boundary and validate there. -- **Exhaustive matching is the compiler's job.** A match on a sum type must fail compilation when a new variant is added without handling, so the compiler tells the next agent where to add a case. Use your language's idiom: a `never`-typed binding in TypeScript, an unannotated `match` in Rust, `-Wincomplete-patterns` in Haskell, sealed-class match exhaustiveness in Kotlin. - **Derive types from authoritative schemas.** When a protocol buffer, OpenAPI spec, GraphQL schema, database migration, design-token file, or another file owns a shape, derive from it instead of hand-rolling a parallel type. See the **encode-lessons-in-structure** principle skill. -- **Strengthen a type only where partiality appears.** A runtime assertion, null check, or "this should never happen" throw marks the place a type is too weak. Push that check up into the type. Then stop. The type system's job is to track the cases each use site must handle, not to describe the data as precisely as possible. Prefer total functions. `sum` of an empty list is 0, so it takes the plain list. `head` of an empty list has no answer, so it demands the non-empty one. If nothing would otherwise panic, keep the plain type. diff --git a/plugins/pstack/skills/recall/SKILL.md b/plugins/pstack/skills/recall/SKILL.md index 42037fd2..0baacbf4 100644 --- a/plugins/pstack/skills/recall/SKILL.md +++ b/plugins/pstack/skills/recall/SKILL.md @@ -15,7 +15,7 @@ Transcripts live at `~/.claude/projects/<encoded-cwd>/<uuid>.jsonl`, where `<enc 1. Classify, then route. One specific prior chat to resume is the `session-pickup` playbook, not this. Turning habits into a durable skill is `automate-me`. A human-readable summary of your work is a different task. Recall loads working context across recent chats before you act. If the user already gave you a full state capsule (paths, branch, the change), use it and skip the mining. 2. Lock the scope before searching. Pin the window ("recent" is a real range, default the last 7 days), the topic if named, and the workspace (default the active one. Never read another project's transcripts without being asked). State the scope back. Never quietly turn "all" into "recent N". -3. Fan out across your chat history. Spawn parallel subagents on a fast, cheap model, each taking a slice of the corpus. Tell every subagent to order candidates by real modification time (`ls -t`) and never by UUID name, grep the topic first and then read only the matching chats and only their relevant regions, and skip the current chat plus obvious noise (subagent, eval, and test chats). Each returns the same schema, one block per chat: topic, the user's goal, decisions, open threads, struggles and corrections, and artifacts (PRs, tickets, branches), each citing the chat UUID. For one or two chats, skip the fan-out and search directly. The raw transcripts stay in the subagents. The main thread gets only their findings. +3. Fan out across your chat history. Spawn parallel subagents on a fast, cheap model, each taking a slice of the corpus. Tell every subagent to order candidates by real modification time (`ls -t`) and never by UUID name, grep the topic first and then read only the matching chats and only their relevant regions, and skip the current chat plus obvious noise (subagent, eval, and test chats). Each returns the same schema, one block per chat: topic, the user's goal, decisions, open threads, struggles and corrections, and artifacts (PRs, tickets, branches), each citing the chat UUID. For one or two chats, skip the fan-out and search directly. The raw transcripts stay in the subagents. The main thread gets only their findings. Pick each subagent's effort for the job, per the Subagents section of the **poteto-mode** skill. 4. Sweep the shared record whenever the topic names a feature, file, subsystem, area, or bug. This is the default, not a judgment call, and "my work on X" does not exempt it. Hand it to the **why** skill's source investigators, but steer their question from "why was this built this way" to "what's the current state, what's been tried and didn't hold, and what are users still reporting". Reuse its per-source playbooks, run the investigators in parallel with the chat-history mining, and inherit its posture: one investigator per source, null results are findings, skip an unavailable MCP and say so. Fold what comes back into the brief. Skip this step only for pure activity recall with no named target ("what did I do this week"), where your own history and live state are the entire answer. 5. Verify against live state. Take the PRs, branches, and tickets that the mining and the sweep surfaced and check them with `git` and `gh`. When the answer hinges on what an agent actually did (the tools it ran, files it read, errors it hit), read the full transcript, not just a trimmed local copy. 6. Write the brief to the contract below. Group by thread. Stay on the named topic. diff --git a/plugins/pstack/skills/reflect/SKILL.md b/plugins/pstack/skills/reflect/SKILL.md index 85cbc75a..a1d550b7 100644 --- a/plugins/pstack/skills/reflect/SKILL.md +++ b/plugins/pstack/skills/reflect/SKILL.md @@ -17,7 +17,7 @@ Before fanning out, resolve and verify this conversation's transcript through th ## 2. Spawn three reviewers in parallel -Start all three read-only lanes in one fan-out phase through provider dispatch. Reviewers need MCP access to look up the tickets, chat threads, and observability traces the transcript references, so keep them native to the parent. The prompt forbids file writes; the parent applies edits. +Start all three read-only lanes in one fan-out phase through provider dispatch. Reviewers need MCP access to look up the tickets, chat threads, and observability traces the transcript references, so keep them native to the parent. The prompt forbids file writes; the parent applies edits. Pick each subagent's effort for the job, per the Subagents section of the **poteto-mode** skill. | Lens | Model descriptor | Prompt template | |---|---|---| diff --git a/plugins/pstack/skills/setup-pstack/SKILL.md b/plugins/pstack/skills/setup-pstack/SKILL.md index d0e598a4..34ddf78d 100644 --- a/plugins/pstack/skills/setup-pstack/SKILL.md +++ b/plugins/pstack/skills/setup-pstack/SKILL.md @@ -33,15 +33,21 @@ Use the harness and tool surface running this skill: Claude Code or Codex. Envir ### 2. Load current state -Inspect `<config-home>/pstack-models.md` and the existing integration: resolve Claude's single model-sheet import, or read Codex's single bounded model block, reporting missing targets and stopping on unreadable, malformed, or ambiguous state. Use the sole surviving configuration, or the editable sheet when both configurations agree; on Codex, the sheet remains authoritative and the block is its mirror, recoverable when the sheet is absent. If surviving configurations differ, show both sources and their differing assignments and require an explicit source choice before normalization or probing, without merging them automatically. Use first-run defaults only when neither source contains recoverable configuration, then apply the existing normalization, role-completion, and validation rules to the selected state. Before matrix validation, normalize only the rolling-alias predecessors that earlier pstack releases generated. A provider-qualified Claude model is migratable when its model component starts with `claude-fable-` or `claude-opus-` and the remaining revision contains only digits and hyphens. Replace that component in memory with `fable` or `opus`, preserving the provider, effort, role, and lane order. Record each original and normalized descriptor for the confirmation in step 7. This migration is valid loaded state and does not require a separate operator choice. +Inspect `<config-home>/pstack-models.md` and the existing integration: resolve Claude's single model-sheet import, or read Codex's single bounded model block, reporting missing targets and stopping on unreadable, malformed, or ambiguous state. Use the sole surviving configuration, or the editable sheet when both configurations agree; on Codex, the sheet remains authoritative and the block is its mirror, recoverable when the sheet is absent. If surviving configurations differ, show both sources and their differing assignments and require an explicit source choice before normalization or probing, without merging them automatically. Use first-run defaults only when neither source contains recoverable configuration, then apply the existing normalization, role-completion, and validation rules to the selected state. Before matrix validation, normalize only versioned predecessors of the catalog's [rolling aliases](../poteto-mode/references/model-catalog.md#kept-predecessors-and-normalization). Match in this order: an exact model-matrix row, then a kept predecessor, then a rolling-alias revision. A model that is its own matrix row, such as a registered exact pin, is never migratable. Any other provider-qualified Claude model is migratable when its model component starts with a rolling alias's versioned-predecessor prefix and the remaining revision contains only digits and hyphens. Replace that component in memory with the alias, preserving the provider, effort, role, and lane order. Record each original and normalized descriptor for the confirmation in step 7. This migration is valid loaded state and does not require a separate operator choice. -Treat the normalized values as current role-to-family assignments. Overlay those rows on the complete first-run role map, the catalog's [role defaults](../poteto-mode/references/model-catalog.md#role-defaults). Materialize any missing documented role row from that map on the next successful write. A duplicate role row is inconsistent state; report it and resolve it before probing. A row whose role is not in the role defaults, such as `how critics`, is from a retired role. Drop it and list it at confirmation. A bare host-native slug from an older sheet is also invalid because it does not say which provider owns it. A versioned Claude model outside the two migration families remains inconsistent state. A value whose `(provider, model)` is a [kept predecessor](../poteto-mode/references/model-catalog.md#kept-predecessors-and-normalization) is not inconsistent state. On a rerun, propose replacing every occurrence with its family's current model at the same effort, and ask. If the operator declines, that family's model stays the kept predecessor for steps 3 to 7: probe it, render it, and write it unchanged. If the sheet is missing, seed from the role defaults, whose descriptors carry the model matrix's Default effort cells. +The line `codex writer sandbox: full-access` or `codex writer sandbox: bounded` is a setting, not a role; [Codex writer sandbox](../poteto-mode/references/provider-dispatch.md#codex-writer-sandbox) says how a parent uses it. Set it aside before the role overlay, the retired-role rule, and descriptor validation, and carry it through steps 6 to 8. A second setting line, or another value, is inconsistent state. + +Treat the normalized values as current role-to-family assignments. Overlay those rows on the complete first-run role map, the catalog's [role defaults](../poteto-mode/references/model-catalog.md#role-defaults). Materialize any missing documented role row from that map on the next successful write. A duplicate role row is inconsistent state; report it and resolve it before probing. A row whose role is not in the role defaults, such as `how critics`, is from a retired role. Drop it and list it at confirmation. A bare host-native slug from an older sheet is also invalid because it does not say which provider owns it. A versioned Claude model that matches no matrix row, kept predecessor, or rolling alias remains inconsistent state. A value whose `(provider, model)` is a [kept predecessor](../poteto-mode/references/model-catalog.md#kept-predecessors-and-normalization) is not inconsistent state. On a rerun, propose replacing every occurrence with its family's current model at the same effort, and ask. If the operator declines, that family's model stays the kept predecessor for steps 3 to 7: probe it, render it, and write it unchanged. If the sheet is missing, seed from the role defaults, whose descriptors carry the model matrix's Default effort cells. Then ask whether to keep these role-to-family assignments or change named roles. Keeping them is the default. Apply only role changes the operator names; never offer a reset of a customized sheet to the first-run assignments. A changed role may use any model-matrix family, `inherit-parent`, or `auto`. +If the loaded state has no setting line, ask once whether Codex writer lanes that the external runner launches should run with Codex's full-access sandbox. Say that full access lets the writer launch a browser and write dot-folders such as `.agents/`, and that it removes every sandbox limit, including filesystem confinement to the worktree. Declining is the default. Record `codex writer sandbox: full-access` on a yes and `codex writer sandbox: bounded` on a decline, so a rerun does not ask again. When the line exists, keep it unless the operator names a change, and do not ask. + ### 3. Parse per-family efforts -Read the catalog's model matrix and kept predecessors. Every non-alias value must match `<provider>:<model>@<effort>`. Map it to exactly one matrix family by `(provider, model)`, matching a kept predecessor to its family, require its effort to appear in the Selectable efforts cell of that row or of the kept predecessor's row, and collect the effort. Reject `ultra` for every row whose Selectable efforts cell does not list it. `inherit-parent` and `auto` rows carry no family effort. +Read the catalog's model matrix and kept predecessors. Every non-alias role value must match `<provider>:<model>@<effort>`. Map it to exactly one matrix family by `(provider, model)`, matching a kept predecessor to its family, require its effort to appear in the Selectable efforts cell of that row or of the kept predecessor's row, and collect the effort. Reject `ultra` for every row whose Selectable efforts cell does not list it. `inherit-parent` and `auto` rows carry no family effort. + +Lanes for these roles write files: `feature, refactoring`, `bug-fix`, `perf-issue`, and `hillclimb`, whose playbooks delegate with `isolated-write`; `hardest tasks`, which takes the hardest code changes; and `arena runners`, `architect runners` and `swarm workers`, which write to their own worktrees or output directories. Refuse any of those roles on a provider whose Access modes cell in the [route table](../poteto-mode/references/model-catalog.md#routes) lacks `isolated-write`: name the role and the provider, and say why. The runner refuses that provider's writer lane before launch, so the read-only probe in step 5 would pass and the role would fail at dispatch. Ask for another family or alias for that role, and do not probe or write until it changes. Every other role may use a read-only provider. An unmatched provider/model, out-of-domain effort, or duplicate role is inconsistent state. Stop, show the conflicting rows verbatim, and ask for an explicit matrix family or alias replacement. If one or more families have mixed efforts, show every conflicting family and role row, then ask for one normalized effort per family from its Selectable efforts cell. Do not invent a precedence rule. Do not probe or write while any inconsistency is unresolved. @@ -79,15 +85,17 @@ Build the new sheet in memory. Do not write it yet. - First run: start from the catalog's role defaults, with the step 2 role changes applied. - Rerun: start from the normalized complete role map from step 2, with the step 2 role changes applied, preserving each loaded row's lane order and family (or alias) per lane. +Keep the setting line from step 2 where the loaded sheet had it. When step 2 adds it, put it after the last role row. + Rewrite every matrix-family descriptor to `provider:model@<requested effort for that family>`, using each family's model chosen in step 2. Leave `inherit-parent` and `auto` unchanged. An effort-only rerun cannot change a role's family. Changing Grok's effort updates every Grok occurrence and does not move a Sol role onto Grok. Refuse an unqualified slug, an unavailable route, a model outside the model matrix and its kept predecessors, or a provider/model mismatch. ### 7. Confirm and commit -Show any rolling-alias migrations as original and normalized descriptors and any retired-role rows dropped in step 2. Then show the source file read in step 2, the new destination `<config-home>/pstack-models.md`, the route table for this parent, and every rendered role and descriptor. Carry the loaded assignments into the new destination; only explicit role or effort changes and the documented normalization may change them. Ask for confirmation before writing. +Show any rolling-alias migrations as original and normalized descriptors and any retired-role rows dropped in step 2. Then show the source file read in step 2, the new destination `<config-home>/pstack-models.md`, the route table for this parent, every rendered role and descriptor, and the setting line. Carry the loaded assignments into the new destination; only explicit role or effort changes and the documented normalization may change them. Ask for confirmation before writing. Why and Reflect require the parent's live MCP surface. Keep their investigator, reviewer, and synthesizer roles on `inherit-parent` or `auto`; the bounded external runner deliberately omits ambient MCPs. `inherit-parent` and `auto` always validate, but say when they reduce a panel's provider diversity. For panel roles, one lane runs per entry. The list length is the fan-out count. `arena cross-judge pool` is a list from which Arena chooses a provider different from the parent and base candidate when possible. `swarm workers` is the default for every worker unless a race explicitly assigns another descriptor. -Every non-alias value must match `<provider>:<model>@<effort>` and must have passed step 5. +Every non-alias role value must match `<provider>:<model>@<effort>` and must have passed step 5. After the operator confirms, separately check the parent's effective write permission for **both actual destinations** (the sheet and parent integration, resolved through `CODEX_HOME` when set) and any directory creation needed. Render both artifacts in memory and perform this check before the first write or directory creation. Native probes succeeding does not grant configuration-write permission. A restricted parent may still be allowed to write under `CODEX_HOME`; assess each resolved destination rather than rejecting it just because it is outside the workspace. If either destination is denied or its permission remains unknown, stop with the destination-specific explanation: leave every existing artifact byte-identical and create neither artifact on a first run. Do not test permission by overwriting a target, write only the allowed artifact, or automatically request escalation. @@ -110,3 +118,7 @@ Do not copy the model sheet between harnesses without rerunning the parent-speci Before declaring setup complete, run one small read-only mixed panel from this parent: every distinct chosen descriptor, distinct output/receipt paths, and an independent cross-judge. Launch Claude-native agents and every external process in the background with retained handles, then drain them. Verify the native transcript entries and every external receipt. A structural config check or unit test is not a substitute. Report the sheet path, parent route table, requested-effort probe results, smoke results, and external elapsed/token/cost receipts. Re-running this skill re-probes and updates the same sheet. Do not claim the provider exposed hidden applied-effort observability. + +### 10. Offer a formal verification skill (optional) + +Check whether the project already has a formal verification skill (a project skill that runs a model checker, a proof, or another verifier against the code). If not, offer once: "want a formal verification skill, so agents can prove a key property of your code holds in every case in scope, not only the cases tests reach? I can generate one with /create-formal-verification-skill." On yes, invoke `/create-formal-verification-skill` (resolves wherever pstack is installed: workspace, user, or plugin). On no, move on without pushing. diff --git a/plugins/pstack/skills/sharpen/SKILL.md b/plugins/pstack/skills/sharpen/SKILL.md new file mode 100644 index 00000000..c1cfebe4 --- /dev/null +++ b/plugins/pstack/skills/sharpen/SKILL.md @@ -0,0 +1,7 @@ +--- +name: sharpen +description: "Sharpen a task before starting: the expert approach, what the user is really after, and the right level of specification. Use for /sharpen." +disable-model-invocation: true +--- + +Before starting, ask how a top expert would approach this task (their raw sources, their real methods, an outside check that the result is right), what the user is really after, and whether the task is over- or underspecified. Loosen constraints that don't serve the goal as readily as you add ones that do, and leave choices that are the user's to make open instead of deciding them. Fold in only what makes the work better, not bigger, but always keep one outside check that the result is right, without narrating the framing, then do the work. diff --git a/plugins/pstack/skills/show-me-your-work/SKILL.md b/plugins/pstack/skills/show-me-your-work/SKILL.md index 65c369b8..a9c61992 100644 --- a/plugins/pstack/skills/show-me-your-work/SKILL.md +++ b/plugins/pstack/skills/show-me-your-work/SKILL.md @@ -63,7 +63,7 @@ Correct the log, not the story. The audit never edits or removes a row, even an ## Cross-model review of the trail -Before handing back, spawn a subagent on a different model family from the one that did the work. Self-review is not a substitute. The subagent reads the audit trail and the run's transcript, then flags what the user should pay attention to. Not a redo of the work, a scan for what's suboptimal or risky. +Before handing back, spawn a subagent on a different model family from the one that did the work. Self-review is not a substitute. The subagent reads the audit trail and the run's transcript, then flags what the user should pay attention to. Not a redo of the work, a scan for what's suboptimal or risky. Pick each subagent's effort for the job, per the Subagents section of the **poteto-mode** skill. - Decisions logged with weak or absent evidence. - Verification steps skipped or claimed without proof in the transcript. diff --git a/plugins/pstack/skills/swarm/SKILL.md b/plugins/pstack/skills/swarm/SKILL.md index 2a6a8654..4e2f8c67 100644 --- a/plugins/pstack/skills/swarm/SKILL.md +++ b/plugins/pstack/skills/swarm/SKILL.md @@ -5,8 +5,6 @@ description: "Fan out N parallel workers, drain them, and return one report. Use # Swarm -Fan out N parallel workers. They may cover separate slices, race the same brief, or mix both. The parent waits, aggregates, and returns one report. - **Dispatch contract.** Resolve each worker descriptor through [`provider-dispatch.md`](../poteto-mode/references/provider-dispatch.md). The parent starts native and external lanes; workers never route themselves. On Codex, resolve remaining Claude tool names via [`codex-tools.md`](../poteto-mode/references/codex-tools.md). ## Start @@ -22,8 +20,7 @@ Open a todolist with one entry per phase before launching anything. 1. State the done predicate and the artifact or report the swarm must return. 2. Choose the shape. Partition into slices, race N workers on identical briefs, or mix both. For a race or mixed shape, declare `first pass`, `rank all`, or `best-of` before spawning. -3. Set N from the user or derive it from the shape. N is total workers, not the number that run at once. -4. Pick the worker descriptor from `swarm workers` in the current harness's pstack model sheet when present. Otherwise use its [role default](../poteto-mode/references/model-catalog.md#role-defaults). For a model race, name each arm's descriptor up front. +4. Pick the worker descriptor from `swarm workers` in the current harness's pstack model sheet when present. Otherwise use its [role default](../poteto-mode/references/model-catalog.md#role-defaults). For a model race, name each arm's descriptor up front. Pick each subagent's effort for the job, per the Subagents section of the **poteto-mode** skill. 5. Give each worker its own writable output when it writes. When workers verify or measure commits, each brief names the exact SHAs. A measurement brief also names the method (sample count, what one sample is, order). The worker records both in its result. ## Phase B: Fan out @@ -40,8 +37,6 @@ If a worker drops out, proceed with N-1 and note the provider, model, and receip Read the terminal results. Drop a result that does not record the SHAs and method its brief names, and respawn that worker once. After a second miss, record a gap. A gap does not count as a pass. For coverage, every required slice needs a result. For a race, apply the selection rule declared up front. Use first pass, rank all, or best-of. Do not paste raw worker dumps. -Keep a compact result table, one-line evidenced issues, and explicit gaps or dropouts. - ## Phase D: Report Return one consolidated in-chat report with the table, issue one-liners, gaps or dropouts, and the race rule when used. diff --git a/plugins/pstack/skills/typescript-best-practices/SKILL.md b/plugins/pstack/skills/typescript-best-practices/SKILL.md index 2bf543fb..33ebef76 100644 --- a/plugins/pstack/skills/typescript-best-practices/SKILL.md +++ b/plugins/pstack/skills/typescript-best-practices/SKILL.md @@ -11,7 +11,7 @@ Apply the **type-system-discipline** principle skill first. | Rule | Summary | |------|---------| | Discriminated unions | Model variants with a `kind` literal discriminant so impossible states can't be represented. No optional-field bags. | -| Branded types | Brand primitives with `& { readonly __brand: "X" }` so they can't be mixed up. Validate once at the boundary. | +| Branded types | Brand primitives with `& { readonly __brand: "X" }` so they can't be mixed up. Validate once at the boundary. A brand comes from a verifying type guard (`input is AgentId`), never from `as`. | | Constructive modeling | Build the shape so the illegal value can't be constructed. `[T, ...T[]]` for non-empty, `[T, T][]` for even length, `start` plus `duration` for a range. Not a runtime guard, not a wish for refinement types. | | Simplest total type | Keep `T[]` while every operation on it stays total. Strengthen to `NonEmpty<T>` only where the loose type forces `!`, a cast, or a "should never happen" throw. | | `unknown` over `any` | External data is `unknown`. | diff --git a/plugins/pstack/skills/typescript-best-practices/references/patterns.md b/plugins/pstack/skills/typescript-best-practices/references/patterns.md index c21d876b..f8dd99b6 100644 --- a/plugins/pstack/skills/typescript-best-practices/references/patterns.md +++ b/plugins/pstack/skills/typescript-best-practices/references/patterns.md @@ -9,9 +9,13 @@ Brand primitives so they can't be mixed up. Validate once at the boundary. Downs ```ts type AgentId = string & { readonly __brand: "AgentId" }; +function isAgentId(input: string): input is AgentId { + return isUUID(input); +} + function parseAgentId(input: string): AgentId { - if (!isUUID(input)) throw new Error(`Invalid agent id: ${input}`); - return input as AgentId; + if (!isAgentId(input)) throw new Error(`Invalid agent id: ${input}`); + return input; } function focusAgent(id: AgentId): void { diff --git a/plugins/pstack/skills/why/SKILL.md b/plugins/pstack/skills/why/SKILL.md index d010ce08..ec98ea3e 100644 --- a/plugins/pstack/skills/why/SKILL.md +++ b/plugins/pstack/skills/why/SKILL.md @@ -7,7 +7,7 @@ description: "Use for 'why does X work this way', 'why we picked Y', design rati Investigate the motivation and intent behind code. The companion `how` skill answers what the code does and how it works. `why` answers what forces led to its shape. -**Dispatch contract.** Resolve every configured role through [`provider-dispatch.md`](../poteto-mode/references/provider-dispatch.md). Investigators require the parent's live MCP surface, so the default and supported portable route is `inherit-parent` (or its `auto` alias). Pass the code anchor by path. On Codex, resolve remaining Claude tool names via [`codex-tools.md`](../poteto-mode/references/codex-tools.md). +**Dispatch contract.** Resolve every configured role through [`provider-dispatch.md`](../poteto-mode/references/provider-dispatch.md). Investigators require the parent's live MCP surface, so the default and supported portable route is `inherit-parent` (or its `auto` alias). Pass the code anchor by path. On Codex, resolve remaining Claude tool names via [`codex-tools.md`](../poteto-mode/references/codex-tools.md). Pick each subagent's effort for the job, per the Subagents section of the **poteto-mode** skill. Every lane keeps MCP access. Do not pick a subagent type or execution mode that strips it, because that disables the MCP-backed investigators and the synthesizer's citation spot-checks. Investigators still shouldn't write anything. diff --git a/tests/skill-collision-repro.sh b/tests/skill-collision-repro.sh index 27c6b97f..5be0f98d 100755 --- a/tests/skill-collision-repro.sh +++ b/tests/skill-collision-repro.sh @@ -49,20 +49,26 @@ fi # provider reports may still appear in runner fixtures, but no shipped # descriptor, native-agent model field, or live test invocation may pin one. # The registry's released pairs record old pins so they keep resolving to their family. +# The registered exact pin claude-opus-5-5 is its own family, with its own generated +# agents, so only that token is removed before the second match; any other pin on the +# same line still fails. # registry.test.ts scans TypeScript and JavaScript string literals for the same pins. +rolling='fable|opus|sonnet' legacy_model_pins="$( grep -REn \ --include='*.md' --include='*.ts' --include='*.sh' \ - 'claude:claude-(fable|opus)-[0-9]|^model: claude-(fable|opus)-[0-9]|--model claude-(fable|opus)-[0-9]' \ + "claude:claude-($rolling)-[0-9]|^model: claude-($rolling)-[0-9]|--model claude-($rolling)-[0-9]" \ "$repo/plugins/pstack" "$repo/tests" "$repo/README.md" "$repo/docs/reference.md" \ - 2>/dev/null | grep -Ev '^[^:]*/skills/poteto-mode/scripts/runner/registry/models\.ts:[0-9]+: \{ pair: "claude:claude-(fable|opus)-[0-9]+", family: "(fable|opus)" \},$' || true + 2>/dev/null | grep -Ev '^[^:]*/skills/poteto-mode/scripts/runner/registry/models\.ts:[0-9]+: \{ pair: "claude:claude-(fable|opus)-[0-9]+", family: "(fable|opus)" \},$' | + sed -E 's/claude-opus-5-5([^0-9A-Za-z_.-]|$)/\1/g' | + grep -E "claude:claude-($rolling)-[0-9]|:[0-9]+:model: claude-($rolling)-[0-9]|--model claude-($rolling)-[0-9]" || true )" if [ -n "$legacy_model_pins" ]; then - note "FAIL: active Fable or Opus configuration still pins a model revision:" + note "FAIL: active Fable, Opus or Sonnet configuration still pins a model revision:" note "$legacy_model_pins" fail=1 else - note "ok: active Fable and Opus configuration uses rolling aliases" + note "ok: active Fable, Opus and Sonnet configuration uses rolling aliases" fi setup="$repo/plugins/pstack/skills/setup-pstack/SKILL.md" @@ -307,14 +313,13 @@ grep -Fq 'An appended stack child keeps its recorded parent tip until that paren grep -Fq 'For fork heads, take the order from the verified local parent ancestry' "$shipping" || shipping_safety_bad="${shipping_safety_bad}Shipping tries to infer a fork stack from shared trunk bases"$'\n' grep -Fq 'only the current bottom PR through Shipping, one at a time' "$autopilot_stack" || shipping_safety_bad="${shipping_safety_bad}Autopilot-stack offers merge-when-ready outside the current bottom frontier"$'\n' grep -Fq -- '--force-with-lease="refs/heads/$branch:$captured_sha"' "$autopilot_full" || shipping_safety_bad="${shipping_safety_bad}Autopilot-full does not bind its post-rebase push to the captured SHA"$'\n' -grep -Fq "A private-stack child fetches its parent's exact tip" "$autopilot_full" || shipping_safety_bad="${shipping_safety_bad}Autopilot-full rebases a private-stack child onto trunk instead of its parent"$'\n' grep -Fq 'Record the selected exact commit as `<target-tip>`' "$autopilot_full" || shipping_safety_bad="${shipping_safety_bad}Autopilot-full does not record the exact rebase target"$'\n' -grep -Fq 'git rebase --onto "$target_tip" "$current_base_sha" -- "$branch"' "$autopilot_full" || shipping_safety_bad="${shipping_safety_bad}Autopilot-full does not isolate child commits when changing a private-stack parent"$'\n' +grep -Fq 'git rebase --onto "$target_tip" "$current_base_sha" -- "$branch"' "$autopilot_full" || shipping_safety_bad="${shipping_safety_bad}Autopilot-full does not isolate its own commits when changing the rebase target"$'\n' grep -Fq 'git fetch -- "$head_url" "refs/heads/$branch"' "$opening_a_pr" || shipping_safety_bad="${shipping_safety_bad}Opening a PR does not refresh a fork head through its push repository URL"$'\n' grep -Fq 'resolve and validate `<head-url>` through Shipping step 1, capture it as `head_url`' "$opening_a_pr" || shipping_safety_bad="${shipping_safety_bad}Opening a PR does not resolve its head URL before a shared-worktree refresh"$'\n' grep -Fq 'git fetch -- "$head_url" "refs/heads/$head_branch"' "$multi_phase_plan" || shipping_safety_bad="${shipping_safety_bad}Multi-phase plan does not fetch a live-lane head through its push repository URL"$'\n' grep -Fq 'Resolve and validate `<head-url>` through Shipping step 1 and capture it as `head_url` for the live-lane fetch' "$multi_phase_plan" || shipping_safety_bad="${shipping_safety_bad}Multi-phase plan does not resolve its head URL before a live-lane fetch"$'\n' -autopilot_full_rewrite_order="$(awk '/^2\. / { capture = index($0, "git ls-remote"); rebase = index($0, "git rebase --onto"); if (capture == 0 || rebase == 0 || capture >= rebase) print $0 }' "$autopilot_full")" +autopilot_full_rewrite_order="$(awk '/^2\. / { step = 1 } /^3\. / { step = 0 } step { text = text $0 "\n" } END { capture = index(text, "git ls-remote"); rebase = index(text, "git rebase --onto"); if (capture == 0 || rebase == 0 || capture >= rebase) print "step 2" }' "$autopilot_full")" if [ -n "$autopilot_full_rewrite_order" ]; then shipping_safety_bad="${shipping_safety_bad}Autopilot-full does not capture the published head before rebasing"$'\n' fi @@ -322,7 +327,7 @@ grep -Fq "Shipping step 4's disarm-and-confirm rule" "$autopilot_full" || shippi grep -Fq "Shipping step 5's server-enforced expected-head flow" "$autopilot_full" || shipping_safety_bad="${shipping_safety_bad}Autopilot-full does not bind owner merges to the current landing SHA"$'\n' grep -Fq -- '--force-with-lease="refs/heads/$branch:$captured_sha"' "$autopilot_stack" || shipping_safety_bad="${shipping_safety_bad}Autopilot-stack does not bind its topology push to the captured SHA"$'\n' grep -Fq "Shipping step 4's disarm-and-confirm rule" "$autopilot_stack" || shipping_safety_bad="${shipping_safety_bad}Autopilot-stack can rewrite an armed pull request"$'\n' -grep -Fq "step 6's captured-SHA lease flow" "$autopilot_stack" || shipping_safety_bad="${shipping_safety_bad}Autopilot-stack drift handling does not reuse its guarded push flow"$'\n' +grep -Fq "step 2's captured-SHA lease flow" "$autopilot_stack" || shipping_safety_bad="${shipping_safety_bad}Autopilot-stack drift handling does not reuse its guarded push flow"$'\n' if [ -n "$shipping_safety_bad" ]; then note "FAIL: forge-neutral landing safety rules regressed:" note "$shipping_safety_bad"