diff --git a/README.md b/README.md index a4e69103..4611533c 100644 --- a/README.md +++ b/README.md @@ -134,12 +134,14 @@ $ quest --root examples/export unblocked 1 quest/m0/export.md $ quest --root examples/export unblocked --json --limit 1 -[{"path":"quest/m0/export.md","dependents":1}] +[{"path":"quest/m0/export.md","dependents":1,"models":[{"harness":"claude","model":"claude-opus-5-5","effort":"low"},{"harness":"codex","model":"gpt-6.1-sol","effort":"medium"}]}] ``` Each plain output line contains the count, a tab, and the path. On a terminal, a heading explains the count. `--json` prints an ordered array, including `[]` -when no quests are ready; `--limit ` requires a positive integer. +when no quests are ready, and gives each quest the ranked `models` that write +it, as [Route work by size](#route-work-by-size) describes; `--limit ` +requires a positive integer. Invalid trees fail with diagnostics and no ranking. `quest-spawn` uses the ranking unless you explicitly select work in another order. @@ -1096,6 +1098,50 @@ arrive together, it takes a third; a repeated Ctrl-C, which Just does not forward, kills at the second. There is no fairness guarantee, and no timeout except `--quick`'s: any other command waits as long as the host stays busy. +## Route work by size + +`quest` ranks the harnesses, models, and efforts that write work of each size, +the same for every repository unless its `AGENTS.md` names others: + +| Size | Writer | Fallback | +|---|---|---| +| XS | Claude Code `claude-haiku-5-5` high | Codex `gpt-6.1-sol` low | +| S | Claude Code `claude-opus-5-5` low | Codex `gpt-6.1-sol` medium | +| M | Claude Code `claude-opus-5-5` medium | Codex `gpt-6.1-sol` high | +| L | Claude Code `claude-opus-5-5` high | Codex `gpt-6.1-sol` xhigh | +| XL | Claude Code `claude-opus-5-5` xhigh | none | + +`quest unblocked --json` lists each ready quest's ranked `models`: its size's +list, or the next size's while a `Maintainer:` item under its `## Waiting` +leaves a decision open, since settling it is part of the work; an XL stays +XL. `/quest-spawn` starts a quest on the first entry that can run, and falls +to the next only while nothing is written: when the CLI is missing or logged +out, or the agent stops at its usage limit with nothing beyond the claim +commit. Once a model writes, the PR is its own. + +A `/quest-iterate` stays on the PR's writer, the model its `## Models` +record's first `Implementer:` line names, so the other model can always +review it. The coordinator dispatching it sizes the work left on the PR, not +the quest's original size, so a merge-and-record iterate on an L quest runs at +XS, and `quest models` prints the entry to run: + +```console +$ quest models XS +claude claude-haiku-5-5 high +codex gpt-6.1-sol low + +$ quest models XS --writer claude-opus-5-5 +claude claude-opus-5-5 low +``` + +`--writer` prints the size's entry for that model, or else its entry at the +nearest size that lists it, the larger on a tie; a model no list names is an +error. `--json` prints an array of `{harness, model, effort}` objects. The +iterate records the size it was given in its `Implementer:` line, as +`Implementer: Claude Code, claude-opus-5-5, low effort (iterate, XS)`, so +the model check can audit the routing. `quest review` still reviews the +whole branch at the PR's size. + ## Review locally `quest review` has a model that wrote none of the code review the checkout's @@ -1108,7 +1154,7 @@ size. Each size ranks its reviewers, the same for every repository: | S | Codex `gpt-6.1-sol` medium | Claude Code `claude-opus-5-5` low | | M, or none | Codex `gpt-6.1-sol` medium | Claude Code `claude-opus-5-5` medium | | L | Codex `gpt-6.1-sol` high | Claude Code `claude-opus-5-5` high | -| XL | Codex `gpt-6.1-sol` xhigh | Claude Code `claude-opus-5-5` high | +| XL | Codex `gpt-6.1-sol` xhigh | Claude Code `claude-opus-5-5` xhigh | A quest branch takes its quest's size, read from the base branch's tip so a change cannot lower its own review; any other PR, such as a skill's own, takes diff --git a/assets/AGENTS.md b/assets/AGENTS.md index 2aec5d4c..3c8e7c07 100644 --- a/assets/AGENTS.md +++ b/assets/AGENTS.md @@ -175,6 +175,17 @@ Run no more sub-agents at once than the harness allows; queue the rest and start Relay no interim report: resume a sub-agent that ended its turn on one, telling it to block until its work finishes. A long-running coordinator lets compaction bound its context: it keeps what must outlast compaction in the forge and the quest tree, and reads a PR's state afresh before acting on it rather than trusting earlier turns. +Run each agent at an entry of `quest`'s ranked lists unless the repository's `AGENTS.md` names other models: as a native sub-agent on this session's harness, else through `quest agent`. +Size each `/quest-iterate` you dispatch as a quest, by the work left on its PR, and name that size in its brief. +Start it on the PR's writer, the model of its first `Implementer:` line, at the entry `quest models --writer ` prints, waiting out its usage limit so the other model can review; a PR with no known writer takes its size's first entry that can run. + +`quest agent start --harness --model --effort --permission-mode ` runs a headless agent on that CLI's own login; start it as a background command in its own isolated worktree, so the harness reports its exit as it would a sub-agent's. +`--permission-mode` is Claude Code's permission mode, or Codex's sandbox mode. +It prints the session id, then the outcome and the agent's final reply, and exits 3 at the account's usage limit, or 4 when the CLI is logged out, printing the login command. +Tell the agent that it cannot prompt, so it ends with its decisions and recommendations in its final reply, and that ending its turn kills its background commands, so it waits in the foreground. +Answer it in the background with `quest agent resume`, passing the same options, the worktree, the session id, and the answer. +Stop one with `quest agent stop `, and each you started that still runs before you finish; `--all` would stop other sessions' agents too. + ## Questions Open each prompt with the context to answer it without opening the quest or PR: what the work is, the problem it hit, and what each option would change. diff --git a/assets/CONTEXT.md b/assets/CONTEXT.md index 6ab89f6d..ec079b3b 100644 --- a/assets/CONTEXT.md +++ b/assets/CONTEXT.md @@ -97,7 +97,7 @@ Run the repository's checks locally while the PR is a draft; CI may run only on Before a PR is marked ready for review, and before each push to one that is, a model that wrote none of its code reviews the committed branch locally, as `/quest-review` describes. The PR body's `## Models` section records its size as `Size: M` (its quest's, or for a skill's own PR, as a quest is sized) and one `Implementer: Codex, gpt-6.1-sol, high effort` line per implementing session, added when it first pushes. -Read your model and effort rather than guess, writing `unknown` for any you cannot find: Claude Code knows its model and exports `CLAUDE_EFFORT`; Codex records `model` and `effort` in the last `turn_context` of `~/.codex/sessions/*/*/*/rollout-*-$CODEX_THREAD_ID.jsonl`; Grok records `model_id` and `reasoning_effort` in `~/.grok/sessions/*/$GROK_SESSION_ID/chat_history.jsonl`. +Read your model and effort rather than guess, writing `unknown` for any you cannot find: Claude Code knows its model and exports `CLAUDE_EFFORT`, and Codex records `model` and `effort` in the last `turn_context` of `~/.codex/sessions/*/*/*/rollout-*-$CODEX_THREAD_ID.jsonl`. GitHub gives secrets to workflows on branches pushed to the repository, as quest branches usually are, so keep pull request CI, which runs unreviewed code, read-only: a read-only `GITHUB_TOKEN`, and write credentials only in environments limited to protected branches. Flag any workflow that breaks this to the user, and never add one. diff --git a/assets/skills/iterate.md b/assets/skills/iterate.md index d131e1a1..b7b824d4 100644 --- a/assets/skills/iterate.md +++ b/assets/skills/iterate.md @@ -9,6 +9,7 @@ Parse the arguments to determine the PR number. Any other text is the user's fee Take it with `quest iterate ` before acting, passing a token you already hold for it to keep that hold, and keep the hold token it prints; `quest iterate --help` explains holds. If it refuses because another session holds the PR, stop and report what it printed; taking over a stale hold is the user's choice. If you cannot push to the PR's branch, stop and report. +Your `Implementer:` line ends with the size your brief gives the work, as `Implementer: Claude Code, claude-opus-5-5, low effort (iterate, XS)`. Read the PR, its linked issues and quest, the reviews and comments posted on it, and the surrounding code. `quest status ` reports its checks, reviews by head, and unresolved threads in one call. diff --git a/assets/skills/spawn.md b/assets/skills/spawn.md index e29bec1a..17cb26be 100644 --- a/assets/skills/spawn.md +++ b/assets/skills/spawn.md @@ -62,20 +62,9 @@ Resume that agent by messaging it, or with `quest agent resume` for a headless a If it cannot be resumed, start a new agent with the same brief plus its claim PR, its checkout, and the token of any hold it took on the PR, which `quest status ` shows, so the new one continues that work and keeps that hold rather than claiming again. While the error persists, such as a rate limit before its reset, resume and start nothing until it clears, and report it once rather than per agent. -## Other harnesses - -A sub-agent runs only this session's harness. -To start a quest on another, such as Codex from Claude Code when the user asks for it, run it as a headless agent through `quest agent`, which uses that CLI's own login. -Stop and report if it says the agent CLI is not installed. - -Start each agent as a background command, in its own isolated worktree, so the harness reports its exit as it would a sub-agent's: -`quest agent start --harness --model --effort --permission-mode `. -Pass the model and effort the user named; `--permission-mode` is Claude Code's permission mode, or Codex's sandbox mode. -It prints the session id first, then the outcome and the agent's final reply. -Tell the agent that it cannot prompt, so it ends with any decisions and its recommendations in its final reply, and that ending its turn ends the process and kills its background commands, so it waits in the foreground. -Ask the user, then resume that session in the background with the answer, as you would message a sub-agent: `quest agent resume` with the same options, the worktree, the session id, and the answer. -Stop one agent with `quest agent stop `. -When the spawn ends, run `quest agent stop --all`, which stops any agent still running. - -Exit status 3 means the harness's account hit its usage limit, and 4 that its CLI is logged out, printing the login command. -Start nothing more on that harness, report its outage once, and list each interrupted quest with its worktree and session id so the user can resume it later. +## Models + +Start each quest on the first entry of its `models` that can run. +Fall to the next when `quest agent` reports its CLI missing or logged out, or the agent stops on a usage limit with nothing written beyond the claim commit, continuing in that checkout and claim PR. +Once a model has written anything, the PR is its: list the quest with its worktree and session id to resume after the reset. +Skip a quest whose list runs out, and report each harness's outage once. diff --git a/assets/skills/unblock.md b/assets/skills/unblock.md index 7b40fec1..b6fff760 100644 --- a/assets/skills/unblock.md +++ b/assets/skills/unblock.md @@ -1,6 +1,6 @@ --- description: Walk the maintainer through every quest and draft PR waiting on them, and clear waits that have resolved. -guide: Format, Creation, Deletion, Summaries +guide: Format, Creation, Deletion, Coordination, Summaries skills: grill, plan --- diff --git a/quest/m0/model-routing/README.md b/quest/m0/model-routing/README.md index a5f83f07..dbc52dd2 100644 --- a/quest/m0/model-routing/README.md +++ b/quest/m0/model-routing/README.md @@ -5,16 +5,16 @@ `/quest-spawn` starts each ready quest, and each coordinator its `/quest-iterate` dispatches, on the first harness and model that can run from a ranked list `quest` builds in for the work's size, the same for every -repository unless its `AGENTS.md` overrides it: Opus writes at every size, -with Sol as the fallback through M, and a usage limit falls through to the -next entry. An iterate is sized by the work left on its PR, not its quest's +repository unless its `AGENTS.md` overrides it: Haiku writes XS and Opus +the rest, with Sol as the fallback through L, and a usage limit falls through +to the next entry. An iterate is sized by the work left on its PR, not its quest's original size, and runs on the model that first wrote the PR, so the other can always review it. Sol reviews, at an effort set by the PR's size, and Opus reviews whatever Sol wrote, so no model reviews its own code. Every PR records the models and efforts behind it, and fixes link the PR that caused them, so the rankings can be audited and retuned. This epic's own work is one live spawn in moq-dev/quest from Claude Code that starts an XS -and an S quest on Opus, each recorded and reviewed by Sol. +quest on Haiku and an S quest on Opus, each recorded and reviewed by Sol. Non-goals: routing kix.quest's runner jobs, which pick their own accounts and models (their PRs still record their models); billing agents to API @@ -81,19 +81,19 @@ are mostly judgment. - Revised 2026-11-05 (user, /quest-plan): Grok leaves routing and the council, so XS starts on Opus low; Sol reviews at the effort it would write at (XS medium, S high, M xhigh, L and XL xhigh); and coordinators - size each `/quest-iterate` by its remaining work. The tables and the - iterate decisions are in [Route by size](/quest/m0/model-routing/models.md). + size each `/quest-iterate` by its remaining work, starting it on the PR's + first writer, which waits out its own usage limit rather than fall back. - Revised again 2026-11-05 evening (user, /quest-plan): xhigh is XL's alone. Haiku high writes XS; Opus writes S low, M medium, L high, XL xhigh; Sol falls back one rung lower, now through L; Sol reviews follow [Single-pass review](https://github.com/moq-dev/quest/pull/514)'s - table. The tables and the benchmark evidence are in - [Route by size](/quest/m0/model-routing/models.md). + table. The lists live in `src/cli/models.ts` (`quest models`), and the + benchmark evidence in + [moq-dev/quest#555](https://github.com/moq-dev/quest/pull/555). - Which milestone? ✅ m0 (rejected: m1). It ranks first among m0's work after Kix, by the 2026-11-01 order: land work, then Kix, then the rest. ## Required -- [Route by size](/quest/m0/model-routing/models.md) - `/quest-spawn` and iterate dispatches start on the first entry of the size's ranked list that can run, an iterate sized by its remaining work - [kix records its follow-up sessions](/quest/m0/model-routing/kix-sessions.md) - kix `iterate` and `merge` jobs that change an existing PR record their `Implementer:` line in a PR comment - [moq-dev pins](/quest/m0/model-routing/pins.md) - moq-dev/moq and moq-dev/moq.pro bump their `quest` pin to pick up the routing diff --git a/quest/m0/model-routing/models.md b/quest/m0/model-routing/models.md deleted file mode 100644 index 17e44ef2..00000000 --- a/quest/m0/model-routing/models.md +++ /dev/null @@ -1,148 +0,0 @@ -# [L] Spawns and iterates route down a ranked list by size - -## Goal - -The `quest` binary ranks harnesses and models for each size: Opus writes the -code and Sol reviews it. `quest unblocked --json` reports each ready quest's -ranked list, and `/quest-spawn` starts the quest on the first entry that can -run: its CLI installed, logged in, and under its usage limit. A coordinator -that dispatches `/quest-iterate` (`/quest-complete`, `/quest-unblock`, -`/quest-merge`) sizes the work left on the PR, not the quest's original size, -and starts the iterate down that size's list, so a merge-and-record iterate on -an L quest runs at XS. A quest or iterate is skipped only when its whole list -is out. A repository's `AGENTS.md` may override the lists in prose; otherwise -every repository gets the same ones. - -## Plan - -Decided 2026-11-04 (user, /quest-plan); the evidence is in -[the epic](/quest/m0/model-routing/README.md). - -- Where does each quest's model come from? - - ✅ Lists built into `quest`, the same for every repository; a - repository's `AGENTS.md` overrides them in prose (user, replacing a - `MODELS.md` file per repository) - - A `MODELS.md` in each repository, parsed by `quest` - - Per quest, in its plan -- One model per size, or a ranked list? - - ✅ A ranked list, so a usage limit falls through to the next entry - (user, replacing "skip the quest and report the outage") -- Which models? ✅ Haiku writes XS and Opus writes S and up, with Sol as - the fallback through L; XL has no fallback and waits for Opus (user, - 2026-11-05, revised below; replacing the 2026-11-04 lists that gave XS to - Grok). -- Grok? ✅ Removed from routing and from every review (user, 2026-11-05: - its weekly usage is close to zero and its results were weak), so - `quest agent` gains no Grok harness; the "quest agent starts Grok" quest is - deleted. -- A quest with an open decision? ✅ Routes one size up (user). The signal: - a `Maintainer:` item under its `## Waiting`. An XL stays XL. - -The lists, pinned in `src/cli/models.ts` beside the review lists -`quest review` reads (user, 2026-11-05, revised the same evening below): - -| Size | Writes (spawn and iterate) | Reviews (`quest review`) | -|---|---|---| -| XS | claude `claude-haiku-5-5` high → codex `gpt-6.1-sol` low | codex `gpt-6.1-sol` low → claude `claude-opus-5-5` low | -| S | claude `claude-opus-5-5` low → codex `gpt-6.1-sol` medium | codex `gpt-6.1-sol` medium → claude `claude-opus-5-5` low | -| M | claude `claude-opus-5-5` medium → codex `gpt-6.1-sol` high | codex `gpt-6.1-sol` medium → claude `claude-opus-5-5` medium | -| L | claude `claude-opus-5-5` high → codex `gpt-6.1-sol` xhigh | codex `gpt-6.1-sol` high → claude `claude-opus-5-5` high | -| XL | claude `claude-opus-5-5` xhigh | codex `gpt-6.1-sol` xhigh → claude `claude-opus-5-5` xhigh | - -Opus takes a review only when Sol wrote the code, and never reviews XL in -practice, since Sol writes no XL. - -Revised 2026-11-05 evening (user, /quest-plan, while spawning the -[token waste](/quest/m0/token-waste/README.md) quests), replacing writes of -XS Opus low, S Opus medium, M Opus high, L and XL Opus xhigh, and Sol -fallbacks of medium, high, and xhigh for XS to M. Agents already running -keep the efforts they started with. Decision prompts, chosen options -marked ✅: - -- Goal: ✅ every size gets a cheaper writer, so xhigh is XL's alone. -- Opus writes: ✅ one rung lower, S low, M medium, L high, XL xhigh (the - user's proposal). -- XS writer: ✅ Haiku high, then Sol low (rejected: Haiku medium, Opus low, - Sol medium, DeepSeek through opencode). The user asked whether Haiku high - beats Opus low. No published benchmark measures Opus 5.5 at low effort. - [Bito's coding benchmark](https://bito.ai/blog/claude-haiku-5-5-vs-sonnet-5-5-vs-opus-5-5/) - (October 2026, judged by Opus) scored Haiku 5.5 high 87% on one-shot asks - and 82% on long builds against Opus 5.5 medium's 90% and 87%, at 1/4 to - 1/23 of the cost; Haiku medium scored 76-79% with about three times the - variance between sessions. Anthropic warns that low effort on long agent - prompts skips checks and follow-through. models.dev prices Haiku 5.5 at - $0.10/$0.50 per million input/output tokens, five times that for a - request over 100k tokens, against Opus 5.5's $4/$20. DeepSeek would need - an opencode harness and its sandbox in `quest agent`, which left on - 2026-11-04. The [model check](/quest/m1/model-check.md) moves XS back to - Opus if Haiku's review scores lag. -- Sol fallbacks: ✅ one rung lower, XS low, S medium, M high, and a new L - fallback at xhigh; XL keeps none (the user's pick; rejected: unchanged). -- Review efforts: ✅ [Single-pass review](https://github.com/moq-dev/quest/pull/514)'s - Sol column (XS low, S medium, M medium, L high, XL xhigh) replaces the - rule that a review runs at the effort Sol would write at, since that - decision ([moq-dev/quest#509](https://github.com/moq-dev/quest/pull/509)) - is newer than this quest's - ([moq-dev/quest#507](https://github.com/moq-dev/quest/pull/507)) and - carries the cost data. That quest's PR changes `REVIEWERS`; this one adds - the write lists. Opus's review efforts follow its write efforts - (rejected: the write list mirrors the reviews). - -Agent's choice, for the implementer to revisit: an iterate whose writer is -missing from its size's list runs at that writer's effort for the nearest -size that lists it, so an XS iterate on an Opus PR runs Opus low and an L -iterate on a Haiku PR runs Haiku high. - -Iterates, decided 2026-11-05 (user, /quest-plan, after a `/quest-complete` -run in moq-dev/moq sent many merge-and-record iterates): - -- Which list does an iterate use? ✅ The same write list, indexed by the - iterate's size (user). Not chosen: a separate iterate list. -- Who sizes it? ✅ The dispatching coordinator, with the quest rubric (the - agent's effort to finish, verify, and land what is left, judgment - included), from the PR's state: conflicts, unanswered findings, decisions - to apply, rework. It names the size in the brief, and the iterate's - `Implementer:` line in the PR's `## Models` section carries it, such as - `Implementer: Claude Code, claude-opus-5-5, low effort (iterate, XS)`, so - the [model check](/quest/m1/model-check.md) can audit it. Not chosen: - `quest` estimating it from those signals. -- What if two models have written a PR? ✅ They never do: one writer per - PR (user, 2026-11-05, from a CodeRabbit finding on - [moq-dev/quest#507](https://github.com/moq-dev/quest/pull/507)). Each - review list holds only Sol and Opus, and `quest review` excludes every - model that wrote the PR, so a PR both wrote would have no reviewer. A PR's - iterates run on the model that first wrote it, waiting out its usage limit - rather than falling back; the fallback applies only when spawning a fresh - quest. Not chosen: a person reviews such a PR; the model that wrote fewer - lines reviews it. -- Does the iterate's review shrink too? ✅ No: `quest review` reviews the - whole branch, so it keeps the PR's size (user). A merge-only change is - already exempt from review. -- Where does it live? ✅ This quest, resized from M to L (user). Not - chosen: a separate child. - -- Each entry names its model, so a review passes it to `quest review - --exclude-model` and never uses the model that wrote the code. -- `quest unblocked --json` adds the ranked list per quest, after the - decision bump. An epic whose README carries no size has none; the - spawner then inherits its own settings. -- `/quest-spawn`: follow the repository's `AGENTS.md` where it names other - models, and otherwise the list. Start a quest on a harness matching the - session's as a native sub-agent, and any other through `quest agent` on - the logged-in account. When `quest agent` cannot start an entry (CLI - missing, logged out) or the agent exits on a usage limit while its - checkout still holds nothing beyond the claim commit (no edits, no local - or pushed commits), move to the next entry, resuming from the claim PR and - checkout. Once a model has written anything there, the PR has its writer: - wait for it rather than fall back, as iterates do. With no - entry left, skip the quest. Report each harness's outage once. -- `/quest-complete`, `/quest-unblock`, and `/quest-merge`: size each - `/quest-iterate` they dispatch by its remaining work and start it on the - PR's writer (its first `Implementer:` model) at that size's effort, - natively or through `quest agent`; at that writer's usage limit, wait for - it rather than fall back. -- Remove Grok from what ships: its line in `assets/CONTEXT.md`'s model and - effort lookup, and any Grok harness code or test that exists by then. -- Docs: the README documents the lists, the decision bump, iterate sizing, - the override, and `quest unblocked --json`'s new field. The - [model check](/quest/m1/model-check.md) audits the lists each milestone. diff --git a/quest/m0/model-routing/pins.md b/quest/m0/model-routing/pins.md index 00440584..9fa92082 100644 --- a/quest/m0/model-routing/pins.md +++ b/quest/m0/model-routing/pins.md @@ -27,4 +27,4 @@ repository uses the same settings, so nothing in them changes but the pin. ## Required -- [Route by size](/quest/m0/model-routing/models.md) - `/quest-spawn` reads the built-in size table +- [kix records its follow-up sessions](/quest/m0/model-routing/kix-sessions.md) - the last other quest in the epic, which the bumped commit must hold diff --git a/src/cli/main.ts b/src/cli/main.ts index 094e850d..544346c8 100644 --- a/src/cli/main.ts +++ b/src/cli/main.ts @@ -54,7 +54,7 @@ import { watch, type WatchOptions } from "./land-watch"; import { landed } from "./landed"; import { type Report, fixLinks, moveQuest, removeQuest } from "./move"; import { type ResolveOptions, resolveFile } from "./resolve"; -import { DEFAULT_SIZE, REVIEWERS } from "./models"; +import { DEFAULT_SIZE, REVIEWERS, WRITERS, questWriters, writerAt } from "./models"; import { ReviewInputError, ReviewLimitError, @@ -256,7 +256,9 @@ export function main(argv: string[], out: Output): number | Promise { "Given a path, print what blocks that quest instead.\n\n" + "Counts exclude structural epic membership. Other prerequisites may remain for dependents. " + "Invalid trees fail before printing a ranking. Prints a count and path per line; --json prints an " + - "ordered array of {path, dependents} objects.\n\n" + + "ordered array of {path, dependents, models} objects, where models ranks the {harness, model, effort} " + + "entries that write the quest: its size's list, or the next size's while a Maintainer: item under its " + + "## Waiting leaves a decision open (quest models lists them).\n\n" + "--unclaimed ranks the fetched upstream default branch's tree instead of the checkout, then drops " + "quests someone holds: a quest branch upstream, or an open PR from that branch in any repository. A " + "branch whose tip is a merged PR's head holds nothing; a closed, unmerged PR's branch still does. A " + @@ -289,10 +291,15 @@ export function main(argv: string[], out: Output): number | Promise { explain(docs, find(docs, path), path); return; } - const show = (quests: RankedQuest[]) => { + const show = (quests: RankedQuest[], docs: Doc[]) => { const shown = quests.slice(0, options.limit); if (options.json) { - print(JSON.stringify(shown)); + const byPath = new Map(docs.map((doc) => [doc.path, doc])); + print( + JSON.stringify( + shown.map((quest) => ({ ...quest, models: questWriters(byPath.get(quest.path)!) })), + ), + ); return; } if (theme) @@ -301,7 +308,7 @@ export function main(argv: string[], out: Output): number | Promise { }; if (!options.unclaimed) { const tree = checkout(root(), warn); - if (valid(tree)) show(unblocked(tree.docs)); + if (valid(tree)) show(unblocked(tree.docs), tree.docs); return; } // Readiness comes from the upstream default branch, so a stale checkout cannot offer @@ -316,7 +323,7 @@ export function main(argv: string[], out: Output): number | Promise { const ranked = unblocked(tree.docs); const branches = ranked.map((quest) => questBranch(quest.path)); const holdings = up === null ? [] : (await held(root(), up, branches, refs?.branches)).claims; - return { ranked, holdings }; + return { ranked, holdings, docs: tree.docs }; })(), (result) => { if (result === null) return 1; @@ -327,7 +334,10 @@ export function main(argv: string[], out: Output): number | Promise { if (claim) warn(`skipped ${quest.path}: held by ${describe(claim.branch, claim.pull, now)}`); } - show(result.ranked.filter((quest) => !byBranch.has(questBranch(quest.path)))); + show( + result.ranked.filter((quest) => !byBranch.has(questBranch(quest.path))), + result.docs, + ); }, ); }, @@ -976,6 +986,37 @@ export function main(argv: string[], out: Output): number | Promise { ); }); + program + .command("models") + .summary("Print the harnesses, models, and efforts that write work of a size, ranked") + .description( + "Print the ranked entries that write work of a size, the same for every repository unless its AGENTS.md " + + "names others: " + + SIZES.map( + (size) => + `${size} ${WRITERS[size].map((e) => `${e.harness} ${e.model} ${e.effort}`).join(", then ")}`, + ).join("; ") + + ". A spawn starts a quest on the first entry that can run, as quest unblocked --json lists them. " + + "--writer prints the one entry that runs that model on the work instead, for an iterate, which stays " + + "on its PR's writer: the size's own entry for the model, else its entry at the nearest size that " + + "lists it, the larger on a tie, so an XS iterate on an Opus PR runs Opus at S's effort. A model no " + + "list names is an error. Prints harness, model, and effort per line, tab-separated; --json prints an " + + "array of {harness, model, effort} objects.", + ) + .addArgument(new Argument("", "The work's size").choices(SIZES)) + .option("--writer ", "Print only the entry that runs this model, the PR's first Implementer") + .option("--json", "Print a JSON array") + .action((size: Size, options: { writer?: string; json?: boolean }) => { + let entries = WRITERS[size]; + if (options.writer !== undefined) { + const entry = writerAt(size, options.writer); + if (entry === null) throw new UsageError(`no write list names ${options.writer}`); + entries = [entry]; + } + if (options.json) print(JSON.stringify(entries)); + else for (const e of entries) print(`${e.harness}\t${e.model}\t${e.effort}`); + }); + const reviewers = SIZES.map( (size) => `${size} ${REVIEWERS[size].map((e) => `${e.harness} ${e.model} ${e.effort}`).join(", then ")}`, ).join("; "); diff --git a/src/cli/models.ts b/src/cli/models.ts index 36114a7a..afa9bd6e 100644 --- a/src/cli/models.ts +++ b/src/cli/models.ts @@ -1,8 +1,8 @@ // The harnesses, models, and efforts `quest` runs for each size, the same for // every repository; a repository's AGENTS.md may override them in prose, which // the session following it passes as flags. The model check -// (quest/m0/model-check.md) audits these lists each milestone. -import type { Size } from "../core"; +// (quest/m1/model-check.md) audits these lists each milestone. +import { type Doc, SIZES, type Size, onMaintainer, size, waits } from "../core"; import type { HarnessName } from "./agent"; /** One harness running one model at one effort. */ @@ -17,16 +17,56 @@ export const DEFAULT_SIZE: Size = "M"; const sol = (effort: string): Model => ({ harness: "codex", model: "gpt-6.1-sol", effort }); const opus = (effort: string): Model => ({ harness: "claude", model: "claude-opus-5-5", effort }); +const haiku = (effort: string): Model => ({ harness: "claude", model: "claude-haiku-5-5", effort }); + +/** + * Each size's writers, ranked: a spawn starts the quest on the first that can + * run, falling through only while nothing is written, since the model that + * writes a PR runs all its iterates. + */ +export const WRITERS: Record = { + XS: [haiku("high"), sol("low")], + S: [opus("low"), sol("medium")], + M: [opus("medium"), sol("high")], + L: [opus("high"), sol("xhigh")], + XL: [opus("xhigh")], +}; /** * Each PR size's reviewers, ranked: the first whose model wrote none of the - * code reviews it, so Opus reviews what Sol wrote. Sol's efforts are the - * user's; Opus's are its review efforts from before the lists were ranked. + * code reviews it, so Opus reviews what Sol wrote, at its own write efforts. */ export const REVIEWERS: Record = { XS: [sol("low"), opus("low")], S: [sol("medium"), opus("low")], M: [sol("medium"), opus("medium")], L: [sol("high"), opus("high")], - XL: [sol("xhigh"), opus("high")], + XL: [sol("xhigh"), opus("xhigh")], }; + +/** + * The writers that start quest `doc`: its size's list, or the next size's + * while a `Maintainer:` wait leaves it a decision to settle. + */ +export function questWriters(doc: Doc): readonly Model[] { + const sized = size(doc); + // `quest check` requires a size of every quest that is not an epic, and only those can be ready. + if (sized === null) throw new Error(`${doc.path} has no size`); + const bump = waits(doc).some(onMaintainer) ? 1 : 0; + return WRITERS[SIZES[Math.min(SIZES.indexOf(sized) + bump, SIZES.length - 1)]]; +} + +/** + * The entry that runs `model` on work of `size`: its own in that size's list, + * else its entry at the nearest size that lists it, the larger on a tie, or + * null when no list names it. + */ +export function writerAt(size: Size, model: string): Model | null { + const at = SIZES.indexOf(size); + const listed = SIZES.flatMap((each, index) => { + const entry = WRITERS[each].find((e) => e.model === model); + return entry ? [{ entry, distance: Math.abs(index - at), index }] : []; + }); + listed.sort((a, b) => a.distance - b.distance || b.index - a.index); + return listed[0]?.entry ?? null; +} diff --git a/test/cli.test.ts b/test/cli.test.ts index 63850a9d..234a1660 100644 --- a/test/cli.test.ts +++ b/test/cli.test.ts @@ -152,9 +152,11 @@ test("a document that is not UTF-8 is an error", () => { test("unblocked prints ranked counts and paths, with deterministic JSON and a limit", () => { const tree = Tree.baseline(); expect(tree.run("unblocked")).toEqual({ code: 0, stdout: "1\tquest/m0/epic/one.md\n", stderr: "" }); + const opusLow = '{"harness":"claude","model":"claude-opus-5-5","effort":"low"}'; + const solMedium = '{"harness":"codex","model":"gpt-6.1-sol","effort":"medium"}'; expect(tree.run("unblocked", "--json", "--limit", "1")).toEqual({ code: 0, - stdout: '[{"path":"quest/m0/epic/one.md","dependents":1}]\n', + stdout: `[{"path":"quest/m0/epic/one.md","dependents":1,"models":[${opusLow},${solMedium}]}]\n`, stderr: "", }); tree.write("quest/m0/epic/two.md", "# [S] Two\n\n## Goal\n\nWork.\n"); @@ -162,6 +164,41 @@ test("unblocked prints ranked counts and paths, with deterministic JSON and a li expect(JSON.parse(tree.run("unblocked", "--json").stdout)).toHaveLength(2); }); +test("unblocked routes a quest with a decision open one size up, and XL stays XL", () => { + const models = (tree: Tree) => + JSON.parse(tree.run("unblocked", "--json").stdout).map((quest: { models: { effort: string }[] }) => + quest.models.map((entry) => entry.effort).join(" "), + ); + const tree = Tree.baseline().append("quest/m0/epic/one.md", "\n## Waiting\n\n- moq-dev/moq: a release.\n"); + expect(models(tree)).toEqual(["low medium"]); + tree.append("quest/m0/epic/one.md", "- Maintainer: pick a name.\n"); + expect(models(tree)).toEqual(["medium high"]); + tree.write("quest/m0/epic/one.md", "# [XL] One\n\n## Goal\n\nWork.\n\n## Waiting\n\n- Maintainer: pick.\n"); + expect(models(tree)).toEqual(["xhigh"]); +}); + +test("models ranks a size's writers, or the one entry that runs a PR's writer", () => { + const tree = Tree.baseline(); + expect(tree.run("models", "XS")).toEqual({ + code: 0, + stdout: "claude\tclaude-haiku-5-5\thigh\ncodex\tgpt-6.1-sol\tlow\n", + stderr: "", + }); + expect(JSON.parse(tree.run("models", "XL", "--json").stdout)).toEqual([ + { harness: "claude", model: "claude-opus-5-5", effort: "xhigh" }, + ]); + const writer = (size: string, model: string) => tree.run("models", size, "--writer", model).stdout; + expect(writer("M", "gpt-6.1-sol")).toBe("codex\tgpt-6.1-sol\thigh\n"); + // A writer its size does not list runs at the nearest size that lists it. + expect(writer("XS", "claude-opus-5-5")).toBe("claude\tclaude-opus-5-5\tlow\n"); + expect(writer("L", "claude-haiku-5-5")).toBe("claude\tclaude-haiku-5-5\thigh\n"); + expect(writer("XL", "gpt-6.1-sol")).toBe("codex\tgpt-6.1-sol\txhigh\n"); + const unknown = tree.run("models", "S", "--writer", "unknown"); + expect(unknown.code).toBe(2); + expect(unknown.stderr).toContain("no write list names unknown"); + expect(tree.run("models", "XXL").code).toBe(2); +}); + test("unblocked explains the count on a terminal, including NO_COLOR", () => { const result = Tree.baseline().runOn({ colour: false }, "unblocked"); expect(result.code).toBe(0); diff --git a/test/review.test.ts b/test/review.test.ts index c0706734..6f1d65ed 100644 --- a/test/review.test.ts +++ b/test/review.test.ts @@ -306,13 +306,13 @@ test("each size's reviewer is the first whose model wrote none of the code", () expect(at("S", codex)).toBe("claude claude-opus-5-5 low"); expect(at("M", codex)).toBe("claude claude-opus-5-5 medium"); expect(at("L", codex)).toBe("claude claude-opus-5-5 high"); - expect(at("XL", codex)).toBe("claude claude-opus-5-5 high"); + expect(at("XL", codex)).toBe("claude claude-opus-5-5 xhigh"); // The PR's recorded implementers replace the running harness: Sol reviews Opus's code from Claude Code. expect(at("S", authored({ exclude: ["claude-opus-5-5"] }, { CLAUDECODE: "1" }))).toBe("codex gpt-6.1-sol medium"); expect(at("S", authored({ exclude: ["gpt-6.1-sol"] }, { CLAUDECODE: "1" }))).toBe("claude claude-opus-5-5 low"); // No harness need be detected once the implementers are named. - expect(at("M", authored({ exclude: ["grok-4.7"] }, {}))).toBe("codex gpt-6.1-sol medium"); + expect(at("M", authored({ exclude: ["other-model"] }, {}))).toBe("codex gpt-6.1-sol medium"); const both = authored({ exclude: ["gpt-6.1-sol", "claude-opus-5-5"] }, {}); expect(() => pickReviewer("M", {}, both)).toThrow("wrote the code; pass --harness and --model");