diff --git a/quest/m0/README.md b/quest/m0/README.md index 0b670c65..7156348f 100644 --- a/quest/m0/README.md +++ b/quest/m0/README.md @@ -43,6 +43,21 @@ Decided 2026-10-30 (user, /quest-plan), replacing "First public release": The land work is what moq-dev's repositories use every day, and the maintainer ranked it first in the 2026-10-30 and 2026-11-01 sessions. +- How do moq-dev's repositories land? Decided 2026-11-01 (maintainer, + /quest-plan), reversing the 2026-10-26 move to a batched gate: + - ✅ Through GitHub's merge queue again, one PR at a time, with no batching + and no pre-checks. A host watcher runs the Codex land fixer on each PR the + queue drops and re-enqueues it, so only decisions reach a person. The + batched gate was left for GitHub's inability to resolve quest-file + conflicts (71% of moq's 246 removals); the async fixer now absorbs those. + - Keep the host queue, serial + + The [queue watcher](/quest/m0/queue-watcher/README.md) ranks first and + records the full interview: moq first, self-hosted `merge_group` runners, + host-gate work frozen, build concurrency 1, re-enqueue at the back, + unlimited attempts while progressing, flaky re-enqueues, and fixing picks + GitHub refuses. The land work above now means the watcher first, then the + fixer and its host. Decided while planning m0 (2026-09-26), replacing the earlier copied-files and ownership-manifest design: @@ -67,13 +82,11 @@ ownership-manifest design: ## Required -- [Merge through a host gate](/quest/m0/host-land/README.md) - inline merge picks, small batches landed from the maintainer's host, and a Codex fixer instead of bisection -- [The land fixer handles every mechanical failure](/quest/m0/land-fixer/README.md) - attempts, registry network, triage, flaky tests, local mergeability, protected edits, and fork commits +- [Land through the merge queue with an async fixer](/quest/m0/queue-watcher/README.md) - GitHub's merge queue lands one PR at a time, and a host watcher fixes and re-enqueues each PR it drops +- [Merge through a host gate](/quest/m0/host-land/README.md) - the host's fixer containers, and the frozen batched gate for the repositories not yet switched +- [The land fixer resolves every mechanical failure](/quest/m0/land-fixer/README.md) - lockfile merges, protected edits as ready commits, workflow pushes, and the PR's own CI as context - [Landing priority](/quest/m0/land-priority.md) - cheap checks wait a bounded time and the land daemon's check runs ahead of agent builds - [Drop emptied Required headings](/quest/m0/required-heading.md) - a clean driver merge that empties a `Required` list passes `quest check` -- [Current conflict reports](/quest/m0/land-conflicts-refresh.md) - a queued PR's conflict report updates when the queue ahead of it changes -- [Land checks at gate priority](/quest/m0/land-priority-sandbox.md) - `quest land serve` passes `QUEST_GATE_PRIORITY` to its sandbox -- [Squash mismatch comment](/quest/m0/land-squash-mismatch.md) - a post-squash `tree_mismatch` posts a CRITICAL comment on the merged PR - [Current landing status](/quest/m0/status-landing.md) - `quest status` drops a stale `landing-stopped` line and ignores superseded cancelled checks - [Kix decision UI](/quest/m0/kix/README.md) - planning and implementation through prompts and summaries, run on the maintainer's own runners - [Codex cloud off](/quest/m0/codex-cloud-off.md) - automatic Codex cloud reviews stop once no open PR relies on them diff --git a/quest/m0/helpers/README.md b/quest/m0/helpers/README.md index a980ccdb..b8448124 100644 --- a/quest/m0/helpers/README.md +++ b/quest/m0/helpers/README.md @@ -72,4 +72,3 @@ commands from Sep-Oct. Session counts in each child come from that window. - [quest merge --land gaps](/quest/m0/helpers/merge-land.md) - squash on squash-only repositories, PR URLs outside a checkout, labels released after direct merges, and a warning when auto-merge outlives its head - [Headless agents](/quest/m0/helpers/headless-agents.md) - `/quest-spawn api` launches, resumes, and stops billed agents through one command auto mode allows -- [moq-dev/quest lands through quest land](/quest/m0/helpers/land-adopt.md) - this repository's `main` changes only through its own gate diff --git a/quest/m0/helpers/land-adopt.md b/quest/m0/helpers/land-adopt.md deleted file mode 100644 index 6a1830d4..00000000 --- a/quest/m0/helpers/land-adopt.md +++ /dev/null @@ -1,46 +0,0 @@ -# [S] moq-dev/quest lands through quest land - -## Goal - -PRs to moq-dev/quest's `main` land only through -`quest land`, on a hosted runner, running -`just check` and `just test` once per batch. GitHub's merge button cannot -land around it. A PR from a fork lands through it too. - -## Plan - -Decided 2026-10-26 (user): the repository adopts its own gate first. - -[moq-dev/quest#236](https://github.com/moq-dev/quest/pull/236) added the -Land workflow and `CONTRIBUTING.md`'s landing rule. The setup was finished on -2026-10-29, and [moq-dev/quest#283](https://github.com/moq-dev/quest/pull/283) -landed through the gate on 2026-10-30, verifying the same-repository path. -What remains is the fork path. - -- **Push credential (user, 2026-10-30).** The repository moved to the moq-dev - organization, where moq-bot is installed on every repository and has - Workflows write, which lets it land PRs that change `.github/workflows`. - No secret holds its key here: the land job trades its GitHub ID token at - `token.moq.dev` ([moq-dev/moq#7275](https://github.com/moq-dev/moq/pull/7275)), - which issues a moq-bot token scoped to this repository only for its - `land.yml` on `main` ([moq-dev/quest#284](https://github.com/moq-dev/quest/pull/284)). - So no other job may request an ID token, which `test/workflows.test.ts` - guards. -- **Ruleset (done 2026-10-29).** `main`'s ruleset blocks deletion and non-fast-forward - pushes and requires two checks: `check` from GitHub Actions, and - `quest land` pinned to moq-bot, which has no checks or statuses - permission and so never reports it. Its bypass actors are moq-bot and - the admin role, which keeps an emergency bypass. It replaced the classic - protection that required `check`. -- **Forks (user, 2026-10-27).** The fork path was never verified live. This - quest is done only once a fork PR lands through the host gate's squash-merge - API and shows as merged; any fix it needs lands here. The API merge needs - no maintainer edits to the fork. -- **Host gate (user, 2026-10-30).** The gate moves to the maintainer's host - ([gate host](/quest/m0/host-land/host.md)), which merges through the API with moq-bot's - App key instead of the `token.moq.dev` exchange above, and land.yml goes - away. Verify the fork path on the host gate. - -## Required - -- [Gate host](/quest/m0/host-land/host.md) - the live gate the fork path is verified on diff --git a/quest/m0/host-land/README.md b/quest/m0/host-land/README.md index f2d51578..92bf55bd 100644 --- a/quest/m0/host-land/README.md +++ b/quest/m0/host-land/README.md @@ -1,22 +1,20 @@ -# Merge through a host gate +# Merge through a host gate until the merge queue replaces it ## Goal -Picking a PR to merge starts no long-lived agent. `/quest-merge` queues it -inline only when a counted review names its head, or the head is trivial; -otherwise `/quest-iterate` runs first. A gate on the maintainer's host lands -small batches: a base PR plus up to floor(log₂ n) others that merge cleanly, -checked once on the chain's tip and pushed as one squash commit per PR. A -conflict on the base PR or a failing batch hands the PRs to an asynchronous -Codex fixer, which makes only mechanical fixes and requeues the healthy PRs. -Bisection goes away. Since 2026-11-01 those PRs stay queued and parked while -the fixer works, and only a handback takes one out of the queue; see the -[land fixer epic](/quest/m0/land-fixer/README.md). +The maintainer's host runs the land fixer in ephemeral nspawn containers for +the [queue watcher](/quest/m0/queue-watcher/README.md), and keeps landing the +repositories not yet on GitHub's merge queue (moq-dev/quest, moq.pro, rust2 +and rind) through the host gate until they +[follow](/quest/m0/queue-watcher/follow.md). The gate is frozen: it gets only +fixes the switch needs, and is +[retired](/quest/m0/queue-watcher/retire.md) once nothing lands through it. -Non-goals: other repositories adopting the gate (moq-dev/moq, moq-dev/moq.pro -plan their own; moq.pro needs only `allow_auto_merge` and the ruleset, since -moq-dev is on GitHub Team), and changing the review policy beyond the -mechanical-fix exemption. +`/quest-merge` queues a pick inline only when a counted review names its head, +or the head is trivial; otherwise `/quest-iterate` runs first. That part of +the 2026-10-30 design stands. + +Non-goals: changing the review policy beyond the mechanical-fix exemption. ## Plan @@ -95,9 +93,20 @@ can hand off instead. ([moq-dev/quest#355](https://github.com/moq-dev/quest/pull/355)) went further: nothing is ejected for a failure, and the blamed PR is parked for the fixer. +- Superseded 2026-11-01 (maintainer, /quest-plan; see the + [queue watcher](/quest/m0/queue-watcher/README.md)): each repository lands + through GitHub's merge queue again, one PR at a time, and a host watcher + runs this epic's fixer on each PR the queue drops and re-enqueues it. + moq-dev/moq moves first. The batched gate described above keeps serving the + other repositories until they follow, with its work frozen except for fixes + the switch needs. Its end-to-end batch run and its docs are dropped; the + watcher epic has its own. Deleted with it: keeping an in-flight batch check + across daemon restarts, current queue-conflict reports, the post-squash + mismatch comment, and gate priority for batch checks, all host-gate + improvements the freeze stops. + ## Required - [Gate host](/quest/m0/host-land/host.md) - kixelated/nixos runs the daemon as its own user, checks in nspawn containers, and retires the Actions runner - [Fixer proxy access](/quest/m0/host-land/fixer.md) - connect the nixos fixer container to CLIProxyAPI with the gate client key and maintainer login - [Fixer TLS](/quest/m0/host-land/fixer-tls.md) - the fixer's container trusts public certificates, so workerd and the Worker tests run there -- [Daemon restart keeps its check](/quest/m0/host-land/reattach.md) - a deploy no longer reruns the batch it interrupted diff --git a/quest/m0/host-land/fixer.md b/quest/m0/host-land/fixer.md index 0f8fc86c..4ba4fbd5 100644 --- a/quest/m0/host-land/fixer.md +++ b/quest/m0/host-land/fixer.md @@ -4,7 +4,8 @@ The nixos fixer container reaches CLIProxyAPI with the gate's client key, while Codex login and Git/GitHub credentials stay outside the container. Verify a real -mechanical fix and unchanged requeue through the host daemon. +mechanical fix and unchanged requeue through the +[queue watcher](/quest/m0/queue-watcher/README.md) or the host daemon. ## Plan @@ -23,7 +24,8 @@ back for them. - ✅ Keep as decided The quest CLI implements `quest land fix`, durable collection, validation, -lease-pushes, Review records, and one attempt per maintainer pick. Host work +lease-pushes, Review records, and one attempt per maintainer pick (the +watcher's [stop rules](/quest/m0/queue-watcher/stop.md) replace that limit). Host work remains in kixelated/nixos: the ephemeral fixer container (`qland-fix`) reaches CLIProxyAPI and, like the check containers, the public internet without private ranges (`modules/ci-egress.nix`), so its builds fetch dependencies; diff --git a/quest/m0/host-land/host.md b/quest/m0/host-land/host.md index 7f355a18..c5343c18 100644 --- a/quest/m0/host-land/host.md +++ b/quest/m0/host-land/host.md @@ -106,8 +106,11 @@ What remains: quest-land-fixer.socket canceled" when several `*-socket-refresh` services restart at once (user, 2026-10-31). Find the race and fix it, without retrying the test. -- Once [moq-dev/quest#324](https://github.com/moq-dev/quest/pull/324) lands - (user, 2026-10-31): export `QUEST_GATE_PRIORITY=1` in the generated - `quest-land-check-command`, then bump the pinned quest, so a land check's - nested `quest gate` waits for memory but not load. Don't use - `QUEST_GATE_ADMITTED`, which also skips the memory check. + +Amended 2026-11-01 (maintainer, /quest-plan; see the +[queue watcher](/quest/m0/queue-watcher/README.md)): host-gate work is frozen +except fixes the switch needs. The `QUEST_GATE_PRIORITY=1` export in the +generated check command (planned 2026-10-31) is dropped with it. The items +above stay, since the fixer container and its launcher serve the watcher too, +and the gate keeps serving the repositories not yet switched. The watcher's +own unit is [Watcher on the host](/quest/m0/queue-watcher/host.md). diff --git a/quest/m0/host-land/reattach.md b/quest/m0/host-land/reattach.md deleted file mode 100644 index 154e439f..00000000 --- a/quest/m0/host-land/reattach.md +++ /dev/null @@ -1,28 +0,0 @@ -# [M] A restarted land daemon keeps its in-flight check - -## Goal - -Restarting `quest land serve`, as every host deploy does, never discards a -running batch check. The new daemon picks up the check the old one started -and lands, ejects, or hands back from its result, so a deploy costs no -rerun of a 30-40 minute batch. - -## Plan - -Decided 2026-10-31 (user, /quest-plan of the follow-ups from moq.pro's and rust2's cutover to the host gate and the fixer repairs [moq-dev/quest#320](https://github.com/moq-dev/quest/pull/320), [#325](https://github.com/moq-dev/quest/pull/325), [#327](https://github.com/moq-dev/quest/pull/327) and [#328](https://github.com/moq-dev/quest/pull/328)). - -- On 2026-10-31 each of six deploys killed moq's and moq.pro's in-flight - checks: the check client is the daemon's child, and with - `KillMode=process` it still exits on its next write to the dead daemon's - pipe. No PR was ejected, but each batch reran from scratch. -- Fixers already survive this way: a detached supervisor records the job and - the next daemon collects it (`src/cli/land-fixer.ts`). Give a batch check - the same shape: a detached client that writes its log and exit status to - the state directory, and a daemon that resumes a recorded batch on start - instead of building a new one. -- The host side (the check launcher, `KillMode=process`) is the - [gate host](/quest/m0/host-land/host.md)'s. - -## Related - -- [Gate host](/quest/m0/host-land/host.md) - the units this runs under diff --git a/quest/m0/land-conflicts-refresh.md b/quest/m0/land-conflicts-refresh.md deleted file mode 100644 index 2dd7447a..00000000 --- a/quest/m0/land-conflicts-refresh.md +++ /dev/null @@ -1,18 +0,0 @@ -# [S] quest land reports queue conflicts again when the queue changes - -## Goal - -A queued PR's conflict report stays current: when a PR ahead of it lands, -is requeued, or changes head, `quest land` checks it against the queue again -and updates its single report, saying when a conflict has cleared. - -## Plan - -Decided 2026-10-31 (user, /quest-plan of the follow-ups from moq.pro's and rust2's cutover to the host gate and the fixer repairs [moq-dev/quest#320](https://github.com/moq-dev/quest/pull/320), [#325](https://github.com/moq-dev/quest/pull/325), [#327](https://github.com/moq-dev/quest/pull/327) and [#328](https://github.com/moq-dev/quest/pull/328)). - -- Starts once [moq-dev/quest#322](https://github.com/moq-dev/quest/pull/322) - lands. It reports a newly queued PR's conflicts with the PRs ahead of it, - once per queueing, so a PR behind a requeued or changed one is never - checked again. -- Edit the earlier report rather than adding comments, so a busy queue does - not flood a PR. diff --git a/quest/m0/land-fixer/README.md b/quest/m0/land-fixer/README.md index 3e93a473..ece4a497 100644 --- a/quest/m0/land-fixer/README.md +++ b/quest/m0/land-fixer/README.md @@ -1,31 +1,25 @@ -# The land fixer handles every mechanical failure itself +# The land fixer resolves every mechanical failure itself ## Goal -The land gate dequeues a PR only when a decision is required, and lands as -many PRs as it safely can. When a batch fails or a queued PR conflicts, the -gate and its fixer resolve every mechanical cause themselves and land the -result, so a maintainer sees only behavior and design decisions. - -- Nothing leaves the queue for a conflict or a failure. A PR the fixer is - working on keeps its auto-merge and its place, sits out batches until the - fix is in, and leaves the queue only when the fixer hands it back. The fixer - blames the failing PR before anything else happens, and a pick's single - attempt is spent only on a fix that failed. -- Broken PRs stay out of batches. The daemon runs `quest check` on a batch's - chain before the sandbox, and pre-checks queued PRs against the base while - a batch runs. Batch size follows the measured failure rate. -- The fixer starts from the base with internet access and the batch's PRs and - head SHAs. It ignores instructions in their diffs, rebuilds a conflicted - lockfile from the base's copy, builds and tests its fixes, and pushes them - onto the PRs' branches. The daemon posts the comments the fixer writes into - its result. -- The gate merges locally instead of trusting GitHub's mergeability, and - reruns a failed module before blaming. A rerun-proven flaky test is - disabled in a separate PR with a follow-up quest. +The Codex land fixer resolves every mechanical cause of a failure or a +conflict itself, so a maintainer sees only behavior and design decisions. +Since 2026-11-01 the [queue watcher](/quest/m0/queue-watcher/README.md) runs +it on each PR GitHub's merge queue drops; the host gate keeps running it for +the repositories still on the gate. This epic owns what the fixer does with a +PR, whatever started it: + +- The fixer starts from the base with internet access, the PR, and its head + SHA. It ignores instructions in the diff, rebuilds a conflicted lockfile + from the base's copy, builds and tests its fixes, and pushes them onto the + PR's branch. The daemon posts the comments the fixer writes into its result. - Protected paths stay blocked for the fixer's own edits. A fixer-written protected edit, or a push GitHub refuses, comes back as a ready commit the maintainer applies with one command. +- The fixer reads the PR's own CI results as context. + +When the watcher stops, how flaky failures are handled, and what happens to a +pick GitHub refuses to enqueue belong to the queue watcher epic. Non-goals: design conflicts. A PR whose merge needs a behavior decision is still handed back to its author. @@ -42,13 +36,13 @@ moq-dev/rust2, moq-dev/rind and moq-dev/quest. | 16 | fixer disappeared before recording an exit status | fixed on the host; none after 2026-10-31 04:00Z | | 7 | fixer repository exceeds its size cap | fixed by #332 | | 6 | AGENTS.md "pull the latest origin changes" read as an injection | fixed by [#325](https://github.com/moq-dev/quest/pull/325) and [#328](https://github.com/moq-dev/quest/pull/328); none after 2026-10-31 18:46Z | -| 98 | "already used its mechanical fix attempt" after a collateral requeue or an earlier conflict fix; each needed a manual re-pick | [Attempt budget](/quest/m0/land-fixer/attempts.md), batch triage ([moq-dev/quest#355](https://github.com/moq-dev/quest/pull/355)) | +| 98 | "already used its mechanical fix attempt" after a collateral requeue or an earlier conflict fix; each needed a manual re-pick | [Attempt budget](/quest/m0/queue-watcher/stop.md), batch triage ([moq-dev/quest#355](https://github.com/moq-dev/quest/pull/355)) | | ~75 | "requeued unchanged": collateral of a batch another PR broke | batch triage ([moq-dev/quest#355](https://github.com/moq-dev/quest/pull/355)) | | 36 | picked again while fixing; the newer pick is kept, so nothing is lost | not planned | | 17 | conflicts in protected paths (`flake.nix`, `.github/`), mostly mechanical; resolved by hand on moq-dev/moq#7344, #7315, #7318 | blocked by design; fixer-written protected edits become a ready commit ([Protected edits as ready commits](/quest/m0/land-fixer/protected-commit.md)) | | 17 of the last 40 jobs | `cargo` fails offline ("failed to get `uniffi`"), so Rust fixes were requeued uncompiled: moq-dev/moq#7357 sank three batches, and #6985's lockfile could not be regenerated | fixed by [moq-dev/quest#356](https://github.com/moq-dev/quest/pull/356): the prompt allows dependency fetches and requires a build and test | | 2 | GitHub refused the push to a fork because the merge touches workflows: moq-dev/moq#7439, #7467 | handed back with a ready commit for `quest land apply` by [moq-dev/quest#358](https://github.com/moq-dev/quest/pull/358) | -| several batches | a flaky ffmpeg timeout ([moq-dev/moq#7480](https://github.com/moq-dev/moq/pull/7480)) ejected whole batches | [Flaky tests](/quest/m0/land-fixer/flaky.md) | +| several batches | a flaky ffmpeg timeout ([moq-dev/moq#7480](https://github.com/moq-dev/moq/pull/7480)) ejected whole batches | [Flaky failures](/quest/m0/queue-watcher/flaky.md) | | ~15 | design conflicts, correctly handed back | none | Decision prompts, selected options marked ✅: @@ -57,9 +51,9 @@ Decision prompts, selected options marked ✅: - Yes, as stated (triage and local mergeability stay separate) - Narrower: only the attempt budget and offline builds - ✅ Broader: also fold batch triage ([moq-dev/quest#355](https://github.com/moq-dev/quest/pull/355)) - and [Local mergeability](/quest/m0/land-fixer/mergeability.md) into + and [Local mergeability](/quest/m0/queue-watcher/bad-picks.md) into this epic -- How attempts count: see [Attempt budget](/quest/m0/land-fixer/attempts.md). +- How attempts count: see [Attempt budget](/quest/m0/queue-watcher/stop.md). - How the fixer gets dependencies: - Host prefetches the job's lockfile dependencies and mounts them read-only - ✅ Allow registry network. The host already gave the fixer container @@ -71,7 +65,7 @@ Decision prompts, selected options marked ✅: - Protected paths: the fixer first could edit them (moq-dev/quest#357), then not; see the 2026-11-01 audit below. - Fork pushes GitHub refuses: ✅ hand back a ready commit, which `quest land apply` rebuilds for a maintainer to push (Grant moq-bot the `workflows` permission · Leave it manual). -- Flaky tests: see [Flaky tests](/quest/m0/land-fixer/flaky.md). +- Flaky tests: see [Flaky failures](/quest/m0/queue-watcher/flaky.md). - Milestone for the new quests: ✅ m0 · m1 - Restated 2026-11-01 (user, a second /quest-plan): "the merge queue can autonomously fix most issues. The fixer inspects the batch and tries to @@ -107,7 +101,7 @@ Decision prompts, selected options marked ✅: disables auto-merge. Batch triage ([moq-dev/quest#355](https://github.com/moq-dev/quest/pull/355)) built this parking: a PR is parked while a fixer job holds it (`src/cli/land-park.ts`), and - [Local mergeability](/quest/m0/land-fixer/mergeability.md) follows the same + [Local mergeability](/quest/m0/queue-watcher/bad-picks.md) follows the same rule. - Audit of the merge-queue plan, 2026-11-01 (an independent read-only agent; the maintainer then decided each point). On moq over 24 hours: 128 @@ -122,15 +116,32 @@ Decision prompts, selected options marked ✅: the daemon's markers, `quest status`) · keep the token - Flaky tests: ✅ disable only a rerun-proven flaky test, in a separate PR · keep the fixer's judgment and the base PR - - New work, all picked: ship [Attempt budget](/quest/m0/land-fixer/attempts.md) - right after triage; [the batch check](/quest/m0/land-fixer/batch-check.md); - [pre-checks](/quest/m0/land-fixer/precheck.md); and - [tuning](/quest/m0/land-fixer/tuning.md) + - New work, all picked: ship the attempt budget (now + [Stop rules](/quest/m0/queue-watcher/stop.md)) right after triage; the + batch check ([moq-dev/quest#379](https://github.com/moq-dev/quest/pull/379)); + pre-checks ([#377](https://github.com/moq-dev/quest/pull/377)); and + tuning ([#376](https://github.com/moq-dev/quest/pull/376)). All three were + closed unmerged by the merge-queue switch below. - Amended 2026-11-01 (maintainer, /quest-plan in moq-dev/moq): a fixer merge on [moq-dev/moq#7318](https://github.com/moq-dev/moq/pull/7318) rebuilt `bun.lock` from the PR's side and undid the base's advisory fix; see [Lockfile merges](/quest/m0/land-fixer/lockfile.md). +- Superseded 2026-11-01 (maintainer, /quest-plan; see the + [queue watcher](/quest/m0/queue-watcher/README.md)): repositories land + through GitHub's merge queue one PR at a time, and a host watcher runs this + fixer on each PR the queue drops and re-enqueues it. Batching is gone, so + this epic keeps only the fixer's own quality work. Moved to the watcher: + the attempt budget (now [Stop rules](/quest/m0/queue-watcher/stop.md)), + local mergeability and CI-failure parking (now + [Bad picks](/quest/m0/queue-watcher/bad-picks.md)), and flaky tests (now + [Flaky failures](/quest/m0/queue-watcher/flaky.md), without the rerun). + Deleted: the batch quest check, pre-checks, and throughput tuning, which + only served batches (the fixer cap and reasoning effort moved to + [Fix and re-enqueue](/quest/m0/queue-watcher/fix.md)); and queue status in + the required check, since GitHub's merge queue shows each PR's position + itself. + The README's landing section and `CONTRIBUTING.md` list what the fixer resolves itself and what it hands back. That docs update is the epic's own work, done once the Required quests below have merged. @@ -138,18 +149,11 @@ work, done once the Required quests below have merged. ## Required - [Workflow permission](/quest/m0/land-fixer/workflows-permission.md) - waits on the maintainer to grant moq-bot `workflows`, so pushes touching workflows are accepted -- [Attempt budget](/quest/m0/land-fixer/attempts.md) - only a fix that fails again spends a pick's single attempt - [Lockfile merges](/quest/m0/land-fixer/lockfile.md) - a conflicted lockfile is rebuilt from the base's copy, so a fix never reverts the base's dependency changes -- [Batch quest check](/quest/m0/land-fixer/batch-check.md) - the daemon runs `quest check` on the batch's chain before spending a sandbox check -- [Pre-check queued PRs](/quest/m0/land-fixer/precheck.md) - compile and lint the next PRs against the base while a batch runs, parking broken ones before they batch -- [Throughput tuning](/quest/m0/land-fixer/tuning.md) - batch size from the failure rate, no wasted base-conflict rounds, rerere, a fixer cap, and an explicit reasoning effort -- [CI failures park](/quest/m0/land-fixer/ci-park.md) - a PR whose own required CI fails is parked for the fixer with the failing log, not ejected -- [Queue status in the required check](/quest/m0/land-fixer/check-status.md) - the gate shows each PR's queue state in its required check run, never completing it -- [PR CI as a signal](/quest/m0/land-fixer/pr-signal.md) - `/quest-merge` and the fixer read a PR's own CI without gating on it; public repos may run it, private ones need not -- [Flaky tests](/quest/m0/land-fixer/flaky.md) - the gate reruns a failed module, and a rerun-proven flaky test is disabled in a separate PR with a follow-up quest -- [Local mergeability](/quest/m0/land-fixer/mergeability.md) - `quest land` merges with the quest driver instead of ejecting on GitHub's conflict flag - [Protected edits as ready commits](/quest/m0/land-fixer/protected-commit.md) - a fixer edit `validateFix` rejects for touching protected paths comes back as a commit the maintainer applies +- [PR CI as a signal](/quest/m0/land-fixer/pr-signal.md) - `/quest-merge` and the fixer read a PR's own CI without gating on it; public repos may run it, private ones need not ## Related -- [Merge through a host gate](/quest/m0/host-land/README.md) - the gate and fixer this epic extends +- [Queue watcher](/quest/m0/queue-watcher/README.md) - runs this fixer on each PR the merge queue drops +- [Merge through a host gate](/quest/m0/host-land/README.md) - the gate that runs it for the repositories not yet switched diff --git a/quest/m0/land-fixer/attempts.md b/quest/m0/land-fixer/attempts.md deleted file mode 100644 index 74848074..00000000 --- a/quest/m0/land-fixer/attempts.md +++ /dev/null @@ -1,41 +0,0 @@ -# [S] Only a failed fix spends a pick's attempt - -## Goal - -A maintainer pick's single fixer attempt is spent only when a head the fixer -pushed fails again, or when the fixer hands back. A "requeued unchanged" -verdict, and a fix that only merged the base (cleanly or with resolved -conflicts) leave the attempt unspent. A PR that conflicts again as the base -moves is then fixed again without a re-pick. - -## Plan - -Decided 2026-11-01 (user, /quest-plan of the land fixer audit; see the -[epic](/quest/m0/land-fixer/README.md)). In 2026-10-30 to 11-01, 98 handbacks -said only "This maintainer pick already used its mechanical fix attempt", and -each needed a manual re-pick. Most had spent the attempt on a requeue-unchanged -verdict as collateral of another PR's failure, or on an earlier conflict fix -before the base moved again. - -- How attempts count: - - ✅ Only failed fixes count: requeue-unchanged and base-merge fixes leave - the attempt unspent; it is spent when a pushed fix fails again, or on a - handback - - One attempt per head - - N attempts per pick (for example 3) - - Keep one per pick - -The attempt record is `fix-attempt-.json` (`src/cli/land-fixer.ts`): its -`pick`, `requeued`, and `head` fields, read by `handbackRepeated`. Add an -`outcome` field (`unchanged`, `merged`, or `fixed`) and have -`handbackRepeated` count only `fixed`. A record written before this field -existed has no `outcome` and counts as spent, as today, so a deploy never -hands a pick a second attempt it already used. A head the fixer changed beyond a base merge that then fails its -checks still hands back, so a bad fix cannot loop. - -Batch triage ([moq-dev/quest#355](https://github.com/moq-dev/quest/pull/355)) added the record's -`blamed` field: the fix job that blamed the pick, or `false` while the pick was -only collateral of other PRs' failures. `spent` (`src/cli/land-fixer.ts`) -counts an attempt as spent only once a job blamed it, and a record without -`blamed` counts as spent. Narrow `spent` to blamed outcomes that are `fixed` -or a handback; this quest owns the counting rule for every outcome. diff --git a/quest/m0/land-fixer/batch-check.md b/quest/m0/land-fixer/batch-check.md deleted file mode 100644 index 1ab6c8f0..00000000 --- a/quest/m0/land-fixer/batch-check.md +++ /dev/null @@ -1,21 +0,0 @@ -# [XS] The daemon runs quest check on the batch before the sandbox - -## Goal - -Before it spends a sandbox check on a batch, `quest land` runs `quest check` -on the batch's combined squash chain inside the daemon. A failure there is -blamed on the link that introduced it, the same way -batch triage ([moq-dev/quest#355](https://github.com/moq-dev/quest/pull/355)) blames a failing check: that -PR is parked for the fixer, and the batch is rebuilt without it. No sandbox -time is spent. - -## Plan - -Decided 2026-11-01 (maintainer, from the audit of the merge-queue plan; see -the [epic](/quest/m0/land-fixer/README.md)). In the audit's 24 hours on moq, -`check quest` was the first failing module in 9 of 66 failed batches (about -14%), even though each PR passed `quest check` alone. Two PRs can each pass -alone yet together leave a dangling link or a `Required` cycle. The daemon -already screens each candidate alone (`screener()` in `src/cli/land.ts`), and -`quest check` on the chain tip costs well under a second, so check the chain -too. diff --git a/quest/m0/land-fixer/check-status.md b/quest/m0/land-fixer/check-status.md deleted file mode 100644 index 8cc10f87..00000000 --- a/quest/m0/land-fixer/check-status.md +++ /dev/null @@ -1,38 +0,0 @@ -# [S] The gate shows queue status in its required check - -## Goal - -Each queued PR's required `quest land` check run shows where the PR stands: -queued at a position, waiting on named checks, in a batch, parked for the -fixer, or behind a halted queue. The gate only ever sets that check to queued -or in progress. It never marks it completed, so GitHub's auto-merge can never -fire around the gate. - -## Plan - -Decided 2026-11-01 (user, settling -[moq-dev/quest#355](https://github.com/moq-dev/quest/pull/355)'s decision 7, -[record](https://github.com/moq-dev/quest/pull/355#issuecomment-6561562421)): -"Comment once, but should plan using the require Github check for status? If -possible?" Triage posts the one comment; this quest is the check-run part. - -It is possible. The App already owns the required `quest land` check name; -the rulesets require it, and nothing reports it today. The App can create a -check run with that name on each queued PR's head and update its `status` -(`queued` / `in_progress`) and `output.title` and `output.summary`. While no -conclusion is ever set, the required check stays unsatisfied. The gate keeps -landing through its own push or the pinned squash merge, as now. - -- One check run per pinned head, updated in place; a new head gets a new run. -- Never set `conclusion`: a `success` would let auto-merge land the PR - without the gate, and any other conclusion would read as a failure. -- Needs the App's `checks: write`, an installation permission the maintainer - accepts. Check it during this work; it may already be granted for the - existing check. -- The fixer's comments, which the daemon posts on the PR, explain - decisions; the check run shows state only. - -## Related - -- [A PR whose own CI fails is parked](/quest/m0/land-fixer/ci-park.md) - one of the states shown -- Batch triage ([moq-dev/quest#355](https://github.com/moq-dev/quest/pull/355)) - added the parked, waiting, and halted states diff --git a/quest/m0/land-fixer/ci-park.md b/quest/m0/land-fixer/ci-park.md deleted file mode 100644 index 4e5f3ec4..00000000 --- a/quest/m0/land-fixer/ci-park.md +++ /dev/null @@ -1,30 +0,0 @@ -# [S] A PR whose own CI fails is parked for the fixer - -## Goal - -A queued PR whose required GitHub checks fail is not ejected. The gate -fetches the failing checks' logs, parks the PR, and starts a fixer job with -those logs, the same way it handles a failing batch. The PR leaves the queue -only if the fixer hands it back. - -## Plan - -Decided 2026-11-01 (user, settling -[moq-dev/quest#355](https://github.com/moq-dev/quest/pull/355)'s decision 8, -[record](https://github.com/moq-dev/quest/pull/355#issuecomment-6561562421)): - -- A PR whose own required GitHub CI fails: - - ✅ Park it with the failing check's log for the fixer - - Keep ejecting it at once - -It builds on the parking mechanism from batch triage ([moq-dev/quest#355](https://github.com/moq-dev/quest/pull/355)) -(`src/cli/land-park.ts`, and `fail(...)` in `src/cli/land-serve.ts`). Fetch -each failing check run's log through the App's `checks: read` and -`actions: read`, and trim it the way the batch failure excerpt is trimmed. A -fix the fixer pushes reruns the PR's CI. The PR rejoins batches once its -required checks pass, as with -triage's `checks_pending` rule. - -## Related - -- [Queue status in the required check](/quest/m0/land-fixer/check-status.md) - shows the PR's parked state diff --git a/quest/m0/land-fixer/flaky.md b/quest/m0/land-fixer/flaky.md deleted file mode 100644 index 7b07a04d..00000000 --- a/quest/m0/land-fixer/flaky.md +++ /dev/null @@ -1,69 +0,0 @@ -# [M] The gate reruns a failure, then disables a flaky test - -## Goal - -When a batch's check fails, `quest land` reruns the failing check modules -once on the same tip before blaming or ejecting anything. A test that failed -and then passed on that rerun is flaky. The batch lands, and the fixer opens a -separate PR that disables the test with a reason and adds a follow-up quest -to fix or delete it in the repository's m0 `flaky` epic. The gate queues that -PR like any other, with no human pick needed. A failure the fixer only -suspects is nondeterministic is not disabled. - -## Plan - -Decided 2026-11-01 (user, /quest-plan of the land fixer audit; see the -[epic](/quest/m0/land-fixer/README.md)). A flaky ffmpeg render timeout in -moq's interop tests ejected whole batches on 2026-11-01. The fixer then -recognized it as flaky, but only after the batch's PRs had been ejected and -their attempts spent. The fix ([moq-dev/moq#7480](https://github.com/moq-dev/moq/pull/7480)) -was written and landed by hand. - -- What the gate does with a flaky failure: - - ✅ Rerun the failed module once - - The fixer names the flake - - Nothing new -- User follow-up: "make a rule to immediately disable any flakey tests and - create a follow-up to fix/delete them". -- Who disables: - - The fixer opens and queues its own PR - - The fixer opens a PR, and a maintainer picks it - - Inside the next batch: the gate adds the skip commit to the batch it - lands next, tested with it (picked first, then superseded) - - ✅ Superseded 2026-11-01 (user, second /quest-plan): the fixer pushes the - disabling commit, with its investigation quest, onto the batch's base PR, - so it lands with this batch: "disable the flaky test in the base PR". -- What counts as flaky: - - Only a rerun that passes - - ✅ A rerun that passes, or a failure the fixer judges nondeterministic -- Where the follow-up goes: - - ✅ A standing `flaky` epic in the repository's m0, one quest per - disabled test - - The earliest milestone, ungrouped - - m1 - -- Superseded again 2026-11-01 (maintainer, from the audit of the merge-queue - plan): PRs land as squashes, so a disable commit on the batch's base PR - would land inside that unrelated PR's commit, under its title; and - disabling on the fixer's judgment alone risks hiding real bugs. - - ✅ Only a rerun-proven flaky test, disabled in a separate PR the gate - queues - - Keep: rerun or the fixer's judgment, with the commit on the base PR - -How the gate reruns one module is up to the repository. `quest land` passes -the failed module names to the check command (for example through -`QUEST_LAND_RERUN`). A check command that ignores them reruns whole. moq's -`sh/dispatch.sh` already reports `ci: failed`. - -The fixer writes the skip in the test framework's own form, with the reason -naming the follow-up quest: `#[ignore = "flaky: /quest/m0/flaky/.md"]`, -`test.skip`, `pytest.mark.skip`, and so on. A disabled test that is a quest's -acceptance test still counts as its quest's work. The follow-up quest's Goal -says to fix the test or delete it, and links the batch logs that showed the -flake. - -Flaky verdicts never spend a PR's attempt -([Attempt budget](/quest/m0/land-fixer/attempts.md)), and a batch whose only -failure was flaky lands without ejecting anyone. Since batch triage -([moq-dev/quest#355](https://github.com/moq-dev/quest/pull/355)), a failing batch's PRs are parked -for the fixer rather than ejected; the rerun comes before that. diff --git a/quest/m0/land-fixer/mergeability.md b/quest/m0/land-fixer/mergeability.md deleted file mode 100644 index d7643194..00000000 --- a/quest/m0/land-fixer/mergeability.md +++ /dev/null @@ -1,34 +0,0 @@ -# [S] quest land merges locally instead of trusting GitHub's mergeability - -## Goal - -`quest land` never ejects a PR as `merge_conflict` because of GitHub's -mergeability alone. When the squash API refuses a PR that GitHub marks as -conflicting, the gate merges the base into it locally with the quest merge -driver, pushes that merge, and lands the PR. When the local merge also -conflicts, the PR stays queued and parked while the fixer resolves it; it -leaves the queue only on a handback (2026-11-01, see the -[epic](/quest/m0/land-fixer/README.md)). - -## Plan - -Decided 2026-10-31 (user, /quest-plan of the follow-ups from a moq-dev/moq -`/quest-complete` run). GitHub reported conflicts on -[moq-dev/moq#7407](https://github.com/moq-dev/moq/pull/7407), which merges -cleanly, and on [moq-dev/moq#7382](https://github.com/moq-dev/moq/pull/7382), -whose only conflict was in quest link lists, which the driver resolves. - -`mergePinned` (`src/cli/land-serve.ts`) returns `merge_conflict` when -`pull.mergeable === false` or `mergeable_state === "dirty"`. Hand that case -to `prepareHead` (`src/cli/land-head.ts`), which already merges with the -driver through `mergeBase` (`src/cli/land.ts`), instead of ejecting. - -Since batch triage ([moq-dev/quest#355](https://github.com/moq-dev/quest/pull/355)), that -`merge_conflict` already parks the PR for the fixer (`fail` in -`src/cli/land-serve.ts`) instead of ejecting it, and the rest of the batch -stays queued; this quest adds the local merge before parking. - -GitHub also flags the gate's own freshly pushed base merge as conflicting, from -the pin's stale mergeability ([moq-dev/quest#342](https://github.com/moq-dev/quest/pull/342)). -`mergePinned` now retries that case, since the merge holds the tested base. What -remains is a head the gate did not merge in that round, which GitHub flags anyway. diff --git a/quest/m0/land-fixer/pr-signal.md b/quest/m0/land-fixer/pr-signal.md index 9521a5cf..99cf442f 100644 --- a/quest/m0/land-fixer/pr-signal.md +++ b/quest/m0/land-fixer/pr-signal.md @@ -28,13 +28,15 @@ this because it's free, closed source do not)". moq-dev/moq plans its side in `quest status` already lists a PR's checks. `/quest-merge` should say when a non-required check failed on the head it is about to pick, without refusing. -The fixer gets a summary per PR (check name, conclusion, run URL, and a -trimmed log of each failure) in its job input, next to the batch log. A -failing non-required check never parks or ejects a PR on its own; the parked -case is only for required checks, in -CI failures park (`quest/m0/land-fixer/ci-park.md`), which -[moq-dev/quest#363](https://github.com/moq-dev/quest/pull/363) plans. +The fixer gets a summary of the PR's checks (check name, conclusion, run URL, +and a trimmed log of each failure) in its job input, next to the queue's +failure log. A failing non-required check never stops a PR on its own; a +failing required check keeps GitHub from enqueuing it, which +[Bad picks](/quest/m0/queue-watcher/bad-picks.md) handles. + +Retargeted 2026-11-01 to the [queue watcher](/quest/m0/queue-watcher/README.md), +which replaces the batched host gate. ## Related -- Batch triage ([moq-dev/quest#355](https://github.com/moq-dev/quest/pull/355)) - where the fixer uses the signal +- [Fix and re-enqueue](/quest/m0/queue-watcher/fix.md) - where the fixer uses the signal diff --git a/quest/m0/land-fixer/precheck.md b/quest/m0/land-fixer/precheck.md deleted file mode 100644 index f789734b..00000000 --- a/quest/m0/land-fixer/precheck.md +++ /dev/null @@ -1,29 +0,0 @@ -# [M] Queued PRs are pre-checked while a batch runs - -## Goal - -While a batch's check runs, the daemon pre-checks the next queued PRs, each -alone against the current base, with the repository's quick command (compile -and lint, not tests), at a lower priority than the batch check. A PR that -fails is parked for the fixer with the output before it enters a batch, so a -broken PR stops costing a whole batch. - -## Plan - -Decided 2026-11-01 (maintainer, from the audit of the merge-queue plan; see -the [epic](/quest/m0/land-fixer/README.md)). The audit found the batch -failure rate is the binding bottleneck: about half of moq's sandbox checks -failed, which means roughly 20-25% of PRs were broken against the current -base. A batch of k PRs lands about k·(1−p)^k PRs per check, so cutting p from -0.2 to 0.1 roughly doubles throughput. moq's `check rs` fails in about 26 s, -so a compile and lint pre-check catches most of these cheaply. - -- The repository declares the pre-check command beside its check command - (moq: something like `just check-changed `). Run it in the same kind - of sandbox as the batch check, never on the host. -- Run below the batch check's priority so it cannot slow the batch; the - `land-priority` work already provides the priority classes. -- Pre-check a PR at most once per (head, base) pair; a moved base re-queues - it for pre-check only when the PR is next in line. -- A pre-check failure parks the PR through batch triage's - ([moq-dev/quest#355](https://github.com/moq-dev/quest/pull/355)) mechanism, and the fixer gets the pre-check log as its failure. diff --git a/quest/m0/land-fixer/tuning.md b/quest/m0/land-fixer/tuning.md deleted file mode 100644 index c9230247..00000000 --- a/quest/m0/land-fixer/tuning.md +++ /dev/null @@ -1,34 +0,0 @@ -# [S] Tune batching and the fixer for throughput - -## Goal - -Once failing PRs are parked instead of ejected, the gate stops paying for -choices made while a failure ejected a whole batch: - -- Batch size follows the measured per-PR failure rate, not halving on each - failure. -- A conflict on the batch's base PR parks that PR and fills the batch in the - same round, without spending a round on it. -- The gate's repository enables `git rerere`, so it replays conflict - resolutions it has already made. -- Concurrent fixer jobs are capped and run below the batch check's priority. -- The fixer runs with an explicit reasoning effort; today Codex reports - `reasoning effort: none`. - -## Plan - -Decided 2026-11-01 (maintainer, from the audit of the merge-queue plan; see -the [epic](/quest/m0/land-fixer/README.md)). Audit findings on moq's 24 hours: - -- Halving after each failure kept batches at 2-3 PRs. With failing PRs - parked, size from the recent failure rate instead (about 4-5 at today's - rate), keeping `batch-policy.json` as the state. -- 156 rounds were a base-PR conflict that ran no check, about 1.5-2.5 hours - of lane time a day. -- 44 of 79 conflicting PRs conflicted 2-7 times; `rerere` lets the gate's - own base merges replay resolutions. -- Seven fixer jobs started within three minutes on a host already at load - 18-25 of 32 cores, and timeouts under load caused several flaky failures. -- Fixer job logs show `reasoning effort: none`. `land-fix.ts` passes a model - but no `model_reasoning_effort`; set one, defaulting to `medium`, with a - daemon option to override it. diff --git a/quest/m0/land-fixer/workflows-permission.md b/quest/m0/land-fixer/workflows-permission.md index 4bc879c8..53081ae6 100644 --- a/quest/m0/land-fixer/workflows-permission.md +++ b/quest/m0/land-fixer/workflows-permission.md @@ -3,8 +3,9 @@ ## Goal The moq-bot GitHub App (ID 1039466) has the `workflows: write` repository -permission, and every installation that lands through the host gate has -accepted it: moq-dev/moq, moq.pro, rust2, rind and quest. GitHub then accepts +permission, and every installation the land fixer pushes to, through the +queue watcher or the host gate, has accepted it: moq-dev/moq, moq.pro, rust2, +rind and quest. GitHub then accepts a fixer push that touches `.github/workflows`, including a base merge pushed to a fork's branch. This quest waits on the maintainer. To check, open the App's settings under Permissions; GitHub lists pending permission requests on diff --git a/quest/m0/land-priority-sandbox.md b/quest/m0/land-priority-sandbox.md deleted file mode 100644 index a1b2d6e5..00000000 --- a/quest/m0/land-priority-sandbox.md +++ /dev/null @@ -1,16 +0,0 @@ -# [XS] Land checks run at gate priority - -## Goal - -`quest land serve` runs each batch check with `QUEST_GATE_PRIORITY=1`, so a -repository's nested `quest gate` calls admit the land check ahead of agent -load. - -## Plan - -Decided 2026-10-31 (user, /quest-plan of the follow-ups from moq.pro's and rust2's cutover to the host gate and the fixer repairs [moq-dev/quest#320](https://github.com/moq-dev/quest/pull/320), [#325](https://github.com/moq-dev/quest/pull/325), [#327](https://github.com/moq-dev/quest/pull/327) and [#328](https://github.com/moq-dev/quest/pull/328)). - -- Starts once [moq-dev/quest#324](https://github.com/moq-dev/quest/pull/324) - lands; it adds the variable. Pass it to the `--sandbox` command from - `src/cli/land-serve.ts`. The host also exports it in its check command, - which the [gate host](/quest/m0/host-land/host.md) owns. diff --git a/quest/m0/land-squash-mismatch.md b/quest/m0/land-squash-mismatch.md deleted file mode 100644 index ba351163..00000000 --- a/quest/m0/land-squash-mismatch.md +++ /dev/null @@ -1,24 +0,0 @@ -# [XS] A post-squash tree mismatch comments on the merged PR - -## Goal - -When GitHub's squash of a landed PR has a different tree from the one the -gate tested, `quest land serve` posts a CRITICAL comment on that merged PR, -naming the squash commit, both trees, and what happened to the rest of the -batch. Today the mismatch reaches only the host's log and warning line, so -nobody reading the PR learns that `main` holds an untested tree. - -## Plan - -Decided 2026-11-01 (user, /quest-plan of the follow-ups from the -2026-11-01 `/quest-complete` sessions): ✅ plan it, [XS], m0. - -- Raised as a follow-up in - [moq-dev/quest#321](https://github.com/moq-dev/quest/pull/321), which made - a post-squash `tree_mismatch` keep the PR merged and the rest of the batch - queued. The detail is built in `src/cli/land-serve.ts` (`CRITICAL: GitHub - squash #N at has tree ...`), then only logged and warned. -- Post it through the daemon's existing comment path, with its marker and - deduplication, so a restarted daemon doesn't post it twice. -- Extend the post-squash tree-mismatch test in `test/land.test.ts` to expect - the comment. diff --git a/quest/m0/queue-watcher/README.md b/quest/m0/queue-watcher/README.md new file mode 100644 index 00000000..39aed80e --- /dev/null +++ b/quest/m0/queue-watcher/README.md @@ -0,0 +1,123 @@ +# Land through GitHub's merge queue with an async fixer + +## Goal + +Each repository lands through GitHub's merge queue again, one PR at a time, +with no batching and no pre-checks. When the queue drops a PR for a failure +or a conflict, a watcher on the maintainer's host runs the existing Codex land +fixer in the background and re-enqueues the PR at the back of the queue. Only +a decision the fixer can't make reaches a person. moq-dev/moq moves first; +moq-dev/quest, moq.pro, rust2 and rind follow once moq proves it, and the +batching host gate (`quest land serve`) is then retired. + +The epic finishes with an end-to-end run on moq-dev/moq: a PR the queue drops +for a failing build is fixed and re-enqueued, a PR dropped for a conflict is +resolved and re-enqueued, a flaky failure is re-enqueued unchanged, and a +design conflict is handed back. The README's landing section describes the +watcher. + +Non-goals: moq's own switch (its ruleset, its `merge_group` workflow, the +runner label, and retiring its `quest land` requirement). moq-dev/moq plans +that in its tree, in +[Land main through GitHub's merge queue](https://github.com/moq-dev/moq/blob/main/quest/m0/ci/land.md), +rewritten by [moq-dev/moq#7589](https://github.com/moq-dev/moq/pull/7589). + +## Plan + +Decided 2026-11-01 (maintainer, /quest-plan), reversing the 2026-10-26 +decision recorded in moq-dev/moq's `quest/m0/ci/land.md` to leave GitHub's +merge queue for a batched gate. That decision's main reason was that GitHub +cannot resolve quest-file conflicts: 71% of moq's 246 merge-conflict removals +touched `quest/`. The async fixer now absorbs those, resolving them with the +quest merge driver and re-enqueuing. The batched host gate measured about 4-6 +merged PRs per hour at a 23% per-PR failure rate, with batches shrinking to +one PR; GitHub's serial queue measured 5.5. + +Decision prompts, selected options marked ✅: + +- Goal: + - ✅ Yes, as stated: a serialized GitHub merge queue, with every failure + fixed asynchronously and re-enqueued. Batching and pre-checks are + dropped: on moq a useful pre-check must compile, which is most of a check. + - Keep the host queue, serial +- Scope: + - ✅ moq first, since it lands the most PRs and so proves the design + fastest. moq-dev/quest, moq.pro, rust2 and rind stay on the host + `quest land` gate until moq proves it, then + [follow](/quest/m0/queue-watcher/follow.md). + - All five at once + - moq and quest +- Runners: + - ✅ Self-hosted on nixos: `merge_group` builds run on the host's runner + (kixelated/nixos `modules/ci-runners.nix`, runner `moq-ci`, label + `moq-queue`), for warm caches and no GitHub minutes + - GitHub-hosted +- Interim: + - Keep improving the host gate + - ✅ Freeze gate work: no more host-gate improvements except fixes the + switch needs. Closed for it: [moq-dev/quest#376](https://github.com/moq-dev/quest/pull/376) + (batch sizing), [#379](https://github.com/moq-dev/quest/pull/379) (batch + quest check), [#375](https://github.com/moq-dev/quest/pull/375) (rerun + before blame), [#377](https://github.com/moq-dev/quest/pull/377) + (pre-checks) and [moq-dev/moq#7588](https://github.com/moq-dev/moq/pull/7588). +- Build concurrency: + - ✅ 1, maybe 2: start the merge queue at build concurrency 1, and consider + 2 only with measured data. At 1, each `merge_group` holds exactly one PR + on top of the base, so a failure blames that PR with no triage. +- Re-enqueue: + - ✅ The back of the queue, so a PR being fixed never jumps PRs that + passed + - The front +- Attempts: + - ✅ Unlimited while progressing; see + [Stop rules](/quest/m0/queue-watcher/stop.md). This replaces batch + triage's ([moq-dev/quest#355](https://github.com/moq-dev/quest/pull/355)) + one attempt per pick. +- Flaky failures: + - ✅ The fixer re-enqueues them unchanged, and a test seen flaking twice is + disabled in a separate PR with a follow-up quest; the rerun before blame + (#375) is dropped. See [Flaky failures](/quest/m0/queue-watcher/flaky.md). +- Picks GitHub refuses to enqueue: + - ✅ The watcher fixes them, then enqueues; see + [Bad picks](/quest/m0/queue-watcher/bad-picks.md) +- Milestone: + - ✅ m0, at the top + +Shared shape, for the children to follow: + +- The watcher is a long-running host daemon, one per repository, that reuses + the host gate's fixer infrastructure rather than a new one: moq-bot's App + key through systemd `LoadCredential`, the ephemeral nspawn fixer container + and its launcher, the detached fixer supervisor and its collection in + `src/cli/land-fixer.ts`, `validateFix`'s protected paths, the ready-commit + handback that `quest land apply` rebuilds, and the daemon posting the + comments the fixer writes. A `quest land watch` subcommand beside + `quest land serve` is the likely form; the first child settles it. +- The fixer prompt (`assets/land-fix.md`) and its quality quests stay in + [the land fixer epic](/quest/m0/land-fixer/README.md); the host container + work stays in [the host gate epic](/quest/m0/host-land/README.md). +- The review policy is unchanged: a head the fixer pushed as a recorded + mechanical fix lands without a new review (maintainer, 2026-10-30). A push + by anyone else ends the pick, as does a person removing the PR from the + queue. +- moq's `merge_group` builds may call `quest gate`; the + [landing priority](/quest/m0/land-priority.md) class + (`QUEST_GATE_PRIORITY`) lets them admit ahead of agent load, set from moq's + workflow or the runner's environment. + +## Required + +- [Picks enter the merge queue](/quest/m0/queue-watcher/pick.md) - `quest merge --land` enqueues the pinned head into GitHub's merge queue and records the pick where the watcher reads it +- [Queue removals](/quest/m0/queue-watcher/removals.md) - the watcher sees each PR the queue drops, why, and the failing build's log +- [Fix and re-enqueue](/quest/m0/queue-watcher/fix.md) - the fixer runs on each dropped PR in the background, and the watcher re-enqueues it at the back +- [Stop rules](/quest/m0/queue-watcher/stop.md) - the fixer keeps going while each round changes something, and hands back on a no-op fix, a repeated failure, or three flaky re-enqueues +- [Bad picks](/quest/m0/queue-watcher/bad-picks.md) - a pick GitHub refuses to enqueue is merged locally with the quest driver or fixed, then pushed and enqueued +- [Flaky failures](/quest/m0/queue-watcher/flaky.md) - a flaky failure is re-enqueued unchanged, and a test seen flaking twice is disabled in a separate PR with a follow-up quest +- [Watcher on the host](/quest/m0/queue-watcher/host.md) - kixelated/nixos runs the watcher for moq-dev/moq beside its fixer container +- [The other repositories follow](/quest/m0/queue-watcher/follow.md) - condition: moq's queue is proven; then moq-dev/quest switches and the others plan theirs +- [Retire the batching gate](/quest/m0/queue-watcher/retire.md) - once no repository lands through `quest land serve`, its batching code and host units go + +## Related + +- [The land fixer](/quest/m0/land-fixer/README.md) - the fixer the watcher runs +- [Merge through a host gate](/quest/m0/host-land/README.md) - the gate the other repositories keep until they follow diff --git a/quest/m0/queue-watcher/bad-picks.md b/quest/m0/queue-watcher/bad-picks.md new file mode 100644 index 00000000..207114e8 --- /dev/null +++ b/quest/m0/queue-watcher/bad-picks.md @@ -0,0 +1,49 @@ +# [M] The watcher fixes a pick GitHub refuses to enqueue + +## Goal + +A picked PR that GitHub refuses to enqueue, because it conflicts with the +base or otherwise can't merge, is fixed by the watcher instead of handed back. +The watcher merges the base into it locally with the quest merge driver; a +clean merge is pushed and enqueued. When the local merge still conflicts, or +the PR's own required checks fail, the fixer resolves it with the conflicting +paths or the failing checks' logs, and the watcher pushes and enqueues the +result. The PR reaches a person only on a handback. The watcher never trusts +GitHub's mergeability flag alone. + +## Plan + +Decided 2026-11-01 (maintainer, /quest-plan; see the +[epic](/quest/m0/queue-watcher/README.md)): + +- Picks GitHub refuses to enqueue: + - ✅ The watcher fixes them, including conflicts resolved with the quest + merge driver, then pushes and enqueues + - Hand them back + +This quest was the host gate's local mergeability quest, decided 2026-10-31 +(user, /quest-plan of the follow-ups from a moq-dev/moq `/quest-complete` +run): GitHub reported conflicts on +[moq-dev/moq#7407](https://github.com/moq-dev/moq/pull/7407), which merges +cleanly, and on [moq-dev/moq#7382](https://github.com/moq-dev/moq/pull/7382), +whose only conflict was in quest link lists, which the driver resolves. It +also absorbs the host gate's plan to park a PR whose own required CI fails for +the fixer with the failing check's log (decided 2026-11-01, settling +[moq-dev/quest#355](https://github.com/moq-dev/quest/pull/355)'s decision 8), +since on a merge-queue base such a PR is a pick GitHub won't enqueue. + +- The watcher learns of refused picks from the marker + [Picks enter the merge queue](/quest/m0/queue-watcher/pick.md) records, and + from picks whose auto-merge stays armed while the PR is conflicting or its + required checks failed. +- Merge locally with `mergeBase` (`src/cli/land.ts`), which already uses the + driver; the host gate's `prepareHead` (`src/cli/land-head.ts`) shows the + push. A clean driver merge needs no review: it is a conflict-free base merge. +- Fetch failing required checks' logs through the App's `checks: read` and + `actions: read`, trimmed like a queue failure's. +- The same [stop rules](/quest/m0/queue-watcher/stop.md) apply. + +## Required + +- [Fix and re-enqueue](/quest/m0/queue-watcher/fix.md) - the fixer path and re-enqueue this reuses +- [Picks enter the merge queue](/quest/m0/queue-watcher/pick.md) - the pick marker a refused pick leaves diff --git a/quest/m0/queue-watcher/fix.md b/quest/m0/queue-watcher/fix.md new file mode 100644 index 00000000..846c616e --- /dev/null +++ b/quest/m0/queue-watcher/fix.md @@ -0,0 +1,40 @@ +# [M] The fixer runs on each dropped PR and the watcher re-enqueues it + +## Goal + +When the merge queue drops a picked PR for a failure or a conflict, the +watcher starts the Codex land fixer on it in the background, with the failing +build's log or the conflicting base. When the fixer fixes the PR, or judges +the failure flaky, the watcher re-enqueues the resulting head at the back of +the queue. When the fixer hands back, the watcher posts its comment and the +PR leaves the watcher's care. The queue keeps landing other PRs meanwhile. + +## Plan + +Decided 2026-11-01 (maintainer, /quest-plan; see the +[epic](/quest/m0/queue-watcher/README.md)): re-enqueue at the back, and keep +going while each round makes progress. + +- Reuse the host gate's fixer: `startFixer`, the detached supervisor, and + `collectFixers` in `src/cli/land-fixer.ts`, the `qland-fix` container and + its launcher, `validateFix`, the ready-commit handback for + `quest land apply`, and the daemon posting the comments in the fixer's + result. The fixer's input is one PR and the dropped head, so the batch + blame step is unused. +- Re-enqueue with `enqueuePullRequest` (no `jump`) and `expectedHeadOid` set + to the head the fixer pushed or the unchanged head, as moq-bot. A fix the + fixer pushed carries its `## Review` record as a mechanical fix, as today. +- A conflict removal starts the fixer on a merge of the current base with the + quest merge driver, as `prepareHead` (`src/cli/land-head.ts`) does. +- Carried over from the dropped + [throughput tuning](https://github.com/moq-dev/quest/pull/376) plan, since + they apply to any fixer: cap concurrent fixer jobs and run them below the + queue's builds in `quest gate`, and pass an explicit + `model_reasoning_effort` (default `medium`, with a daemon option), since + fixer logs show `reasoning effort: none`. +- When to stop is [Stop rules](/quest/m0/queue-watcher/stop.md); flaky + handling is [Flaky failures](/quest/m0/queue-watcher/flaky.md). + +## Required + +- [Queue removals](/quest/m0/queue-watcher/removals.md) - the removals and logs this acts on diff --git a/quest/m0/queue-watcher/flaky.md b/quest/m0/queue-watcher/flaky.md new file mode 100644 index 00000000..cd067c33 --- /dev/null +++ b/quest/m0/queue-watcher/flaky.md @@ -0,0 +1,56 @@ +# [M] A flaky failure is re-enqueued, and a twice-flaky test is disabled + +## Goal + +When the fixer judges a dropped PR's failure flaky, the watcher re-enqueues +the PR unchanged. The watcher keeps a per-repository record of flaky +verdicts by test. A test seen flaking twice is disabled in a separate PR that +also adds a follow-up quest to fix or delete it in the repository's m0 +`flaky` epic; the watcher enqueues that PR like any other, with no human pick +needed. + +## Plan + +Decided 2026-11-01 (maintainer, /quest-plan; see the +[epic](/quest/m0/queue-watcher/README.md)): + +- Flaky failures: + - ✅ The fixer re-enqueues them unchanged, and a test seen flaking twice is + disabled in a separate PR with a follow-up quest. The host gate's rerun + before blame ([moq-dev/quest#375](https://github.com/moq-dev/quest/pull/375), + and moq's module naming for it, + [moq-dev/moq#7588](https://github.com/moq-dev/moq/pull/7588)) is dropped; + both PRs are closed. + +Carried over from the host gate's flaky-test quest, whose decisions still +apply (2026-11-01, user and maintainer): + +- A flaky ffmpeg render timeout in moq's interop tests ejected whole batches + on 2026-11-01, and its fix + ([moq-dev/moq#7480](https://github.com/moq-dev/moq/pull/7480)) was written + by hand. The user asked for "a rule to immediately disable any flakey tests + and create a follow-up to fix/delete them". +- The disable goes in its own PR, never inside another PR's commit, since PRs + land as squashes and an unrelated PR's commit would carry it. +- The follow-up goes in a standing `flaky` epic in the repository's m0, one + quest per disabled test. + +What counts: the second flaky verdict for the same test, across any PRs, in +the watcher's state. A single verdict only re-enqueues, since disabling on one +judgment alone risks hiding a real bug. Each verdict names the test in the +same signature [Stop rules](/quest/m0/queue-watcher/stop.md) uses, and three +flaky re-enqueues of one PR in a row hand it back there. + +The fixer container holds no GitHub credentials, so the fixer returns the +disabling commit in its result and the watcher pushes it to a new branch and +opens the PR as moq-bot, the way the host gate opened its +`quest-land/fix-` base fixes. The fixer writes the skip in the test framework's own form, with the reason +naming the follow-up quest: `#[ignore = "flaky: /quest/m0/flaky/.md"]`, +`test.skip`, `pytest.mark.skip`, and so on. The follow-up quest's Goal says to +fix the test or delete it, and links the `merge_group` runs that showed the +flake. A disabled test that is a quest's acceptance test still counts as its +quest's work. + +## Required + +- [Fix and re-enqueue](/quest/m0/queue-watcher/fix.md) - the fixer's flaky verdict and the re-enqueue diff --git a/quest/m0/queue-watcher/follow.md b/quest/m0/queue-watcher/follow.md new file mode 100644 index 00000000..a5ae8c8a --- /dev/null +++ b/quest/m0/queue-watcher/follow.md @@ -0,0 +1,39 @@ +# [M] The other repositories follow moq to the merge queue + +## Goal + +Condition: the maintainer judges moq-dev/moq's merge queue and watcher +proven, from the live evidence moq's +[land quest](https://github.com/moq-dev/moq/blob/main/quest/m0/ci/land.md) +collects. Ask the maintainer once the watcher has run on moq. + +Then moq-dev/quest lands through GitHub's merge queue with the watcher: a +`merge_group` workflow runs `just check` and `just test`, `main`'s ruleset +requires it and the merge queue instead of the unreported `quest land` +check, the host's `quest land serve` instance for quest is disabled, and a +watcher instance serves it. A PR from a fork lands through the queue. +`CONTRIBUTING.md` and the README's landing section describe it. moq.pro, rust2 +and rind each plan their own switch in their trees. + +## Plan + +Decided 2026-11-01 (maintainer, /quest-plan; see the +[epic](/quest/m0/queue-watcher/README.md)): + +- ✅ moq first; moq-dev/quest, moq.pro, rust2 and rind stay on the host + `quest land` gate until moq proves it, then follow. + +This absorbs the fork-path verification that remained of "moq-dev/quest +lands through quest land" (decided 2026-10-26 and 2026-10-27, user): the +host gate's fork path was never verified live. On the merge queue, GitHub +lands a fork PR itself; verify one live, and fix anything it needs here. + +moq.pro is private, and moq-dev is on GitHub Team. GitHub has offered merge +queues on private repositories only under Enterprise Cloud, so check whether +moq.pro can enable one before planning its switch; if it can't, it stays on +the host gate, and [Retire the batching gate](/quest/m0/queue-watcher/retire.md) +needs a maintainer decision. moq-dev/quest, rust2 and rind are public. + +## Required + +- [Watcher on the host](/quest/m0/queue-watcher/host.md) - moq's watcher runs, so its queue can be proven diff --git a/quest/m0/queue-watcher/host.md b/quest/m0/queue-watcher/host.md new file mode 100644 index 00000000..2dc2a185 --- /dev/null +++ b/quest/m0/queue-watcher/host.md @@ -0,0 +1,35 @@ +# [S] The host runs the queue watcher for moq-dev/moq + +## Goal + +The maintainer's NixOS host runs the queue watcher for moq-dev/moq from +[kixelated/nixos](https://github.com/kixelated/nixos), as its own unprivileged +user under a hardened systemd unit like `quest-land.service`, with moq-bot's +key through `LoadCredential` and the same `qland-fix` fixer container and +launcher. Check it by watching it re-enqueue a PR the queue dropped. + +## Plan + +Decided 2026-11-01 (maintainer, /quest-plan; see the +[epic](/quest/m0/queue-watcher/README.md)): the watcher runs on the host, and +`merge_group` builds run on its self-hosted runner `moq-ci` (label +`moq-queue`, `modules/ci-runners.nix`). + +- Add a watcher instance for moq to the quest-land module, and pin a quest + with the watcher. +- The `quest land serve` instance for moq is disabled at moq's cutover + (`services.quest-land.repositories.moq.enable = false`), which moq's + [land quest](https://github.com/moq-dev/moq/blob/main/quest/m0/ci/land.md) + owns; the fixer container keeps serving the watcher. +- moq-bot may need `actions: read` on moq-dev/moq for the `merge_group` logs; + the maintainer accepts any new installation permission. +- The runner's own setup and the `merge_group` workflow are moq's switch, + not this quest. + +## Required + +- [Fix and re-enqueue](/quest/m0/queue-watcher/fix.md) - the watcher command this deploys + +## Related + +- [Gate host](/quest/m0/host-land/host.md) - the units and fixer container this reuses diff --git a/quest/m0/queue-watcher/pick.md b/quest/m0/queue-watcher/pick.md new file mode 100644 index 00000000..bacf262c --- /dev/null +++ b/quest/m0/queue-watcher/pick.md @@ -0,0 +1,45 @@ +# [S] quest merge --land enqueues into GitHub's merge queue + +## Goal + +On a base with a merge queue, `quest merge --land ` +enqueues the pinned head into GitHub's merge queue, or arms auto-merge bound +to it while its required checks are pending, and records the pick so the +watcher knows which head the maintainer picked. It never refuses a head for +being behind the base, since the queue builds each PR on the current base. A +pick GitHub refuses to enqueue (conflicts, failing required checks, or another +mergeability refusal) is still recorded, and the command says the watcher will +fix it rather than failing. + +## Plan + +Decided 2026-11-01 (maintainer, /quest-plan; see the +[epic](/quest/m0/queue-watcher/README.md)): picks enqueue into GitHub's merge +queue, and the watcher fixes a pick GitHub refuses. + +`land` in `src/cli/hold.ts` already enables auto-merge bound to the head on a +queue base (`snapshot.queue`), which enters the queue once checks pass. What +changes: + +- Skip the behind-or-diverged comparison on a queue base. It exists for + ungated bases, where nothing else tests the PR on the current base. +- Prefer `enqueuePullRequest` with `expectedHeadOid` when the PR is already + mergeable, so the head is pinned and enqueued at once. GitHub enqueues a PR + only once its required checks pass on the PR itself. +- Record the pick durably. On the host gate the pin lives in squash + auto-merge's commit body, which a conflicting PR cannot get, and GitHub drops + auto-merge when the queue removes a PR. Pick a marker the watcher can read + for a refused or removed PR, such as a comment naming the picked head, and + coordinate with [moq-dev/quest#326](https://github.com/moq-dev/quest/pull/326), + which drops the `quest:merge` label. The pick covers that head and the heads + the fixer pushes from it. +- `quest:merge` (or its successor) must not be released when the queue drops + the PR while the watcher still holds it (`dequeued` in `src/cli/hold.ts`); + only a handback ends the pick. +- Bases that require the `quest land` check keep queuing for the host gate + until their repository [follows](/quest/m0/queue-watcher/follow.md). + +## Related + +- [quest merge --land gaps](/quest/m0/helpers/merge-land.md) - the same command's other gaps, in [moq-dev/quest#347](https://github.com/moq-dev/quest/pull/347) +- [Bad picks](/quest/m0/queue-watcher/bad-picks.md) - what the watcher does with a refused pick diff --git a/quest/m0/queue-watcher/removals.md b/quest/m0/queue-watcher/removals.md new file mode 100644 index 00000000..d8fb410e --- /dev/null +++ b/quest/m0/queue-watcher/removals.md @@ -0,0 +1,45 @@ +# [M] The watcher sees each PR the merge queue drops, and why + +## Goal + +A long-running watcher on the host notices every PR that leaves a +repository's merge queue unmerged, within a minute or two, and records why: +a failing `merge_group` build (with the failing jobs' logs), a conflict, a +push to the PR, or a person removing it. It posts a comment on the PR naming +the reason and the failing log's tail. The watcher survives restarts without +missing or repeating a removal. + +## Plan + +Decided 2026-11-01 (maintainer, /quest-plan; see the +[epic](/quest/m0/queue-watcher/README.md)). + +- Poll, don't receive webhooks: the host has no public endpoint. Reconcile + from the picks, not from queue snapshots alone: a PR enqueued and dropped + between two polls, or while the watcher was down, never shows in a + snapshot. Each pass reads every open PR holding an active pick (the marker + [Picks enter the merge queue](/quest/m0/queue-watcher/pick.md) records) with + its `mergeQueueEntry` and its `RemovedFromMergeQueueEvent`s newer than the + last one handled, with `reason`, `actor`, and `enqueuer`. `quest status` + already reads that event (`src/cli/status.ts`). The base's + `mergeQueue { entries }` can still serve as a cheap change signal. The App's + installation token has its own rate limit, separate from the maintainers' + sessions. +- For a failed build, find the PR's `merge_group` workflow runs (event + `merge_group`, head branch `gh-readonly-queue//pr--`) and + fetch the failing jobs' logs through the App's `actions: read`, trimmed the + way the host gate trims a batch failure. At build concurrency 1 the group + holds only this PR, so the failure is its own. +- Classify the reason: a failure or a conflict goes to the fixer + ([Fix and re-enqueue](/quest/m0/queue-watcher/fix.md)); a person's removal, + or a push by anyone but the fixer, ends the pick and is left alone. +- Keep each pick's last handled removal event ID in the daemon's state + directory, as the host gate keeps its queue state, so a restart neither + misses nor repeats one. +- Start it as `quest land watch` beside `quest land serve`, sharing its App + credential options, state directory, and comment marker and + deduplication. + +## Required + +- [Picks enter the merge queue](/quest/m0/queue-watcher/pick.md) - the durable pick marker the watcher reconciles from diff --git a/quest/m0/queue-watcher/retire.md b/quest/m0/queue-watcher/retire.md new file mode 100644 index 00000000..9664692c --- /dev/null +++ b/quest/m0/queue-watcher/retire.md @@ -0,0 +1,26 @@ +# [M] Retire the batching host gate + +## Goal + +Condition: no repository lands through `quest land serve` any more (moq, +moq-dev/quest, moq.pro, rust2 and rind have all switched to the merge queue). + +Then the batching gate is gone: `quest land serve`'s batching, parking, +halting, and squash-push code, its tests, and its README and `CONTRIBUTING.md` +sections are deleted, keeping only what the watcher and `quest land apply` +use. The host's `quest-land` daemon instances and check containers are +removed from kixelated/nixos, while the fixer container stays. Nothing still +requires the unreported `quest land` check, and `quest merge --land` no longer +special-cases it. + +## Plan + +Decided 2026-11-01 (maintainer, /quest-plan; see the +[epic](/quest/m0/queue-watcher/README.md)): the merge queue replaces the +batched gate, and gate work is frozen until then. Once nothing uses it, the +code goes rather than staying as an unmaintained option. Check each +repository's ruleset for a `quest land` requirement before deleting. + +## Required + +- [The other repositories follow](/quest/m0/queue-watcher/follow.md) - the last repositories leave the gate diff --git a/quest/m0/queue-watcher/stop.md b/quest/m0/queue-watcher/stop.md new file mode 100644 index 00000000..24f6ce02 --- /dev/null +++ b/quest/m0/queue-watcher/stop.md @@ -0,0 +1,41 @@ +# [S] The fixer keeps going while it makes progress + +## Goal + +The watcher keeps fixing and re-enqueuing a dropped PR, with no attempt +limit, for as long as each round changes something. It hands the PR back to +a person when: + +- a fix attempt produces no new commit and the failure isn't flaky; +- the same failure (the same failing test or step) recurs right after a fix; +- or the PR has been re-enqueued unchanged as flaky 3 times in a row. + +The fixer handing back on its own (a design decision, a protected edit) still +hands back at once. A handback comment names the rule that fired. + +## Plan + +Decided 2026-11-01 (maintainer, /quest-plan; see the +[epic](/quest/m0/queue-watcher/README.md)): + +- How attempts count: + - ✅ Unlimited while progressing, with the three stop rules above. This + replaces batch triage's ([moq-dev/quest#355](https://github.com/moq-dev/quest/pull/355)) + one attempt per pick, and the earlier plan to spend that attempt only on + a failed fix (2026-11-01, from the audit of 98 "already used its + mechanical fix attempt" handbacks), which this quest held before the + switch. + +Extend the attempt record `fix-attempt-.json` (`src/cli/land-fixer.ts`, +read by `spent` and `handbackRepeated`) into a per-pick history: each round's +outcome (`fixed`, `merged`, `unchanged`, `flaky`, or a handback), the head +before and after, and the failure's signature. The signature is the failing +test or step the fixer names, falling back to the failing job and step from +the `merge_group` run. A record written by the host gate, without these +fields, starts a fresh history. + +A conflict after the base moved is a new failure, not a repeat, so a PR that +keeps conflicting with a busy base keeps being fixed while each fix lands a +new merge. A flaky round resets neither of the other rules' state; three +flaky rounds in a row hand back so a real nondeterministic bug in the PR +reaches a person.