quest: close the merge-queue landing quest; plan queue concurrency, libav sources, bot review cadence

.github/workflows/queue.yml

@@ -3,7 +3,8 @@ name: Queue
 # GitHub's merge queue tests each picked PR on top of main, one at a time, on
 # the host's self-hosted runner, and merges exactly the tree it tested. When the
 # queue drops a PR, moq-dev/quest's queue watcher on the host runs the land
-# fixer and re-enqueues it (quest/m0/ci/land.md).
+# fixer and re-enqueues it
+# (https://github.com/moq-dev/moq/blob/17a81d6c64/quest/m0/ci/land.md).
 #
 # `main`'s ruleset requires this job's `Queue` context. GitHub admits a PR to
 # the queue only once its required checks pass on the PR itself, so the job

quest/m0/README.md

@@ -31,7 +31,7 @@ edge, which embeds moq-relay; it moved there on 2026-09-28.
 - [Backport audio groupDuration](/quest/m0/backport-audio-group-duration.md) - `@moq/publish` 0.5.4 on `release` can pack several Opus frames per group, making Cloudflare's relay drop audio less often
 - [Backport the SETUP-path and iroh-key fixes](/quest/m0/backport-security-fixes.md) - `release` refuses a conflicting SETUP Path and keeps persisted iroh keys private, in one patch release
 - [Backport the joined-head FETCH race fix](/quest/m0/backport-fill-head-race.md) - a `release` relay stops FETCHing a group head its own SUBSCRIBE is bringing, which left live readers hanging
-- [CI cost and latency](/quest/m0/ci/README.md) - cloud CI runs only in a batched merge gate as one `just ci` pass per batch, expensive checks run nightly and gate `release`, caches move to R2, and runners are chosen last
+- [CI cost and latency](/quest/m0/ci/README.md) - cloud CI runs only in GitHub's merge queue, one tested entry at a time, expensive checks run nightly and gate `release`, caches move to R2, and runners are chosen last
 - [More tests under load](/quest/m0/test-flakes-2/README.md) - load-only and queue failures, one quest per flake, fixed at the cause, since each one fails a whole batch
 - [Benchmark sessions can't overlap on a host](/quest/m0/bench-host-lock.md) - a host-level lock keeps a second benchmark or profile session from measuring under the first's load
 - [A waiter's record limit changes no behavior](/quest/m0/kio-waiter-records.md) - two moq-net tests stop failing when kio's waiters remember 16 lists, and the limit becomes a tuning knob worth 3.9% per delivered group

quest/m0/ci/README.md

@@ -3,7 +3,7 @@
 ## Goal
 
 Pull requests are validated locally with `just ci` and get their cloud verdict
-from the [merge queue](/quest/m0/ci/land.md) without waiting behind the
+from the [merge queue](https://github.com/moq-dev/moq/blob/17a81d6c64/quest/m0/ci/land.md) without waiting behind the
 hosted runner limits, and every job spends its minutes on the work its diff
 selected rather than on setup, cold caches, or repeated builds. Concurrent local agents share
 expensive-command capacity, and retained tests earn their cost through useful
@@ -71,13 +71,23 @@ Decided 2026-10-26 (maintainer), from 414 queue merges over 2026-10-23 to
 (about 1 minute per merge), with a median of 18 entries ahead and 12.7% of
 runs failing. GitHub's merge queue cannot test several PRs in one run, so a
 batched gate replaces it: `quest land`, shared through moq-dev/quest, lands
-a tested chain of squash commits ([land](/quest/m0/ci/land.md)). Decided
+a tested chain of squash commits ([land](https://github.com/moq-dev/moq/blob/17a81d6c64/quest/m0/ci/land.md)). Decided
 2026-10-30 (maintainer): it runs as a daemon on the maintainer's host, not as
 a workflow on the self-hosted runner. This supersedes concurrent queue
 entries on a second runner. Batching gains most when failures are rare, so the
 [flake epic](/quest/m0/test-flakes-2/README.md) moved to m0 and
 `MIGRATE.md` Unreleased sections, a top source of conflicts, stop being
-written per PR.
+written per PR. The batched gate was superseded 2026-11-01, below.
+
+Decided 2026-11-01 (maintainer), reversing 2026-10-26: moq serializes
+through GitHub's merge queue again, one entry at a time on the self-hosted
+runner, and moq-dev/quest's queue watcher resolves dropped PRs' conflicts and
+re-enqueues them. Batching and pre-checks are dropped, and the host gate is
+frozen. The landing quest closed 2026-11-04 with busy hours merging a median
+11.0 PRs against the queue baseline's 6.0 (its
+[last version](https://github.com/moq-dev/moq/blob/17a81d6c64/quest/m0/ci/land.md)
+holds the cutover record); a second concurrent entry is a
+[measured trial](/quest/m0/ci/queue-concurrency.md).
 
 Decided 2026-10-23 (maintainer), to speed up the serial queue: structural
 changes first (concurrent entries, parallel modules, one test build), then a
@@ -152,7 +162,6 @@ The historical samples locate candidates; they are not controlled benchmarks.
 ## Required
 
 - [Read-only pull-request CI](/quest/m0/ci/read-only-secrets.md) - write secrets and publisher identities reach only protected branches and tags, never a pushed branch
-- [Land main through GitHub's merge queue](/quest/m0/ci/land.md) - one PR at a time on the self-hosted runner; the host's Codex fixer fixes and re-enqueues every dropped PR
 - [A cache error is a miss](/quest/m0/ci/nix-cache-5xx.md) - a transient `nix.moq.dev` 5xx no longer fails `just ci`
 - [Gate each diff in under five minutes](/quest/m0/ci/five-minutes.md) - `just check`, test phases included, finishes within a worker's 5-minute prompt cache, admission wait included
 - [Measure the parallel frontend](/quest/m0/ci/nightly-levers.md) - a pinned nightly's parallel frontend, adopted only if the numbers beat its risks; Cranelift is ruled out
@@ -165,3 +174,5 @@ The historical samples locate candidates; they are not controlled benchmarks.
 - [The workspace-hack refuses trybuild edges and spares non-media builds](/quest/m0/ci/hakari-followups.md) - `hakari.py` refuses trybuild dev-dependency edges, and non-media Linux test builds stop needing libpipewire and libclang
 - [A release merge opens no stale release PR](/quest/m0/ci/release-pr-after-release.md) - `just rs release` publishes before it proposes, so a merged release isn't re-proposed
 - [Renovate keeps the workspace-hack generated](/quest/m0/ci/dependabot-workspace-hack.md) - a protected scheduled Renovate runs `cargo hakari generate` in cargo update PRs, replacing Dependabot's cargo updates
+- [Build two queue entries at once if it helps](/quest/m0/ci/queue-concurrency.md) - a measured trial of `maximumEntriesToBuild: 2` against the serial lane's burst latency
+- [CI never downloads libav's sources](/quest/m0/ci/libav-sources.md) - `moq-libav` substitutes from `nix.moq.dev`, so an upstream mirror can't fail Check

quest/m0/ci/five-minutes.md

@@ -154,4 +154,3 @@ What remains:
 ## Related
 
 - [Unify dependency features](/quest/m0/ci/hakari.md) - cache hits across feature shapes
-- [Land main through GitHub's merge queue](/quest/m0/ci/land.md) - the queue gate that keeps checks moved out of the local gate

quest/m0/ci/libav-sources.md

@@ -0,0 +1,26 @@
+# [XS] CI never downloads libav's sources from upstream mirrors
+
+## Goal
+
+A `just ci` run never fetches `@moq/libav`'s source tarballs from their
+upstream sites: `moq-libav` (or at least its fixed-output sources) substitutes
+from `nix.moq.dev`, so a flaky upstream mirror can't fail a check. Done when a
+clean checkout's Check builds `moq-libav` with no request to `ffmpeg.org`,
+`downloads.xiph.org` or GitHub archives.
+
+## Plan
+
+Planned 2026-11-04 with the maintainer. Check failed twice on 2026-11-03/04
+for this alone: [moq-dev/moq#7685](https://github.com/moq-dev/moq/pull/7685)
+and [moq-dev/moq#7760](https://github.com/moq-dev/moq/pull/7760) (run
+41777407147: "cannot download ffmpeg-9.0.tar.xz from any mirror", then
+`moq-libav` and the `moq` postinstall failed). `js/libav/package.nix`
+`fetchurl`s `ffmpeg-9.0.tar.xz`, `opus-1.6.1.tar.gz` and
+`emfiberthreads-1.3.tar.gz` whenever `moq-libav` isn't substituted.
+
+Find why `moq-libav` was built at all: whether `sh/gh/cache.sh` pushes it (and
+its sources) with the dev shell's build closure, or whether its derivation
+changed without being pushed. Prefer pushing the built `moq-libav` and its
+source fixed-output derivations to `nix-ci-moq-dev` over mirroring tarballs
+by hand. Related: [A cache error is a miss](/quest/m0/ci/nix-cache-5xx.md)
+covers `nix.moq.dev` errors, not upstream ones.

quest/m0/ci/nightly-levers.md

@@ -28,7 +28,3 @@ explanation of nightly incremental rebuilds costing near-full builds, before
 running the queue experiment.
 
 Public API: none. Wire: none.
-
-## Required
-
-- [Land main through GitHub's merge queue](/quest/m0/ci/land.md) - the structural baseline, replacing concurrent queue entries

quest/m0/ci/queue-concurrency.md

@@ -0,0 +1,33 @@
+# [S] The merge queue builds two entries at once if it measurably helps
+
+## Goal
+
+A measured trial decides whether moq's merge queue builds two entries at once
+(`maximumEntriesToBuild: 2`) or stays at one. Done when the PR records both
+settings' burst p90 enqueue-to-merge latency and median merges per
+never-empty hour over comparable windows, and the maintainer keeps the winner
+in the ruleset.
+
+## Plan
+
+Planned 2026-11-04 with the maintainer. The landing quest
+([last version](https://github.com/moq-dev/moq/blob/17a81d6c64/quest/m0/ci/land.md))
+ran the queue at build concurrency 1 and said to try 2 only with measured
+data. Its performance goal closed on that data (decided 2026-11-04 ✅): over
+2026-11-03 06:47Z to 2026-11-04 06:47Z, the eight hours the queue was never
+empty merged a median of 11.0 PRs against the queue baseline's 6.0.
+
+Why try 2: with a median queue build of 4.4 min, one lane tops out near 13
+merges an hour, and busy hours already run close to that. Bursts of 22-26
+entries on 2026-11-03 pushed p90 enqueue-to-merge to about 58 min (median
+20.6 min), and waiting behind other entries was most of it.
+
+Method: occupancy comes from the PRs' `AddedToMergeQueue`,
+`RemovedFromMergeQueue` and `Merged` timeline events (reconstruct removed and
+unmerged entries too). Compare at least one busy day at each setting, and
+count queue failures per build, since building two means a failure in the
+first entry rebuilds the second. The ruleset change and revert are the
+maintainer's actions (ruleset 2420853); the quest names the setting and
+reports the numbers. The host runner must handle two builds at once without
+the shared mbx cache or `quest gate` admission erasing the gain, so record
+host load during the trial.