Repository navigation
ci: skip the merge queue's checks for trees Pull CI already passed - #1288
Conversation
ac-pplx
left a comment
There was a problem hiding this comment.
The problem. Every merge-queue entry reruns Pull CI's rust_static and common workflows on the merge-group commit, which takes about six minutes. When nothing has landed on main between the PR's last Pull CI run and the enqueue, that commit has the same tree as the synthetic merge Pull CI already passed, on a base with the same tree. The rerun then repeats work on identical inputs. This PR has Pull CI record each pass as a one-day artifact named after the two trees. The queue skips both workflows when it finds a trusted record for the entry's trees.
Example.
10:00 PR A pushed, main at X. Pull CI tests M = merge(X, A), passes, and uploads
pull-ci-passed-<tree(X)>-<tree(M)>
10:40 A enqueued, main still at X. Merge group G = squash(A) on X, so tree(G) == tree(M)
before: rust_static and common rerun on G (~6 min)
after: reuse finds the record, both are skipped, and space-pull-ci, which requires both
to be skipped, passes
If main has moved to Y first: tree(Y) != tree(X), so no record matches and the checks run as today.
Verdict: approved; 2 threads to resolve before merge.
The tree comparison is sound:
- A tree includes
.github/and the scripts, so equal trees run the same jobs. - Pull CI runs a superset of the queue's steps, because archcheck runs only on
pull_request. - Both sides resolve the migration guards' base to the same tree.
resolve-migration-base.shuses the synthetic merge's first parent, and the queue passesmerge_group.base_sha. - Every error, unknown output, and failed or cancelled lookup I traced runs the checks.
- The new tests are collected, and they ran in this PR's Python tests job.
Is it worth it? By the description's own count, 3 of 35 entries would each skip about six minutes, and every other entry now starts its checks one short job later (item 2). That is still a net saving, mostly in 16-core runner minutes and for stacks, and I think the mechanism is worth keeping.
Before merge (inline threads)
- Three trust checks have no test that fails without them
- Every entry that does not reuse now waits for the lookup; say so and bound it
Nits
- A failing
nameis silent..github/workflows/pull-ci.yml:77. A failed command substitution insideprintf's arguments does not triggerset -e. The step stays green, writesname=, and records silently stop. That is safe, because the queue then runs the checks. Assign the name first,name="$(python3 -I ci/scripts/pull-ci-reuse.py name)", thenprintfit. - Gate the record on the results it certifies.
.github/workflows/pull-ci.yml:126. The record steps run because the step above them exited 0. Today that impliesrust_staticandcommonpassed. Addingneeds.rust_static.result == 'success' && needs.common.result == 'success'to both steps'if:keeps that meaning if the step above ever gains another way to pass. - The stack claim is broader than the mechanism.
.github/workflows/merge-queue-ci.yml:23and the description. A push to a lower PR does not rerun Pull CI on the PR above it, so that PR's entry finds no record and runs its checks. That is safe, but the claim is broader than what happens. Suggested wording: "for a stack whose bottom PR was on main's head, each entry whose last Pull CI ran on the current head of the PR below it". - The upload step has no timeout.
.github/workflows/pull-ci.yml:132.continue-on-errorcovers a failure but not a stall. Before this PR, the requiredspace-pull-cijob made no network calls that could hang. Addtimeout-minutes: 2to the step; it took 1.5 s in this PR's run.
A merge-queue entry reruns Pull CI's rust_static and common jobs on the merge group's commit. When main has not moved since the PR's last Pull CI run, the merge group's tree is the synthetic merge's tree that run tested, against a base with the same tree, so the queue reran the same jobs on identical inputs. A stack enqueued after its CI started one such run per PR. Pull CI now records each pass as an artifact named after the base and synthetic-merge trees, kept for a day. A new reuse job in Merge Queue CI looks the entry's trees up and skips rust_static and common when it finds a record from a Pull CI run of this repository's own branches, not a fork's, whose latest attempt passed space-pull-ci. A missing, untrusted or unreadable record runs the checks as before. Main CI still runs every check after the merge. Every other entry now starts its checks after that lookup: a standard runner's pickup, a one-file checkout and three API calls, each capped at 15 s, with the job capped at two minutes.
53a3b8a to
6ae64e0
Compare
|
Thanks for the review. Both threads are fixed in 6ae64e0, along with the nits:
|
Summary
The merge queue no longer reruns Pull CI's checks on a tree that Pull CI already passed. An entry whose trees match such a run takes two short jobs instead of about six minutes. That is rare: 3 of the 35 successful entries since 2026-10-31 03:00 would have qualified.
Cost to every other entry. Its checks now start after the lookup job: a standard runner's pickup (2-8 s in recent queue runs), a one-file checkout, and three API calls capped at 15 s each. The job is capped at two minutes, after which the checks run anyway. So the queue's median and p90 rise by about that job's duration, while the entries that qualify save about six minutes and their 16-core runner time.
Why. A merge-queue entry runs Pull CI's
rust_staticandcommonjobs again on the merge group's commit. Successful Merge Queue CI runs have a median of 5.8 minutes over the last 22. The rerun is worth it when main moved after the PR's last Pull CI run: two PRs that each pass Clippy can still fail it together.When main has not moved, the merge group's commit has exactly the tree Pull CI tested. For example, #1267's squash commit on main,
0456537, has the tree of the synthetic merge its last Pull CI run tested (c8efea9…). Enqueuing a stack makes this worse: GitHub builds one cumulative merge group per PR, each with its own run, so a two-PR stack ran these checks twice more.What changes.
kind_paths, which has the synthetic merge checked out, names the record after the base tree and the merge tree (ci/scripts/pull-ci-reuse.py name).space-pull-ciuploads it as an artifact once every check passed, kept for one day.rust_staticandcommonpassed.reusejob takes the trees of the entry's base and merge-group commit and looks for a record (pull-ci-reuse.py lookup). It trusts a record only when all of these hold:.github/workflows/pull-ci.ymlon apull_requestevent;space-pull-ci.rust_staticandcommonare skipped, andspace-pull-cirequires them to be skipped. Otherwise, including any API error or malformed answer, they run and must pass, as today.common's migration guard compares against the base.When it applies.
How often. Of the 35 successful entries since 2026-10-31 03:00, main had moved between the PR's last Pull CI run and the entry in 32. In all 32, the commits in between changed Rust inputs, so these checks had new inputs to test. The merge queue's slowest runs before that date waited 3-17 minutes for a 16-core runner, and raising the runner limit removed that; none since took over 7.6 minutes.
Validation
scripts/tests/test_pull_ci_reuse.pydrives the script against a scriptedgh. It passes 15 tests:space-pull-cifailed while another job passed, an unreadable or malformed API answer;lookupwithoutGITHUB_REPOSITORYexits 2;namegives a synthetic merge's base and merge trees in a git fixture.reuseas intended.Reuse a passing Pull CI runfinding the record and both checks skipped.Severity and risk
Severity: does not apply; this cuts merge-queue time and runner use.
Risk:
cargo denyadvisory database, within the day a record lives. Main CI runs them after the merge, as it does today when an advisory appears after the merge queue.space-pull-cicheck, which comes from the same run. Fork runs are ignored.AI assistance: Claude Code drafted the change, its tests and this description.
🤖 Generated with Claude Code