Skip to content

feat(pr-pipeline): merge lanes App pull requests through native auto-merge - #9508

Merged
kyle-sexton merged 17 commits into
mainfrom
feat/native-lane-prs
Oct 24, 2026
Merged

kyle-sexton merged 17 commits into
mainfrom
feat/native-lane-prs

Conversation

@kyle-sexton

@kyle-sexton kyle-sexton commented Oct 23, 2026 •

Copy link
Copy Markdown
Member

No related issue: completes the native merge path after #9450, #9459 and #9488

Summary

Fix

  • Link eligibility (evidence --lane-link). Before the verdict, the reader reads the link by babysit_merge_policy._linked_issue: one closing issue here, or, with none, the one issue plain Refs: lines name. A refused or unreadable link reads no verdict and leaves the pull request unarmed, or disarms it.

  • Run-level refusals (lane-room, babysit_merge_policy.lane_room). Each scheduled or hand-started run reads this run's post-merge-dispatch report. It then applies the gate's run-level refusals, in the gate's order:

    • another authority;
    • a post-merge failure;
    • the effective rung off: merge.rung, the loop setting, the rung cap (vars.CLAUDE_LANES_MERGE_RUNG_CAP, passed as the merge job passes it), or a demotion record, an unreadable one included.

    While the gate holds its lane off, nothing is armed and every armed lane pull request is disarmed. At merge.max-in-flight nothing more is armed. The count is the gate's (the queue's lane entries plus the lane's unverified merges) plus armed pull requests the queue has not taken yet.

    When a read fails (a missing or unparsable post-merge report, config, or merge queue), the room is unknown, and unknown holds (owner Q677): nothing is armed and every armed lane pull request is disarmed, as on a refusal. Event runs read no room and never disarm on one.

  • Risk-score hold (evidence --lane-reviews, lane_risk_refusal). After the verdict allows, the reader runs the gate's babysit_risk.score and holding over the inputs the gate builds, at the lane room's rung. The review lanes' evidence is bound to the head as the gate binds it (babysit_merge._lane_evidence). Each of these holds:

    • an open CRITICAL security finding, or a lane not bound to the head;
    • a symlink, submodule or other non-regular mode;
    • a high path, the manifest review, the trust gate, or the cost review.

    claude-review findings are reported only.

  • Link binding in the merge queue (merge-holds.mjs linkHold). verdictHold accepted any pass at the head, so an armed Refs: #A pull request edited to Closes #B merged closing #B on a pass that judged #A. Now ci-status's merge-group step reads the counted pass's verdict.json (one artifact download) and the closing issues (one GraphQL read). It holds unless the live link, by the gate's rule, is the issue the pass judged, linked the same way. This is the reader's verify:spec-changed rule. A trusted approval of the head lifts it, as it lifts the verifier hold. refsIssue moves into merge-holds.mjs, which ci-status fetches as one file, and resolve-pr.mjs re-exports it, so the parity test with refs_issue still covers it.

  • CRITICAL security hold (owner Q693). Before this, the native path held a CRITICAL finding only through the needs-human label. The chain:

    • plugins/review/skills/security-review/SKILL.md:231 (critical-security) tells the model when to hold;
    • ci-workflows pr-review-cli.yml (v0.65.0) applies the label only when the model's hold field says so;
    • pr-require-checks.yml:3712 (human_gated) fails ci-status while the label is on;
    • enable-auto-merge holdReason skips a labelled pull request.

    ci-workflows' apply-review-hold has no severity trigger of its own, so a missed hold let a CRITICAL finding through. The gate counts the findings themselves, and --lane-reviews now does the same on the native path.

  • Default branch. A comment at PULLS_QUERY says it assumes main is the default branch; the gate reads it live (_base_refusal).

  • Permissions. The enable-auto-merge job gains pull-requests: read and checks: read. lane-room runs under the job's GITHUB_TOKEN. The review-lane reads split tokens as the merge job's do: the timeline, comments and lane runs under the job's GITHUB_TOKEN (BABYSIT_VERIFY_GH_TOKEN), and the config, changed-file and permission reads under the lane token (GH_TOKEN).

  • API budget. The job-end step logs only the Python gh calls. createGitHub and head-ref-map.mjs record nothing, so event runs log no line and the lanes-app figure leaves out the JS calls. The second commit's message says the step "reports both"; it reports both tokens' Python readings only.

Cost (measured). On each scheduled run the job reads the lane's room from the post-merge report that the same run's post-merge-dispatch job keeps as a one-day artifact (needs: post-merge-dispatch, downloaded with no run-id, so only this run's artifact is read). It runs no post-merge check of its own. With no report, nothing is armed and every armed lane pull request is disarmed.

New calls per scheduled run:

The post-merge check adds none. Measured locally and read-only.

Event runs skip post-merge-dispatch and run as before; they never arm.

Timeouts. The arm step has 14 minutes (the room read, then the verdict budget). The job has 25, and workflow.test.mjs checks that the step timeouts fit inside it. On the schedule the job now starts after post-merge-dispatch ends; that job took 97-127 s on recent runs (its own comment).

Open question for the owner. --remediate dequeues the lane's queue entries but leaves armed pull requests armed, so GitHub can enqueue one before the next reader run disarms it. The coordinator is filing that follow-up.

Verification

  • node --test .github/scripts/pr-merge/*.test.mjs .github/scripts/pr-verify/*.test.mjs: 342 pass, 0 fail. New tests:
    • through the real CLI and scorer: a lane pull request adding a symlink is not armed, an armed one adding a submodule pointer is disarmed, and a doc change arms;
    • while the gate holds the lane off (demotion, rung cap, post-merge failure), nothing arms, armed pull requests are disarmed, and no verdict is read;
    • at the cap nothing more arms, and an armed pull request already in the queue is not counted twice;
    • an unreadable room arms nothing and disarms every armed lane pull request (the old order, which left them armed, fails it);
    • an event run leaves an armed lane pull request armed;
    • an event run reads no room;
    • pythonRoom passes the rung cap and the job token;
    • linkHold cases from the README "Eligibility" rule;
    • a review-lane hold blocks the arm and disarms;
    • the wiring is pinned in workflow.test.mjs.
  • scripts/ci-lanes-pr-hold.test.sh: 121 pass. The kill switch on (either variable, empty or any value but false) fails a lanes App pull request but not a person's or Dependabot's, and the step reads both switches from vars; making the check inert fails 4 cases. New merge-group cases:
    • a Refs: pass passes;
    • Refs: #1 edited to Closes #2 fails;
    • a Refs-slice pass does not cover the issue closed;
    • two Refs: lines or two closing issues hold;
    • a bad or undownloadable record fails closed;
    • a trusted approval lifts.
  • ci-hold-merge-group, ci-hold-cost-decision and ci-hold-lifted-by pass: 22, 72 and 33 cases.
  • Python: python3 -m unittest discover -s tests -p 'test_babysit_merge*.py': 945 OK. tests.test_babysit_lane_reviews, tests.test_babysit_lane_room, tests.test_babysit_verify_evidence, tests.test_verify_rules_sync, tests.test_merge_holds_sync and tests.test_babysit_merge_policy: 646 OK.
  • Breaking each new hold on purpose fails its tests:
    • effective_rung always C2: 3 lane-room tests fail;
    • roomHold inert: 3 JS tests fail;
    • linkHold returning null: 5 step-body cases fail.
  • scripts/run-ruff.sh check passes, and actionlint is clean on both workflows.
  • Live, read-only:
  • lane_room reads its config, default branch and cap through the run cache. carry-run-cache.mjs:87 carries only carried-run-record, carried-fail-head-ref and carried-on-branch, so a poisoned cache entry cannot change the rung or the cap.
  • Live: lane-room on this repository answered {"refusal": null, "unreadable": null, "count": 0, "cap": 5, "queued": [9510]}.
  • A lanes-App auto-merge enqueues with the lanes App as enqueuer (docs(harness-ops): add harness-ops:prerequisites row to README skills table #9475's AddedToMergeQueueEvent), so the in-flight count sees native merges.
  • The job token's permissions for these reads are unverified until a live scheduled run.

Security notes.

  • A body can name any issue on its Refs: line. That only passes the link check: a pass must still bind with spec_kind: refs to a trusted spec, and in the queue it must still bind the live link.
  • Every new read fails closed: no arm. A refusal derived from the gate's own reading disarms.
  • ci-status runs linkHold from the base commit's merge-holds.mjs, so a pull request cannot rewrite it.

Review at ready

The local code and security reviews at the ready flip (no CI review lane ran on this repository) found these. Fixed in 62c6679:

  • lane_room used the post-merge report without the gate's reuse checks. A re-run of enable-auto-merge downloads the run's first report, up to a day old, so a demotion filed since went unseen. It now applies shared_refusal's age rules and _demotion_changed, and an answer that fails them reads as unreadable, which holds.
  • The report was read only when post-merge-dispatch succeeded. A failed full-run step after the report was kept disarmed every armed lane pull request, which the gate does not do. It is now read whenever that job ran.
  • The merge queue's relink hold said pr-verify judges the new link. pr-verify skips a head it already passed, so the hold now says to push a new head or get an approval.

Fixed in 10c0940 (owner-approved parity with the gate):

  • Risk-score parity. evidence --lane-reviews now runs the gate's whole babysit_risk.score() and holding() over the inputs the gate builds for a lane pull request (score_unattended). It scores at the lane room's effective rung (now in lane-room's report), uses the base config tightened by main's, and passes the cost decision on the linked issue. So these hold Closes and Refs: pull requests alike:

    • HIGH from non-regular modes, such as a symlink or a submodule pointer (_mode_problems);
    • configured high paths, the manifest review, the trust gate and the cost review;
    • the review lanes' evidence, as before.

    Any read error holds. Cost: 38 gh calls per pull request whose verdict already allows, measured read-only on feat(source-control)!: ship the pr_open_state default as ready #9483 including its pull-request and config reads. That is up from 36 for the lane-evidence read alone: one base-tree read for Pulumi directories and the manifest contents. It is spent only on pull requests every other rule would arm or keep armed.

  • Kill switch. ci-status's lanes App step fails while CLAUDE_LANES_DISABLED or CLAUDE_LANES_REPO_DISABLED reads anything but false, the check-kill-switch rule. An armed lane pull request therefore stops in the merge queue. It reads the vars context: no API call. Owner and Dependabot pull requests exit before the check.

Recorded for the owner, not changed here:

  • Body edit after the queue check. An edit after ci-status passes in the merge queue, but before GitHub merges, changes the issue the merge closes. No required check can close a window that opens after it completes; it is the same window the body hold has.

Related

🤖 Generated with Claude Code

kyle-sexton and others added 8 commits October 23, 2026 17:50
…merge

ci-status and enable-auto-merge now require an exact-head pr-verify pass
(#9450), post-merge judges a native merge (#9459), and the native path
applies the hold floor (#9488), so the gate no longer needs to list lanes
App pull requests.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
enable-auto-merge no longer requires a body opening with Closes. The
scheduled reader runs babysit_verify_evidence.py evidence --lane-link,
which reads the issue link by babysit_merge_policy._linked_issue (one
closing issue here, or one plain Refs: issue) before the verdict, and
arms only when both allow. A refused or unreadable link reads no
verdict and leaves the pull request unarmed or disarms it. Event runs
read no link, so an armed Refs pull request is never disarmed on one.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The job logged no api-budget line. Its arm step now names the lane and
job tokens' readings for the reader's gh calls, and a job-end
log-api-budget step, as the merge job has, reports both.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…d in-flight cap

enable-auto-merge skipped the gate's run-level refusals, so after a
demotion the next scheduled run re-armed clean lane pull requests. Each
reader run now runs the post-merge check the merge job runs and reads the
lane's room from it through babysit_merge_policy.lane_room. While the gate
holds the lane off nothing is armed and armed lane pull requests are
disarmed; at merge.max-in-flight nothing more is armed; an unreadable
room arms nothing and disarms nothing. PULLS_QUERY now notes that it
assumes main is the default branch.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…ive link

verdictHold accepted any pass at the head, so an armed Refs: #A pull
request whose body was edited to Closes #B merged closing B on a pass
that judged A. ci-status's merge-group step now reads the counted pass's
verdict.json and the closing issues and holds unless the live link, by
the gate's rule, is the issue the pass judged, linked the same way.
refsIssue moves into merge-holds.mjs, which ci-status fetches alone, and
resolve-pr.mjs re-exports it.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The native path held a CRITICAL security finding only through the
needs-human label, which the security lane applies only when its model
fills the hold field. The gate holds on the counted finding itself.
enable-auto-merge's reader now runs evidence --lane-reviews, which binds
both lanes' evidence to the head as the gate does and applies the gate's
lane rule, so an open CRITICAL finding or an unbound review leaves the
pull request unarmed or disarms it. The README describes the new holds.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…s timeouts

The post-merge check gets its own 6-minute timeout, the arm step 14
minutes for the room read plus the verdict budget, and the job 30, with
workflow.test.mjs checking that the step timeouts fit inside the job's.
The verdict budget now starts after the room read. The pr-pipeline.yaml
activity comment and the job header comment name the new holds.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
kyle-sexton and others added 5 commits October 23, 2026 18:48
Running the post-merge check a third time per scheduled run added about
350 GITHUB_TOKEN requests an hour against a 1,000 an hour limit.
post-merge-dispatch now keeps its report as a one-day artifact, and
enable-auto-merge needs that job and downloads the report from this run
only. With no report the room is unreadable: nothing is armed and
nothing is disarmed. Event runs, which skip post-merge-dispatch, run as
before. The job drops issues: read and its timeout falls to 25 minutes.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…m is unknown

roomHold returned early for an armed pull request before it checked an
unreadable room, so a missing post-merge report or a failed read left
armed lane pull requests armed while the gate refused. Unknown holds:
on a reader run an unreadable room now disarms every armed lane pull
request and arms nothing, as a refusal does. Event runs read no room and
are unchanged.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…e would not reuse

lane_room read the demotion and post-merge refusal straight from the
run's post-merge report. A re-run of enable-auto-merge downloads the
run's first report, up to a day old, so a demotion filed since went
unseen and lane pull requests armed. lane_room now applies the gate's
own reuse checks (shared_refusal's age rules, now one helper, and
_demotion_changed) and reads an answer that fails them as unreadable,
which holds.

The report is read whenever post-merge-dispatch ran, not only when it
succeeded: its full-run steps can fail after the report is kept, and
the gate does not refuse on them.

The merge queue's relink hold no longer says pr-verify judges the new
link: it skips a head it already passed, so a new head or an approval
recovers it.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@kyle-sexton
kyle-sexton marked this pull request as ready for review October 23, 2026 23:24
@chatgpt-codex-connector

chatgpt-codex-connector Bot commented Oct 23, 2026 •

Copy link
Copy Markdown

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review Status Commit Review trigger
📝 Code Review ✅ Completed 2026-10-23T23:26:36.045026Z a553dfc Draft marked ready
ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: a553dfcdaa

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread plugins/source-control/skills/babysit-prs/scripts/babysit_merge_policy.py Outdated

@github-actions github-actions Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

claude-security-review: 1 inline finding(s) on a553dfcdaa0a84e261f7cc1d5adc69f8a5237c9d; the summary is in this lane's pull request comment.

claude-lane: claude-security-review

Comment thread docs/conventions/pr-pipeline.yaml
@github-actions

github-actions Bot commented Oct 23, 2026 •

Copy link
Copy Markdown
Contributor

claude-security-review reviewed d3c80641d3f89c8cee674e13c606272e3b1747b8: 0 finding(s), 0 as inline comments.

No security issues found in the incremental change (since 2948665). critical-findings: 0

Incremental scope. The change touches babysit_merge_policy.py (lane_room), its test, the change note, and the pr-pipeline README. The README and change note are prose that describe the new behaviour.

lane_room now calls shared_refusal. It previously ran only _shared_age_refusal, an identity comparison and _demotion_changed. I read shared_refusal (babysit_merge_policy.py:3955-4041). It runs every check the old code did, plus these:

  • the binding checks: identity, self-logins and lane;
  • the config-floor comparison;
  • a re-read of the default-branch head, with a scan for lane merges the report did not count (new-merge:#<n>).

Every refusal it returns becomes LaneRoom(unreadable=...). The caller treats unreadable as "hold: arm nothing and disarm every armed lanes App pull request". The change therefore only adds ways to refuse. It does not loosen any gate or remove any control.

Fail-closed behaviour.

  • A missing agent identity now returns unreadable where it used to return the generic identity-mismatch text. It still holds.
  • If shared.self_logins is non-empty, it differs from the frozenset() passed here and yields self-logins-differ. That holds the lane. It is a possible availability problem, not a security one.
  • The remaining reads use _get and newest_config_commit. Their failures map to config-unreadable, merged-unreadable or demotion-changed, all of which hold.

Instruction-surface and manifest items. The diff deletes no standing instruction, adds no manifest field, and touches no trust-gate path (trusted-actors, check-trusted-trigger, CODEOWNERS, .gitmodules). The register file was not needed. There is no manifest <field> line to report.

Tests. The new test pins the shared_refusal binding (branch, base_sha, identity, lane, authority, self_logins) and asserts that no in-flight read follows an unusable answer. The mocks in test_babysit_lane_room.py are test-only.

PR context. It contains no embedded directives aimed at the reviewer.

@github-actions github-actions Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

claude-review: 1 inline finding(s) on a553dfcdaa0a84e261f7cc1d5adc69f8a5237c9d; the summary is in this lane's pull request comment.

claude-lane: claude-review

Comment thread docs/conventions/pr-pipeline.yaml
@github-actions

github-actions Bot commented Oct 23, 2026 •

Copy link
Copy Markdown
Contributor

claude-review reviewed d3c80641d3f89c8cee674e13c606272e3b1747b8: 0 finding(s), 0 as inline comments.

Incremental review of the changes since 2948665: the changelog fragment, the pr-pipeline README, babysit_merge_policy.py and test_babysit_lane_room.py. I found nothing to flag.

What changed. lane_room no longer applies its own copy of the age and demotion checks. It now calls the gate's shared_refusal, so it also catches a lane merge that landed after the report was computed (new-merge:#<n>).

What I checked by reading the code. I ran no tests or commands.

  • Call arguments. lane_room passes identity, lane=True, authority=PR_MERGE, self_logins=frozenset(), branch=default and base_sha=config.base_sha. The post-merge report is written by .github/workflows/pr-merge.yml:195 with --lane-merges-only and no --self-login. Its selfLogins is therefore empty and laneMergesOnly is true, so the binding checks in shared_refusal (identity, self logins, lane) pass for a report from the same run. The call cannot reject every report.
  • Dropped checks. The identity, age and demotion checks removed from lane_room are all covered by shared_refusal, and the return order is the same. A missing report, an unusable one and a refusal still produce unreadable or refusal as before.
  • New hold. shared_refusal can now also return floor-differs, config-unreadable, merged-history-truncated, merged-unreadable and new-merge:#<n>. All of them make the room unreadable, which holds and disarms. That matches the PR's stated "unknown holds" rule.
  • Tests. The existing room() helper patches newest_config_commit and _get so that only age and demotion decide the outcome. test_a_lane_merge_the_answer_does_not_count_is_unreadable covers the new new-merge path. It also pins the arguments the gate binds a candidate with, and checks that no in-flight read follows an unusable answer.
  • Docs. The README and changelog edits match the code change.

Docs wording, not reported. The changelog and README describe a report the gate would not reuse as "older than 30 minutes". shared_refusal also rejects a passing report that still has an unverified lane merge once it is older than SHARED_UNVERIFIED_MAX_AGE, which is 2 minutes (babysit_run_cache.MAX_AGE). The old code had the same behaviour, so this change did not introduce it. I'm noting it only because the reworded line still omits that bound.

Manifest fields. None: the diff touches no plugin.json or marketplace.json.

Closing issue. The PR states "No related issue", so there are no acceptance criteria to check.

Hold. None. The files changed here are not in the cost or trust-gate classes.

kyle-sexton and others added 2 commits October 23, 2026 19:33
…d the kill switch

The native path read only the review lanes' part of the gate's risk
score, so a lane pull request adding a symlink or a submodule pointer, or
touching a high path, a manifest, the trust gate or a cost path, armed
where the gate holds it. evidence --lane-reviews now runs babysit_risk.score
and holding over the inputs the gate builds, at the lane room's effective
rung, and fails closed on any read error. ci-status's lanes App step now
fails while the lanes kill switch is on, so an armed lane pull request
stops in the merge queue; other authors are unaffected.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

@github-actions github-actions Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

claude-security-review: 1 inline finding(s) on 39d1971dedd743c90c06790be6bf9ef54a847ebb; the summary is in this lane's pull request comment.

claude-lane: claude-security-review

…records

The lane's room re-checks the demotion records through the job's
GITHUB_TOKEN; without issues: read that read fails, the room reads as
unreadable, and no lane pull request is ever armed.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
lane_room checked the report's age and demotion changes but not whether a
lane merge had landed since it was computed, so a merge whose full run
had already failed did not stop native arming. It now applies the gate's
own reuse rule, shared_refusal, as a candidate at the default branch
head, and any refusal (new-merge:#<n> included) makes the room
unreadable, which disarms and arms nothing.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@kyle-sexton
kyle-sexton added this pull request to the merge queue Oct 24, 2026
Merged via the queue into main with commit 358c028 Oct 24, 2026
43 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant