Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
15 changes: 14 additions & 1 deletion .github/scripts/pr_intake_gate.js
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,8 @@
// re-evaluates — and reopens — the PR when the description is edited or the
// author is assigned to the issue. A triage+ user reopening the PR, removing
// the label, or adding `bypass-issue-check` overrides it, and the override
// sticks.
// sticks. A PR that comes back this way also gets a comment asking the review
// bot for a review, since it doesn't act on `reopened` by itself.
//
// Everything that writes goes through mutate(); when the workflow passes
// ENFORCE=false (its kill switch) the run only logs what it would have done.
Expand All @@ -20,6 +21,10 @@ const OPEN_LABEL = 'help wanted'; // issue label that waives assignment
const MARKER = '<!-- require-linked-issue -->';
const BOT_LOGIN = 'github-actions[bot]';
const MAX_ISSUES = 5;
// cubic starts on `opened` and abandons the run when the gate closes the PR
// seconds later; it ignores `reopened`, so a PR the gate lets back in would
// otherwise wait for its next push to be reviewed.
const REVIEW_REQUEST = '@cubic-dev-ai review this PR';

module.exports = async function run({ github, context, core }) {
const { owner, repo } = context.repo;
Expand Down Expand Up @@ -115,6 +120,8 @@ module.exports = async function run({ github, context, core }) {
if (gated) {
await removeLabel(prNumber, LABEL);
await deleteGateComment(prNumber);
// Not for drafts: cubic picks those up itself on ready_for_review.
if (!pr.draft) await requestReview(prNumber);
Comment on lines 120 to +124

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 (optional) The review request is the only write in pass() with no idempotency, so two runs that let the same PR back in concurrently each post @ cubic-dev-ai review this PR, giving the PR two bot comments (and potentially two cubic reviews) where the base only did a harmless duplicate reopen/unlabel. Fix: post the request only when this run actually removed missing-issue-link (have removeLabel report whether the API returned 404, and skip requestReview when it did), or check for an existing request comment before creating one. Trigger: issues.assigned runs are grouped per issue+assignee and PR-event runs per PR, so assigning the author to two linked issues, or an assignment landing while the author edits the description, runs evaluate() for the same PR in parallel.

Extended reasoning...

Workflow concurrency groups (require-linked-issue.yml concurrency.group) are require-linked-issue-<pr> for pull_request_target events but issue-<issue>-<assignee> for issues events, so runs touching the same PR are not serialized across these paths, and two assignments on different issues get two groups. Both runs execute evaluate(): pulls.get still returns the PR with missing-issue-link (line 74-78, gated=true), both reach pass(). Run A reopens, removes the label, deletes the gate comment, posts REVIEW_REQUEST (line 124). Run B: reopen of an already-open PR succeeds, removeLabel gets 404 which line 270 swallows, deleteGateComment finds nothing or swallows 404 (line 321) — the existing code explicitly tolerates concurrent runs here — but then if (!pr.draft) await requestReview(prNumber) runs unconditionally using the stale pr snapshot and posts a second identical comment. Nothing in requestReview (line 310-312) looks for an existing request. The comment at 308-309 claims 'runs once per return' but that only holds when runs are serialized. Consequence after merge: duplicate…

Verification: nit — triggered when two gate runs that both let the same PR back in overlap, which the workflow's concurrency groups do not prevent across event kinds. Mechanism verified: /home/claude/python-sdk/.github/workflows/require-linked-issue.yml:65 puts pull_request_target/dispatch runs in require-linked-issue-<pr> but issues.assigned runs in issue-<issue>-<assignee>, so an assignment run and…

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2: Make the review request idempotent across concurrent gate runs. Two runs can both reach this unconditional call before either observes the removed label, creating duplicate cubic comments and potentially duplicate reviews; only request it when this run removes the label or when no request comment exists.

Prompt for AI agents
Check if this issue is valid — if so, understand the root cause and fix it. At .github/scripts/pr_intake_gate.js, line 124:

<comment>Make the review request idempotent across concurrent gate runs. Two runs can both reach this unconditional call before either observes the removed label, creating duplicate cubic comments and potentially duplicate reviews; only request it when this run removes the label or when no request comment exists.</comment>

<file context>
@@ -115,6 +120,8 @@ module.exports = async function run({ github, context, core }) {
         await removeLabel(prNumber, LABEL);
         await deleteGateComment(prNumber);
+        // Not for drafts: cubic picks those up itself on ready_for_review.
+        if (!pr.draft) await requestReview(prNumber);
       }
     }
</file context>

}
}

Expand Down Expand Up @@ -298,6 +305,12 @@ module.exports = async function run({ github, context, core }) {
}
}

// Runs once per return: the caller only gets here while the PR still counts
// as gate-closed, and it has just removed the label that says so.
async function requestReview(prNumber) {
await mutate(`request a review on PR #${prNumber}`, () => github.rest.issues.createComment({ owner, repo, issue_number: prNumber, body: REVIEW_REQUEST }));

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 (optional) When createComment in requestReview fails transiently (5xx, secondary rate limit on content creation) after reopen, removeLabel and deleteGateComment have already succeeded, the workflow run goes red on a PR that is now correctly open and clean; on the base branch this path ended green. Re-running the failed job cannot recover: gated (line 78) is now false because the label is gone, so the re-run passes silently and the review request is lost for good. Fix: isolate the review request from the gate's own verdict — catch and core.warning() the createComment error (as reopen() does for 422), or perform it before removeLabel so a re-run still sees the PR as gated and retries it. [also at: .github/scripts/pr_intake_gate.js:124 - When issues.createComment at :311 fails after removeLabel :121 and deleteGateComment :122 have succeeded, the review…]

Extended reasoning...

Path: pass() at .github/scripts/pr_intake_gate.js:116-126. Order of writes: addLabel(bypass) [sticky], reopen (:119), removeLabel (:121), deleteGateComment (:122), requestReview (:124 -> :311). requestReview has no try/catch; any non-2xx from issues.createComment propagates out of evaluate(). For pull_request_target events (:67) the exception fails the job and the PR shows a red 'Require Linked Issue / Evaluate' check on a PR the gate itself just approved. For the issues path (:49-56) it is collected and rethrown at :56, red on the issue-assignment run. Consequence that the dismissing finders did not follow: a re-run of the failed job recomputes gated = action === 'unlabeled' || labels.includes(LABEL) at :78; the label was removed in the first attempt, so gated is false, the gated block is skipped, and the review request is never posted (only the 'unlabeled' action still counts as gated). Neither the red run nor the re-run leaves any hint that a review request was lost. Population: every returning PR; rate: proportional to GitHub's…

Verification: nit — triggered when issues.createComment in requestReview fails transiently (5xx, 403/429 secondary rate limit on content creation) after the preceding writes in pass() succeeded. Mechanism verified in /home/claude/python-sdk/.github/scripts/pr_intake_gate.js: pass() (lines 116-126) runs reopen (119), removeLabel(prNumber, LABEL) (121), deleteGateComment (122), then `if…

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2: Post the review request before clearing missing-issue-link, or otherwise preserve a retryable gate state when createComment fails. As written, a transient comment error fails the run after the PR is reopened and cleaned, and a rerun can skip this path and permanently lose the review request.

Prompt for AI agents
Check if this issue is valid — if so, understand the root cause and fix it. At .github/scripts/pr_intake_gate.js, line 311:

<comment>Post the review request before clearing `missing-issue-link`, or otherwise preserve a retryable gate state when `createComment` fails. As written, a transient comment error fails the run after the PR is reopened and cleaned, and a rerun can skip this path and permanently lose the review request.</comment>

<file context>
@@ -298,6 +305,12 @@ module.exports = async function run({ github, context, core }) {
+  // Runs once per return: the caller only gets here while the PR still counts
+  // as gate-closed, and it has just removed the label that says so.
+  async function requestReview(prNumber) {
+    await mutate(`request a review on PR #${prNumber}`, () => github.rest.issues.createComment({ owner, repo, issue_number: prNumber, body: REVIEW_REQUEST }));
+  }
+
</file context>

}

async function deleteGateComment(prNumber) {
const existing = await findGateComment(prNumber);
if (!existing) return;
Expand Down
30 changes: 22 additions & 8 deletions .github/scripts/pr_intake_gate.test.js
Original file line number Diff line number Diff line change
Expand Up @@ -14,6 +14,7 @@ const gate = require('./pr_intake_gate.js');

const LABEL = 'missing-issue-link';
const BYPASS = 'bypass-issue-check';
const REVIEW_REQUEST = '@cubic-dev-ai review this PR';
const REPO = { owner: 'modelcontextprotocol', repo: 'python-sdk' };

// People. Only the capability flags matter to the gate.
Expand All @@ -29,7 +30,8 @@ const PEOPLE = {
// `prs` / `issues` describe the world before the event; `expect` describes each
// PR afterwards: state, labels, and comment ('closed' = the "this PR has been
// closed" comment, 'closed-draft' = its draft wording, 'cannot-reopen' = the
// refused-reopen comment, null = none).
// refused-reopen comment, null = none). `reviewRequested`, where given, is
// whether the gate left its comment asking the review bot for a review.
// `writes: 0` additionally asserts the gate touched nothing at all.

const scenarios = [
Expand Down Expand Up @@ -78,7 +80,7 @@ const scenarios = [
prs: [pr(3300, 'outsider', { state: 'closed', labels: [LABEL], body: 'Fixes #10', gateComment: true })],
issues: [issue(10, { labels: ['help wanted'] })],
event: edited(3300, 'outsider'),
expect: { 3300: { state: 'open', labels: [], comment: null } },
expect: { 3300: { state: 'open', labels: [], comment: null, reviewRequested: true } },
},
{
name: 'gate-closed PR: maintainer assigns the author on the linked issue → reopened',
Expand All @@ -89,21 +91,21 @@ const scenarios = [
issues: [issue(10, { assignees: ['outsider'] }), issue(99)],
event: assigned(10, 'outsider', 'maintainer'),
expect: {
3300: { state: 'open', labels: [], comment: null },
3301: { state: 'closed', labels: [LABEL], comment: 'closed' },
3300: { state: 'open', labels: [], comment: null, reviewRequested: true },
3301: { state: 'closed', labels: [LABEL], comment: 'closed', reviewRequested: false },
},
},
{
name: 'gate-closed PR: maintainer reopens it → stays open with the sticky bypass label',
prs: [pr(3300, 'outsider', { state: 'open', labels: [LABEL], gateComment: true })], // payload arrives post-reopen
event: reopened(3300, 'maintainer'),
expect: { 3300: { state: 'open', labels: [BYPASS], comment: null } },
expect: { 3300: { state: 'open', labels: [BYPASS], comment: null, reviewRequested: true } },
},
{
name: 'gate-closed PR: triage-role user removes the label → reopened with the sticky bypass label',
prs: [pr(3300, 'outsider', { state: 'closed', labels: [], gateComment: true })], // payload arrives post-unlabel
event: unlabeled(3300, 'triager'),
expect: { 3300: { state: 'open', labels: [BYPASS], comment: null } },
expect: { 3300: { state: 'open', labels: [BYPASS], comment: null, reviewRequested: true } },
},
{
name: 'gate-closed PR: some other bot strips the label → re-checked, label restored, still closed',
Expand Down Expand Up @@ -177,13 +179,20 @@ const scenarios = [
prs: [pr(3300, 'outsider', { state: 'closed', labels: [LABEL], body: 'Fixes #10', gateComment: true, refuseReopen: true })],
issues: [issue(10, { assignees: ['outsider'] })],
event: edited(3300, 'outsider'),
expect: { 3300: { state: 'closed', labels: [LABEL], comment: 'cannot-reopen' } },
expect: { 3300: { state: 'closed', labels: [LABEL], comment: 'cannot-reopen', reviewRequested: false } },
},
{
name: 'gate-closed PR: maintainer adds the bypass label → reopened, and the label sticks',
prs: [pr(3300, 'outsider', { state: 'closed', labels: [LABEL, BYPASS], gateComment: true })], // payload arrives post-label
event: labeled(3300, 'maintainer', BYPASS),
expect: { 3300: { state: 'open', labels: [BYPASS], comment: null } },
expect: { 3300: { state: 'open', labels: [BYPASS], comment: null, reviewRequested: true } },
},
{
name: 'gate-closed draft comes back → reopened, but no review is requested while it is a draft',
prs: [pr(3300, 'outsider', { state: 'closed', draft: true, labels: [LABEL], body: 'Fixes #10', gateComment: true })],
issues: [issue(10, { assignees: ['outsider'] })],
event: assigned(10, 'outsider', 'maintainer'),
expect: { 3300: { state: 'open', labels: [], comment: null, reviewRequested: false } },
},
{
name: 'refused reopen after a label-removal override → both labels on, so the PR stays gate-managed',
Expand Down Expand Up @@ -303,6 +312,11 @@ function observe(world, expect) {
const kind = !body ? null : body.includes("won't let it be reopened") ? 'cannot-reopen' : body.includes('still a draft') ? 'closed-draft' : 'closed';
out[num] = { state: p.state, labels: [...p.labels].sort(), comment: kind };
if ('foreignComments' in expect[num]) out[num].foreignComments = p.comments.length - gateComments.length;
if ('reviewRequested' in expect[num]) {
const requests = p.comments.filter((c) => c.user === 'github-actions[bot]' && c.body === REVIEW_REQUEST);
assert.ok(requests.length <= 1, `PR #${num} has ${requests.length} review requests`);
out[num].reviewRequested = requests.length === 1;
}
}
return out;
}
Expand Down
3 changes: 2 additions & 1 deletion .github/workflows/require-linked-issue.yml
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,8 @@
# otherwise it is labeled `missing-issue-link`, gets one comment, and is closed,
# and it reopens automatically once the author is assigned. Drafts are gated
# too; bots are skipped. A triage+ user reopening the PR, removing the label,
# or adding `bypass-issue-check` overrides.
# or adding `bypass-issue-check` overrides. A PR that comes back open gets a

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P3: When a returning PR is still a draft, pass() removes the gate label but skips requestReview(); this sentence promises a cubic comment for every returning PR. Qualify it as non-draft to keep the workflow documentation accurate.

Prompt for AI agents
Check if this issue is valid — if so, understand the root cause and fix it. At .github/workflows/require-linked-issue.yml, line 9:

<comment>When a returning PR is still a draft, `pass()` removes the gate label but skips `requestReview()`; this sentence promises a cubic comment for every returning PR. Qualify it as non-draft to keep the workflow documentation accurate.</comment>

<file context>
@@ -6,7 +6,8 @@
 # and it reopens automatically once the author is assigned. Drafts are gated
 # too; bots are skipped. A triage+ user reopening the PR, removing the label,
-# or adding `bypass-issue-check` overrides.
+# or adding `bypass-issue-check` overrides. A PR that comes back open gets a
+# comment asking cubic to review it, because cubic doesn't act on `reopened`.
 #
</file context>

# comment asking cubic to review it, because cubic doesn't act on `reopened`.
#
# Operating it:
# - Live by default. To pause it without a revert, set the repository
Expand Down
Loading