classify_inline_comments: buffered unconfirmed calls are replayed after the model re-posts with confirmed=true, duplicating every inline comment

[martintreurnicht]

Summary

When classify_inline_comments is enabled (the default), the inline-comment MCP server buffers create_inline_comment calls that lack confirmed=true and replies to the model with "Comment buffered. … Set confirmed=true to post immediately." In practice the model frequently reacts to that message by re-calling the tool with confirmed=true, which posts the comment live — but the original unconfirmed call stays in the buffer, and the post-buffered-inline-comments step re-posts it after the session. Every inline comment lands twice, ~15–25s apart.

For Vertex/Bedrock users this is unconditional: classifyComments() requires ANTHROPIC_API_KEY, which is unset under OIDC/Vertex auth, so the step logs ANTHROPIC_API_KEY not set — skipping classification, posting all unconfirmed comments and replays the whole buffer. But note that an API key would not prevent these duplicates either: the buffered entries are genuine review comments, so the test/probe classifier correctly marks them "real" and posts them anyway. The classifier filters probes, not duplicates of comments that were already posted via the confirmed=true path.

Environment

• anthropics/claude-code-action@v1, observed at eee73e2 (v1.0.143); buffering logic unchanged at current v1 head
• use_vertex: true (no ANTHROPIC_API_KEY in env), model claude-opus-4-8
• Automated PR review prompt; --allowedTools "mcp__github_inline_comment__create_inline_comment,..."

Observed timeline (from one affected run)

time	event
23:27:03	inline comment A posted live (model re-called with confirmed=true after the "buffered" reply)
23:27:05	inline comment B posted live (same pattern)
23:27:19	Claude session ends ("subtype":"success")
23:27:19	post-step: Found 2 buffered inline comment(s) → ANTHROPIC_API_KEY not set — skipping classification, posting all unconfirmed comments → Posting 2 classified-as-real comment(s)
23:27:20–21	comments A and B posted again → duplicates

We saw this on 5 PRs in a private org repo between June 8–10. Onset correlates with v1.0.141 (the Claude Code 2.1.169 bump) — the buffering feature itself (#1048) shipped earlier, but after that release the model started consistently following the buffered-reply's "Set confirmed=true to post immediately" suggestion with a re-call. It also cascades: the next review run sees two identical unresolved threads and replies to/resolves each separately.

Root cause

src/mcp/github-inline-comment-server.ts buffers when CLASSIFY_ENABLED && confirmed !== true, and nothing reconciles the buffer when the same comment is subsequently posted via a confirmed=true call (or via gh api in the same session). src/entrypoints/post-buffered-inline-comments.ts then posts every buffered entry with confirmed !== false — there is no dedup against comments already present on the PR.

Suggested fixes (any of these would do)

1. Reconcile on confirmed post: when a confirmed=true call posts, drop buffered entries matching the same path+line (+/− body similarity) so they can't be replayed.
2. Dedup in the post-step: before posting, fetch the PR's existing review comments and skip buffered entries whose path+line+body already exist (also makes the step idempotent across retries).
3. Don't invite the re-call: the buffered-reply text "Set confirmed=true to post immediately" is what triggers the double-call. Saying only "buffered; it will be posted after the session — do not re-submit" would remove the trigger for review-style prompts.
4. Fail closed (or default off) without an API key: when ANTHROPIC_API_KEY is absent (Bedrock/Vertex/Foundry), buffering provides no classification value — the current behavior is buffer + unconditional replay. Skipping buffering entirely in that configuration would restore pre-feat(inline-comment): add confirmed param + probe-pattern safety net #1048 semantics for those users.

Workaround

classify_inline_comments: false on the action invocation — restores immediate posting and skips the replay step. Working for us, but the default-on path duplicates comments for any workflow whose model responds to the buffered message with a confirmed re-call.