Skip to content

[Fix] Fast parent events repeatedly retry exhausted OpenRouter accounts - #2128

Draft
roomote-roomote[bot] wants to merge 1 commit into
developfrom
fix/fast-openrouter-credit-retry-storm-0rgj8a6nvljzg
Draft

[Fix] Fast parent events repeatedly retry exhausted OpenRouter accounts#2128
roomote-roomote[bot] wants to merge 1 commit into
developfrom
fix/fast-openrouter-credit-retry-storm-0rgj8a6nvljzg

Conversation

@roomote-roomote

Copy link
Copy Markdown
Contributor

​Opened on behalf of @mrubens. Follow up by mentioning @roomote-roomote, in the web UI, or in Slack.

What changed

  • Recognize OpenRouter credit exhaustion only from the observed structured 402 metadata (limit_source=openrouter_credits) and stop both OpenCode and Fast inference retries immediately.
  • Keep arbitrary 402 responses retryable when they lack that evidence, while preserving the existing terminal handling for errors providers explicitly mark non-retryable.
  • Discard durable Fast parent events after typed non-retryable inference failures so PostgreSQL recovery does not recreate exhausted BullMQ jobs indefinitely.
  • Preserve transient provider retries and the separate missing-session fallback rebuild path.
  • Add regression coverage across provider recovery, Fast inference, durable parent-event delivery, and cold-session recovery.

Why this change was made

Fast parent platform events could exhaust their bounded inference and BullMQ retries after an OpenRouter account-credit failure, then be recreated every minute because the durable PostgreSQL event remained pending. That turned a terminal account condition into an unbounded production retry storm.

Impact

Explicit OpenRouter credit-limit failures now settle once at the inference and durable queue boundaries. Temporary provider outages, rate limits, unknown 402 responses, and missing OpenCode sessions retain their recovery behavior.

@roomote-community

roomote-community Bot commented Sep 3, 2026

Copy link
Copy Markdown
Contributor

1 issue outstanding. See task

  • Durable parent events still retry non-retryable inference failures because their FastAgentInferenceError is wrapped before the new predicate runs. (packages/sdk/src/server/lib/fast-agent-parent-event-queue.ts:449)

Reviewed 2491ca1

await markDiscarded(row.id, deliveryError);
continue;
}
if (isNonRetryableFastAgentInferenceError(error)) {

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

deliverFastAgentParentEventWithLock wraps failures from answerFastAgentQuestion in FastAgentParentEventDeliveryError before they reach this catch block (fast-agent-parent-event.ts:1862). The new predicate only accepts a direct FastAgentInferenceError, so it returns false for this wrapper and the row is left pending and retried by recovery. Inspect the error cause (or unwrap it here) so an OpenRouter credit failure is actually discarded.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant