fix(agent): retry the selected model before falling back

Retry on fail used to wrap the whole fallback chain, so tries 3 with
fallbacks B and C ran A, B, C three times over. A builder who lists
fallbacks wants the selected model retried and the fallbacks tried once
each after its last try fails, which is also how LiteLLM orders retries
and fallbacks and how OpenRouter treats each model in its list.

The executor keeps the retry policy. Each try is now told where it sits
in it through the node metadata (`BlockNodeMetadata.retry`, with the
executor's own `isFinalTry` judgment), and the Agent handler keeps the
fallbacks out of the candidate list until the final try. Every earlier
try runs the primary alone and lets the failure escape for the policy to
replay. Blocks without fallbacks, and blocks with retry off, behave as
before; other handlers ignore the field.

Author: mzxchandra

apps/docs/content/docs/workflows/blocks/agent.mdx

@@ -87,8 +87,8 @@ Some settings live under advanced, or appear only for models that support them:
 - **Reasoning effort / Thinking level.** For models with extended reasoning, how much the model thinks before answering. Higher is more thorough but slower and costs more tokens.
 - **Prompt caching.** For Anthropic Claude models, reuses the system prompt and tool definitions between runs instead of re-reading them every time. Cached input costs a tenth of the normal rate, but writing the cache costs 1.25x, so leave it off for one-off runs and turn it on when the same agent runs repeatedly. The cache covers a prefix only if it reaches 1,024 tokens (2,048 on Haiku) — below that Anthropic ignores it and nothing changes. Entries expire after five minutes of no use.
 - **API key.** Your key for the chosen provider. Hidden on hosted Sim, which supplies one.
-- **Fallback models.** An ordered list of models to try when the request to the selected model fails, whether the provider is overloaded, rate-limited, or down. Sim tries the 2nd choice, then the 3rd, and so on, and `<agent.model>` reports the model that answered. Hosted models need no setup. A model that needs its own key takes it from a workspace environment variable you pick on the row; a model on the same provider as the selected model reuses the block's key. Azure, Bedrock, and Vertex models can only be fallbacks for a selected model of the same family, since they use that model's credentials. The Auto model cannot be a fallback. A fallback runs with the selected model's settings where its provider accepts them: temperature and max output tokens are clamped to the fallback's limits, and when the fallback has a reasoning effort, thinking level, or verbosity setting that the selected model's value does not fit, the row shows that field so you can pick a value for it, otherwise the provider's default applies.
-- **Retry on fail.** Runs the block again after a failure, up to a maximum number of tries with a wait between them. Fallback models work inside each try: one try walks the selected model and then every fallback, and only when all of them fail does the next try begin. A failure that happens after the model already called a tool runs that conversation again on the next model, so keep fallbacks and retry off for agents whose tools must not repeat.
+- **Fallback models.** An ordered list of models to try when the request to the selected model fails, whether the provider is overloaded, rate-limited, or down. Sim tries the 2nd choice, then the 3rd, and so on, once each, and `<agent.model>` reports the model that answered. Hosted models need no setup. A model that needs its own key takes it from a workspace environment variable you pick on the row; a model on the same provider as the selected model reuses the block's key. Azure, Bedrock, and Vertex models can only be fallbacks for a selected model of the same family, since they use that model's credentials. The Auto model cannot be a fallback. A fallback runs with the selected model's settings where its provider accepts them: temperature and max output tokens are clamped to the fallback's limits, and when the fallback has a reasoning effort, thinking level, or verbosity setting that the selected model's value does not fit, the row shows that field so you can pick a value for it, otherwise the provider's default applies.
+- **Retry on fail.** Retries the selected model after a failure, up to a maximum number of tries with a wait between them. When its tries run out, the fallback models are tried in order, once each, with no wait before the first of them. A fallback is never retried. A failure that happens after the model already called a tool runs that conversation again on the next try or the next model, so keep fallbacks and retry off for agents whose tools must not repeat.
 
 OpenAI and Gemini cache automatically at no extra cost and need no setting; their discount is already reflected in what you are charged.
 
@@ -150,5 +150,5 @@ The Agent reads the message from Start with `<start.input>` and returns a result
 <FAQ items={[
   { question: "How does max output tokens work with Anthropic models?", answer: "The Agent block uses each Anthropic model's full max output token limit by default (for example, 64,000 tokens). You can override this with the Max Output Tokens setting. For non-streaming requests that exceed the SDK's internal threshold, the provider automatically uses internal streaming to avoid timeouts." },
   { question: "Can I use the Agent block with a custom or self-hosted model?", answer: "Yes. Use any Ollama or VLLM-compatible model by typing the model name directly into the model combobox, as long as it exposes a compatible API endpoint." },
-  { question: "What happens when my model's provider is down?", answer: "Add fallback models under Additional fields. When the request to the selected model fails, Sim tries the 2nd choice, then the 3rd, in order, and the block succeeds if any of them answers. The log detail shows the model that answered and the models it fell back from. Turn on Retry on fail as well to repeat the whole sequence after a wait." },
+  { question: "What happens when my model's provider is down?", answer: "Add fallback models under Additional fields. When the request to the selected model fails, Sim tries the 2nd choice, then the 3rd, in order, and the block succeeds if any of them answers. The log detail shows the model that answered and the models it fell back from. Turn on Retry on fail as well to give the selected model a few tries first; the fallbacks are tried once each after its last try fails." },
 ]} />

apps/sim/blocks/blocks/agent.ts

@@ -37,7 +37,7 @@ const logger = createLogger('AgentBlock')
 /** Model the agent block falls back to when `model` is unset or the auto pseudo-model. */
 const AGENT_FALLBACK_MODEL = 'claude-sonnet-5'
 
-const FALLBACK_MODELS_DESCRIPTION = `Ordered models tried in sequence when the request to the selected model fails. Each row is { model, apiKey?, reasoningEffort?, thinkingLevel?, verbosity? }; apiKey, when present, must be a whole {{ENV_VAR}} reference, and a tuning value must be one the row model declares. sim-auto is not allowed. Max ${MAX_FALLBACK_MODELS}.`
+const FALLBACK_MODELS_DESCRIPTION = `Ordered models tried in sequence, once each, when the request to the selected model fails; with Retry on fail, after the selected model's tries run out. Each row is { model, apiKey?, reasoningEffort?, thinkingLevel?, verbosity? }; apiKey, when present, must be a whole {{ENV_VAR}} reference, and a tuning value must be one the row model declares. sim-auto is not allowed. Max ${MAX_FALLBACK_MODELS}.`
 const MODELS_WITH_REASONING_EFFORT = getModelsWithReasoningEffort()
 const MODELS_WITH_VERBOSITY = getModelsWithVerbosity()
 const MODELS_WITH_THINKING = getModelsWithThinking()

apps/sim/executor/execution/block-executor.retry.test.ts

@@ -121,6 +121,36 @@ describe('BlockExecutor retry', () => {
     expect(execute).toHaveBeenCalledTimes(1)
   })
 
+  it('tells each try where it sits in the policy, and a block without one nothing', async () => {
+    const block = createBlock({ enabled: true, maxTries: 3, waitBetweenTriesMs: 0 })
+    const execute = vi
+      .fn()
+      .mockRejectedValueOnce(new Error('one'))
+      .mockRejectedValueOnce(new Error('two'))
+      .mockResolvedValueOnce({ ok: true })
+    const state = new ExecutionState()
+    const executor = buildExecutor(block, { canHandle: () => true, execute }, state)
+
+    await executor.execute(createContext(state), createNode(block), block)
+
+    expect(execute.mock.calls.map(([, , , metadata]) => metadata.retry)).toEqual([
+      { attempt: 1, maxTries: 3, isFinalTry: false },
+      { attempt: 2, maxTries: 3, isFinalTry: false },
+      { attempt: 3, maxTries: 3, isFinalTry: true },
+    ])
+    expect(execute.mock.calls[0][3].nodeId).toBe(block.id)
+
+    const plain = createBlock()
+    const executePlain = vi.fn().mockResolvedValue({ ok: true })
+    const plainState = new ExecutionState()
+    await buildExecutor(
+      plain,
+      { canHandle: () => true, execute: executePlain },
+      plainState
+    ).execute(createContext(plainState), createNode(plain), plain)
+    expect(executePlain.mock.calls[0][3]).not.toHaveProperty('retry')
+  })
+
   it('replays any failure and succeeds on a later try', async () => {
     const block = createBlock(enabled)
     const execute = vi

apps/sim/executor/execution/block-executor.ts

@@ -44,6 +44,7 @@ import {
 import {
   type BlockHandler,
   type BlockLog,
+  type BlockRetryAttempt,
   type BlockState,
   type ExecutionContext,
   getNextExecutionOrder,
@@ -277,11 +278,12 @@ export class BlockExecutor {
        * token is drained, so a replay cannot duplicate output the client has
        * already seen.
        */
-      const output = await this.runHandlerWithRetry(blockCtx, block, blockLog, () =>
-        handler.executeWithNode
-          ? handler.executeWithNode(blockCtx, block, resolvedInputs, nodeMetadata)
-          : handler.execute(blockCtx, block, resolvedInputs, nodeMetadata)
-      )
+      const output = await this.runHandlerWithRetry(blockCtx, block, blockLog, (retry) => {
+        const invocationMetadata = retry ? { ...nodeMetadata, retry } : nodeMetadata
+        return handler.executeWithNode
+          ? handler.executeWithNode(blockCtx, block, resolvedInputs, invocationMetadata)
+          : handler.execute(blockCtx, block, resolvedInputs, invocationMetadata)
+      })
 
       completedHandlerCost = readTrustedExecutionCost(output)
 
@@ -546,15 +548,20 @@ export class BlockExecutor {
    * Rethrows the final try's error so the caller's catch — and with it the error
    * port — behaves exactly as it does for a block that never retried. Retrying
    * only ever delays the existing outcome; it never changes it.
+   *
+   * Each try is told where it sits in the policy (`BlockRetryAttempt`). The
+   * policy stays here: a handler cannot ask for another try or skip the wait,
+   * it can only hold work for the try after which no other follows, the way the
+   * Agent block keeps its fallback models for the final try.
    */
   private async runHandlerWithRetry<T>(
     ctx: ExecutionContext,
     block: SerializedBlock,
     blockLog: BlockLog | undefined,
-    invoke: () => Promise<T>
+    invoke: (retry: BlockRetryAttempt | undefined) => Promise<T>
   ): Promise<T> {
     const policy = resolveBlockRetryPolicy(block)
-    if (!policy) return invoke()
+    if (!policy) return invoke(undefined)
 
     const shouldAccumulateFunctionCost = block.metadata?.id === BlockType.FUNCTION
     let accumulatedFunctionCost: TrustedExecutionCost | undefined
@@ -563,7 +570,11 @@ export class BlockExecutor {
       for (;;) {
         tries++
         try {
-          const output = await invoke()
+          const output = await invoke({
+            attempt: tries,
+            maxTries: policy.maxTries,
+            isFinalTry: tries >= policy.maxTries,
+          })
           if (!shouldAccumulateFunctionCost || !accumulatedFunctionCost || !isRecordLike(output)) {
             return output
           }

apps/sim/executor/handlers/agent/agent-handler.test.ts

@@ -616,6 +616,73 @@ describe('AgentBlockHandler', () => {
       expect(blockLog).toMatchObject({ modelFallbacks: ['gpt-4o'] })
     })
 
+    it('holds the fallbacks on a try the executor will replay', async () => {
+      mockExecuteProviderRequest.mockRejectedValueOnce(new Error('overloaded'))
+      const blockLog = openLog()
+      blockLog.modelFallbacks = ['stale-from-earlier-run']
+
+      await expect(
+        handler.execute(
+          { ...mockContext, blockLogs: [blockLog] },
+          mockBlock,
+          { ...baseInputs, fallbackModels: [{ model: 'claude-sonnet-5' }] },
+          { nodeId: mockBlock.id, retry: { attempt: 1, maxTries: 3, isFinalTry: false } }
+        )
+      ).rejects.toThrow('overloaded')
+
+      expect(mockExecuteProviderRequest).toHaveBeenCalledTimes(1)
+      expect(mockAgentLogger.info).toHaveBeenCalledWith('Fallback models held for the final try', {
+        blockId: mockBlock.id,
+        attempt: 1,
+        maxTries: 3,
+      })
+      expect(mockAgentLogger.warn).not.toHaveBeenCalledWith(
+        'Agent model failed; trying fallback',
+        expect.anything()
+      )
+      expect(blockLog.modelFallbacks).toBeUndefined()
+    })
+
+    it('walks the chain on the final try, and on a block that never retries', async () => {
+      mockExecuteProviderRequest
+        .mockRejectedValueOnce(new Error('overloaded'))
+        .mockResolvedValueOnce(providerResponse('claude-sonnet-5'))
+        .mockRejectedValueOnce(new Error('overloaded'))
+        .mockResolvedValueOnce(providerResponse('claude-sonnet-5'))
+      const inputs = { ...baseInputs, fallbackModels: [{ model: 'claude-sonnet-5' }] }
+
+      const onFinalTry = await handler.execute(mockContext, mockBlock, inputs, {
+        nodeId: mockBlock.id,
+        retry: { attempt: 3, maxTries: 3, isFinalTry: true },
+      })
+      const withoutPolicy = await handler.execute(mockContext, mockBlock, inputs, {
+        nodeId: mockBlock.id,
+      })
+
+      expect(mockExecuteProviderRequest).toHaveBeenCalledTimes(4)
+      expect((onFinalTry as { model: string }).model).toBe('claude-sonnet-5')
+      expect((withoutPolicy as { model: string }).model).toBe('claude-sonnet-5')
+      expect(mockAgentLogger.info).not.toHaveBeenCalledWith(
+        'Fallback models held for the final try',
+        expect.anything()
+      )
+    })
+
+    it('says nothing about held fallbacks when none are configured', async () => {
+      mockExecuteProviderRequest.mockRejectedValueOnce(new Error('overloaded'))
+
+      await expect(
+        handler.execute(mockContext, mockBlock, baseInputs, {
+          nodeId: mockBlock.id,
+          retry: { attempt: 1, maxTries: 2, isFinalTry: false },
+        })
+      ).rejects.toThrow('overloaded')
+      expect(mockAgentLogger.info).not.toHaveBeenCalledWith(
+        'Fallback models held for the final try',
+        expect.anything()
+      )
+    })
+
     it('never falls back on a deep-research follow-up turn', async () => {
       mockExecuteProviderRequest.mockRejectedValueOnce(new Error('overloaded'))
 

apps/sim/executor/handlers/agent/agent-handler.ts

@@ -75,7 +75,13 @@ import type {
   ToolInput,
 } from '@/executor/handlers/agent/types'
 import { parseResponseFormat } from '@/executor/handlers/shared/response-format'
-import type { BlockHandler, ExecutionContext, StreamingExecution, UserFile } from '@/executor/types'
+import type {
+  BlockHandler,
+  BlockNodeMetadata,
+  ExecutionContext,
+  StreamingExecution,
+  UserFile,
+} from '@/executor/types'
 import { collectBlockData } from '@/executor/utils/block-data'
 import { stringifyJSON } from '@/executor/utils/json'
 import { projectResolvedSecretDiagnosticContent } from '@/executor/utils/resolved-secret-content-projection'
@@ -283,7 +289,8 @@ export class AgentBlockHandler implements BlockHandler {
   async execute(
     ctx: ExecutionContext,
     block: SerializedBlock,
-    inputs: AgentInputs
+    inputs: AgentInputs,
+    nodeMetadata?: BlockNodeMetadata
   ): Promise<BlockOutput | StreamingExecution> {
     ctx.mcpBlockId = block.id
     const providerErrorRegistry = ctx.resolvedSecretTraceRegistry?.forkForInputPaths(
@@ -483,19 +490,33 @@ export class AgentBlockHandler implements BlockHandler {
       }
 
       /**
+       * Retry on fail retries the selected model; the fallbacks join only on the
+       * try after which the executor promises no other. Until then a failure of
+       * the primary is left to escape, so the executor's policy can replay it.
+       *
        * A follow-up turn of a deep-research interaction lives on the primary's
        * provider; another model has none of that conversation, so a green answer
        * from it would be built on a fresh context. Such a request never falls back.
        */
-      const fallbackCandidates = modelInputs.previousInteractionId
-        ? []
-        : normalizeFallbackModels(filteredInputs.fallbackModels).filter(
-            (candidate) => candidate.model.toLowerCase() !== model.toLowerCase()
-          )
-      if (modelInputs.previousInteractionId && filteredInputs.fallbackModels?.length) {
+      const configuredFallbacks = normalizeFallbackModels(filteredInputs.fallbackModels)
+      const retry = nodeMetadata?.retry
+      const fallbacksHeld = retry !== undefined && !retry.isFinalTry
+      const fallbackCandidates =
+        modelInputs.previousInteractionId || fallbacksHeld
+          ? []
+          : configuredFallbacks.filter(
+              (candidate) => candidate.model.toLowerCase() !== model.toLowerCase()
+            )
+      if (configuredFallbacks.length > 0 && modelInputs.previousInteractionId) {
         logger.info('Fallback models skipped for a deep-research follow-up turn', {
           blockId: block.id,
         })
+      } else if (configuredFallbacks.length > 0 && fallbacksHeld) {
+        logger.info('Fallback models held for the final try', {
+          blockId: block.id,
+          attempt: retry.attempt,
+          maxTries: retry.maxTries,
+        })
       }
       const candidates: ModelCandidate[] = [
         { model, apiKey: modelInputs.apiKey, isPrimary: true },
@@ -2397,8 +2418,9 @@ export class AgentBlockHandler implements BlockHandler {
    *
    * Which model serves the request is decided here, inside one handler
    * invocation. How many invocations the block gets is the executor's retry
-   * policy, which wraps this whole chain: with retry on, every try walks the
-   * chain again from the primary.
+   * policy, and the caller keeps the fallbacks out of the candidate list until
+   * the final try, so with retry on the block runs the primary alone on every
+   * earlier try and walks the whole chain once.
    *
    * Falling through is deliberately as indiscriminate as block retry
    * (`isRetryableBlockError`): a provider error carries no status, so an
@@ -2678,7 +2700,7 @@ export class AgentBlockHandler implements BlockHandler {
    * the executor pushes the entry before running the handler with `endedAt`
    * still empty, which is what tells it apart from earlier runs of the same
    * block in a loop or an earlier retry. An empty list clears the field, since
-   * a retry that succeeds on the primary reuses the entry a failed try wrote.
+   * every try reuses one entry and only the final try can write failed models.
    *
    * A model id can itself come from a resolved reference, so the names are
    * projected through the same secret registry as every other diagnostic and

apps/sim/executor/types.ts

@@ -756,6 +756,22 @@ export interface BlockNodeMetadata {
   originalBlockId?: string
   isLoopNode?: boolean
   executionOrder?: number
+  /** Where this invocation sits in the block's retry policy; absent when the block has none. */
+  retry?: BlockRetryAttempt
+}
+
+/**
+ * One try of a block under its retry policy, told to the handler so it can hold
+ * work for the last try. `isFinalTry` is the executor's own judgment, not
+ * `attempt >= maxTries` recomputed by the handler: what makes a try final is the
+ * policy's business, and a handler that fails on a non-final try is promised
+ * another invocation for any retryable error.
+ */
+export interface BlockRetryAttempt {
+  /** 1-based. */
+  attempt: number
+  maxTries: number
+  isFinalTry: boolean
 }
 
 export interface BlockHandler {