feat(sdk): retry control-plane requests on 502/503 and connection failures

[devin-ai-integration]

Summary

The control-plane retry layer (withRetry in JS, RetryableTransport/AsyncRetryableTransport in Python) so far only retried 429 responses that carry Retry-After. The API answers capacity/busy conditions with 503 (placement_errors.go, pause/fork/snapshot "node is busy, please retry") and gateway/backend failures with 502, none of which send Retry-After, so the SDKs surfaced them immediately. Follow-up to the policy in EN-2619, which called for 502/503 with jittered backoff.

The non-replayable operation list is kept client-side (inline in retry.ts / retry.py) for now. The spec-extension alternative (x-idempotent on every POST + generated SDK lists) was prototyped in e2b-dev/belt#3537 and closed; it can be revisited if/when the API grows an idempotency key.

Retry decision per response (both SDKs):

status ∉ {429, 502, 503}        → return response
Retry-After (delta-seconds)      → wait that long          (all three statuses)
429 without Retry-After          → return response         (unchanged)
502/503 without Retry-After      → min(100ms · 2^attempt, 10s) · jitter[0.5, 1.0)

Everything else is unchanged: same retries option (default 3 → 4 attempts total, 0 disables), retries stop when the wait would pass the request-timeout deadline (60 s bound when timeouts are disabled), intermediate responses are closed, streaming bodies still get a single attempt, envd/volume-content requests are not retried. Exhausted retries return the final response, so ServiceBusyError (503) and the statusCode-carrying SandboxError (502) are raised as before.

504 is deliberately not retried: a backend timeout may have already applied the request (e.g. created a sandbox), whereas 502/503 are rejected before/without processing.

Network errors

Two classes, because only one of them is safe for every operation:

connect-phase failure (request never left)   → retry, any operation
post-write / opaque failure                  → retry only if isReplayable(request)
abort                                        → rethrow

isReplayable(request) is false for the operations that mint a resource without a client-supplied idempotency key (POST /sandboxes, /v2/sandboxes, /sandboxes/{id}/fork, /sandboxes/{id}/snapshots, /api-keys, /admin/teams/{id}/api-keys, /volumes, /secrets, /events/webhooks), so a replay of a request the server may already have processed cannot create a duplicate. POST /v3/templates is replayable: the request is keyed by template name, so a replay resolves to the same template and supersedes the not-yet-started build (verified upstream with a replay test). Every other operation is idempotent or a replay fails with a 404/409 the SDK already tolerates (pause → 409, kill → 404). DELETEs are replayable: a second delete is a no-op server-side; the 404 it returns is the existing kill() → false contract, left as is.

The list lives as NON_REPLAYABLE_OPERATIONS ([method, anchored RegExp]) in packages/js-sdk/src/retry.ts and packages/python-sdk/e2b/retry.py; a new resource-creating POST has to be added to both.

JS: Python already retries connection-establishment failures via ConnectionRetryTransport (pyqwest, E2B_CONNECTION_RETRIES, default 3). JS had nothing, so withRetry now retries when fetch throws — same retries, backoff and deadline as 502/503:

isRetryableFetchError(err, replayable) =
  !(err instanceof DOMException) && (isConnectionError(err) || (replayable && err instanceof Error))

isConnectionError walks cause chains / AggregateError members and matches only errors raised before request bytes are written: Node/undici ECONNREFUSED, ENOTFOUND, EAI_AGAIN, EHOSTUNREACH, ENETUNREACH, ENETDOWN, EHOSTDOWN, UND_ERR_CONNECT_TIMEOUT, any error with syscall connect/getaddrinfo, Bun's ConnectionRefused, Deno's client error (Connect). Post-write failures (ECONNRESET, terminated, Network connection lost.) and the opaque TypeError: Failed to fetch browsers and Workers raise for every network failure are indistinguishable from "request already reached the server" and go through the isReplayable gate.

Python: RetryableTransport/AsyncRetryableTransport additionally retry httpx.ReadError/WriteError/RemoteProtocolError (raised once the request was at least partially written) for replayable operations; ConnectError/ConnectTimeout stay with ConnectionRetryTransport underneath so connect failures aren't double-retried.

Other changes:

• JS: withRateLimitRetry → withRetry; the discarded body's cancel() is no longer awaited — msw's interceptor never settles it, which hung the new client-level test. Against real servers cancel() resolves immediately either way.
• Python: random_ injectable on both transports (keyword-only, like sleep/monotonic) for deterministic tests; the retry-timeout message no longer says "rate-limited".
• Docs for retries in ConnectionOpts / ApiParams, changesets for e2b and @e2b/python-sdk.

Usage

Nothing to opt into — a Sandbox.create() hitting a transient 503 not enough capacity (or an ECONNREFUSED/ENOTFOUND on the way to the API) now succeeds on a later attempt; a sandbox.kill() whose connection drops mid-flight is replayed, a Sandbox.create() in the same situation is not:

const sbx = await Sandbox.create({ retries: 5 })   // up to 5 retries on 429/502/503/connect failure
const sbx = await Sandbox.create({ retries: 0 })   // previous fail-fast behaviour

sbx = Sandbox.create(retries=5)
sbx = Sandbox.create(retries=0)

Tests

• packages/js-sdk/tests/retry.test.ts: backoff sequence with injected random, cap at 10 s, Retry-After preferred for 503, exhaustion, deadline check, 400/404/500/504 untouched; isConnectionError / isRetryableFetchError against fetch error shapes observed in Node, Bun, Deno, Cloudflare Workers and browsers; isReplayable for every listed operation plus replayable neighbours (GET …/fork, …/fork/extra, extra path segment); connect failures retried for POST /sandboxes, post-write errors retried for DELETE /sandboxes/{id} but rethrown for POST /sandboxes; aborts rethrown; retries: 0 passthrough. tests/client.test.ts: E2B client retries POST /sandboxes after a 503.
• packages/python-sdk/tests/test_rate_limit_retry_transport.py: sync + async equivalents (502/503, read/write/protocol errors gated by is_replayable, ConnectError passed through, list coverage).

Run: pnpm exec vitest run tests/retry.test.ts tests/client.test.ts (js-sdk), uv run pytest tests/test_rate_limit_retry_transport.py (python-sdk).

Link to Devin session: https://app.devin.ai/sessions/70b160f61f8e48bda8558173b88d9cd3
Open in Devin Desktop: https://app.devin.ai/desktop/session/70b160f61f8e48bda8558173b88d9cd3?variant=devin
Requested by: @mishushakov

[devin-ai-integration]

🤖 Devin AI Engineer

I'll be helping with this pull request! Here's what you should know:

✅ I will automatically:

• Address comments on this PR. Add '(aside)' to your comment to have me ignore it.
• Look at CI failures and help fix them

Note: I can only respond to comments from users who have write access to this repository.

⚙️ Control Options:

• Disable automatic comment, CI, and merge conflict monitoring

[changeset-bot]

🦋 Changeset detected

Latest commit: 0399882

The changes in this PR will be included in the next version bump.

This PR includes changesets to release 2 packages

Name	Type
@e2b/python-sdk	Patch
e2b	Patch

Not sure what this means? Click here to learn what changesets are.

Click here if you're a maintainer who wants to add another changeset to this PR

[github-actions]

Package Artifacts

Built from 08c4f31. Download artifacts from this workflow run.

JS SDK (e2b@2.51.1-devin-1789734898-retry-502-503.0):

npm install ./e2b-2.51.1-devin-1789734898-retry-502-503.0.tgz

CLI (@e2b/cli@2.20.1-devin-1789734898-retry-502-503.0):

npm install ./e2b-cli-2.20.1-devin-1789734898-retry-502-503.0.tgz

Code Interpreter JS SDK (@e2b/code-interpreter@2.8.1-devin-1789734898-retry-502-503.0):

npm install ./e2b-code-interpreter-2.8.1-devin-1789734898-retry-502-503.0.tgz

Desktop JS SDK (@e2b/desktop@2.4.1-devin-1789734898-retry-502-503.0):

npm install ./e2b-desktop-2.4.1-devin-1789734898-retry-502-503.0.tgz

Python SDK (e2b==2.51.0+devin.1789734898.retry.502.503):

pip install ./e2b-2.51.0+devin.1789734898.retry.502.503-py3-none-any.whl

Code Interpreter Python SDK (e2b-code-interpreter==2.10.0+devin.1789734898.retry.502.503):

pip install ./e2b_code_interpreter-2.10.0+devin.1789734898.retry.502.503-py3-none-any.whl

Desktop Python SDK (e2b-desktop==2.6.0+devin.1789734898.retry.502.503):

pip install ./e2b_desktop-2.6.0+devin.1789734898.retry.502.503-py3-none-any.whl

[devin-ai-integration]

TASTE.md review of the changed code only (retry layer in both SDKs, ConnectionOpts/ApiParams docs, changesets, tests).

Checked: parity (T-1/T-2), option shape (T-3/T-3a, T-10), acronym casing (T-12), named constants (T-47), timeout semantics (T-45), config precedence (T-49), entry points (T-54), error messaging (T-62/T-64), docs (T-69–T-71).

2 minor violations, 1 non-blocking note. No public API surface changes; the retries option and its docs mirror across SDKs, the retry-timeout message is actionable, and the base/max backoff defaults live in named constants in both languages. The two inline comments are the same T-47 nit on the JS and Python side — the jitter range is the only retry tunable spelled as a literal at the call site. The T-3a note on random_ is on an internal transport (not exported from e2b/__init__.py), so it's advisory.

[mishushakov]

for network errors I think in some test we already specify the shape of network error message, could you check if we're missing anything - in particular the sdk has to work cross-environments like Node, Deno, Bun, CF Workers and the Browser

[devin-ai-integration]

for network errors I think in some test we already specify the shape of network error message, could you check if we're missing anything - in particular the sdk has to work cross-environments like Node, Deno, Bun, CF Workers and the Browser

Checked against the shapes in tests/envd/handleEnvdApiError.test.ts / rpc.ts and probed real fetch() failures (refused port + unresolvable host) in each runtime:

Runtime	connect refused	DNS failure	retried?
Node (undici)	TypeError('fetch failed') → cause {code: ECONNREFUSED, syscall: connect}	cause {code: ENOTFOUND, syscall: getaddrinfo}	yes
Bun	TypeError {code: 'ConnectionRefused'}	TypeError {code: ENOTFOUND, syscall: getaddrinfo}	yes
Deno	TypeError('... client error (Connect): tcp connect error')	... client error (Connect): dns error	yes
CF Workers (workerd)	Error('Network connection lost.') {remote: true, retryable: true}	Error('internal error; reference = …') {remote: true}	no
Browser (Chrome)	TypeError('Failed to fetch')	same	no

The strings the existing tests pin (terminated, The socket connection was closed unexpectedly, error reading a body from connection, Network connection lost., network error) are all mid-request drops, i.e. the request may already have reached the server — so those are deliberately not matched (same reason ECONNRESET isn't). Added a test in aba3ca7 pinning every shape above so this stays explicit.

The gap is CF Workers and browsers: there a connect failure is indistinguishable from a mid-request drop (Network connection lost. is the same error the envd tests use for a torn-down connection; Chrome always says Failed to fetch, also for CORS). Two options:

1. keep it strict (current) — no connection retries on CF/browser, zero risk of duplicating a POST /sandboxes;
2. also retry the opaque shapes (Failed to fetch / NetworkError… / Load failed, and CF errors with retryable: true), accepting that a non-idempotent request could be replayed if the drop happened after the request was sent (this is what the AWS SDK does).

Which do you prefer? Happy to do 2 (or 2 restricted to GET/HEAD/DELETE) if you want retries there.

[devin-ai-integration]

Decision (from Mish): keep it strict — connection retries only for the unambiguous connect-phase shapes (Node/Bun/Deno); opaque CF Workers / browser network errors are rethrown as before. No further code change; the test in aba3ca7 pins this.

[claude]

I reviewed this PR and the automated bug hunt found no new bugs (one candidate — retrying non-idempotent POSTs like Sandbox.create on a 502/503 — was flagged as a duplicate of an earlier self-review comment, not independently confirmed safe). Because this changes core control-plane retry semantics, and safety for 502/503 retries rests on an assumption about backend/gateway behavior (that these statuses are always returned before the request is processed) that isn't verifiable from this diff alone, a human look would still be worthwhile — especially since packages/js-sdk/tests/client.test.ts explicitly exercises replaying POST /sandboxes after a 503.

What was reviewed: JS/Python backoff parity (100ms/0.1s base, 10s cap, 0.5-1.0 jitter, same RETRYABLE_STATUSES), the isConnectionError cause-chain/AggregateError walk and its exclusion of post-write errors (ECONNRESET), the request-timeout deadline math, and matching changesets/docstring updates for both SDKs.

Extended reasoning...

Overview

The diff extends control-plane HTTP retry logic in both the JS and Python SDKs to cover 502/503 responses (honoring Retry-After when present, otherwise exponential backoff with jitter, base 100ms/0.1s capped at 10s) and, in JS only, low-level connection-establishment failures via a new isConnectionError helper that walks cause chains and AggregateError members. withRateLimitRetry was renamed to withRetry; Python's RetryableTransport/AsyncRetryableTransport gained a keyword-only random_ dependency and a shared _retry_delay helper. Docs (connectionConfig.ts, connection_config.py), tests (retry.test.ts, client.test.ts, test_rate_limit_retry_transport.py), and two changesets accompany the change. I confirmed JS and Python semantics match (same retryable statuses, backoff base/cap, jitter range) and both sync and async Python transports were updated in parallel, consistent with CLAUDE.md's cross-SDK parity requirement. Changesets exist for both e2b and @ e2b/python-sdk, satisfying the public-surface-change requirement (retry conditions changed, withRateLimitRetry renamed, new keyword-only constructor args).

Security risks

No injection, auth-bypass, or data-exposure risk. The main correctness risk is retry safety: the change now retries requests on 502/503 regardless of HTTP method, which includes non-idempotent operations like Sandbox.create (POST) and pause/fork/snapshot. packages/js-sdk/tests/client.test.ts has a test literally named "client retries sandbox creation after a 503," confirming this is intentional, not an oversight. The PR description argues 502/503 are "rejected before/without processing" by the backend/gateway (unlike 504, which is deliberately excluded because the request may have already been applied), so replaying is safe. That argument is plausible but rests on backend/gateway behavior outside this diff (and outside code this PR touches), so it can't be independently verified from the SDK code alone.

Level of scrutiny

This warrants a higher-than-default level of scrutiny: it changes default retry behavior for every control-plane call across both SDKs (opt-out via retries: 0, not opt-in), and duplicate-sandbox-creation-on-retry is a plausible real-world failure mode if the "502/503 always precede processing" assumption doesn't hold universally for the actual API gateway/placement service. This exact question was already raised inline on the PR itself (retry.ts:9) and resolved only by the PR author, which per review guidance does not count as an independently addressed objection.

Other factors

Test coverage is otherwise solid: backoff sequencing with injected randomness, cap behavior, Retry-After preference, exhaustion, deadline checks, and non-retried statuses (400/404/500/504) are all covered on the JS side; sync/async Python equivalents exist for 502/503. isConnectionError correctly excludes post-write failures (ECONNRESET) and its own tests cover Node/undici, Bun, Deno, and AggregateError cases. Given the design question isn't a coding bug but a domain-knowledge judgment call about backend semantics, I chose to defer rather than flag it as a concrete bug.

This review covers commit 7d27596, which is no longer the latest commit on this pull request; later commits are not covered by it.

[mishushakov]

okay I think we should retry only GET methods for now

[devin-ai-integration]

okay I think we should retry only GET methods for now

Done in 24c7020 — isRetryableFetchError(err, method):

• connect-phase failures (Node/Bun/Deno shapes, request never sent) → retried for any method, as before;
• any other Error thrown by fetch (dropped connection, Failed to fetch, CF Network connection lost.) → retried only when method === 'GET';
• aborts (DOMException, or an aborted request signal) → never retried.

Same retries count, backoff and deadline as the rest. Tests cover GET vs POST/DELETE for each runtime's shape; docs/changeset/PR description updated.

[devin-ai-integration]

Policy is now operation-based and sourced from the spec (9428234 + d75d5c5):

• connect-phase failures (request never left) → retried for every operation, including POST /sandboxes;
• post-write / opaque network errors → retried unless the operation is marked x-idempotent: false in openapi.yml;
• aborts → never.

The marked operations (POST /sandboxes, /v2/sandboxes, /sandboxes/{id}/fork, /sandboxes/{id}/snapshots, /v3/templates, /api-keys, /admin/teams/{id}/api-keys, /volumes, /secrets, /events/webhooks) live in e2b-dev/runtime's spec; scripts/generate-retry-policy.mjs emits them into js-sdk/src/api/retryPolicy.gen.ts and python-sdk/e2b/retry_policy.py as part of make codegen.

Blocked: the runtime side is a patch (extension + spec test + regenerated api.gen.go), not a PR — branch creation on runtime is restricted for the integration. After it lands, bump spec/runtime-ref and re-run make codegen; until then the Generated files check here fails by design (the generator aborts when the pinned spec carries no marked operation).