feat(errors): typed SandboxError subclasses at the runtime seam

[faresobeid]

Problem

Every runtime failure reaches a caller as one flat SandboxError whose only content is a message, so a consumer that must tell "retry later" from "the box is gone" from "the provider refused" has to match prose. Our data-flywheel environment (a long-lived seat over prime sandboxes) does exactly that today: substring checks on str(error) for "being placed on a node", "Failed to route", "routing catalog", "(unavailable)", and a walk down __cause__ looking for FileNotFoundError/httpx 404 — ~40 lines of classification that break each time the platform's wording moves ("routing catalog" is new this month). Two production runs (r2/r3) show what those messages carry: ~95× prime exec failed: Connect RPC failed (unavailable): The sandbox routing catalog is temporarily unavailable; retry shortly, 12× (unavailable): The sandbox is being placed on a node; retry shortly, 2× (unavailable): Failed to route request to sandbox, 119× prime sandbox provisioning failed: Sandbox did not reach RUNNING within 355s — all transient or provisioning faults, all arriving as the same type. verifiers' own whole-rollout retry filter (RetryConfig.include/exclude) matches by exception type name, so nobody can say "retry unavailable, never denied" either. The runtime knows the typed cause (an SDK exception class, an HTTP status, a Connect RPC code) the moment it wraps it; this PR keeps that knowledge in the class.

Reworks #2574 (closed in favour of this): classes instead of an instance code string, so except SandboxUnavailableError and RetryConfig work and the Error.code is derived from the class rather than a second vocabulary; the nine except … raise … from e sites in prime.py keep their shape; no _faults context manager, no cancellation handling, no monkeypatch of the SDK's live-process pump (the cancellation contract goes upstream to prime-sandboxes separately).

API

# verifiers.v1.errors (all exported from verifiers.v1)
class RolloutError(Exception):
    code: ClassVar[str | None] = None          # stable name of the failure class; recorded as trace.Error.code

class SandboxError(RolloutError): ...          # unchanged: the bare type when nothing typed says more
class SandboxNotFoundError(SandboxError):      code = "not_found"     # the path, or the box itself, is gone
class SandboxTimeoutError(SandboxError):       code = "timeout"       # the operation, or the box's lifetime, ran out
class SandboxUnavailableError(SandboxError):   code = "unavailable"   # provider/box transiently unreachable (retry later)
class SandboxDeniedError(SandboxError):        code = "denied"        # auth, billing, permission
class SandboxDiskFullError(SandboxError):      code = "disk_full"     # ENOSPC
class SandboxProvisioningError(SandboxError):  code = "provisioning"  # the box never came up

def sandbox_error(message: str, e: BaseException, *, default: type[SandboxError] = SandboxError) -> SandboxError

sandbox_error is the runtime counterpart of model_error: it reads typed evidence off e and the failures it chains (__cause__, else the implicit __context__ — the prime SDK raises APIError("HTTP 503: …") inside the httpx handler without from), never the message text — FileNotFoundError, TimeoutError/httpx.TimeoutException, OSError(ENOSPC), httpx.HTTPStatusError (404 / 401-403 / 408-504 / 429-5xx), other httpx.TransportError, and connectrpc.ConnectError by code (NOT_FOUND / DEADLINE_EXCEEDED / UNAVAILABLE-RESOURCE_EXHAUSTED-ABORTED / PERMISSION_DENIED-UNAUTHENTICATED). Anything else is default. connectrpc is imported lazily inside the RPC check (it ships with prime-sandboxes; the module stays importable without it).

runtimes/prime.py adds an _error(what, e, default=…) that reads the SDK's own types first (SandboxFileNotFoundError, UnauthorizedError/PaymentRequiredError, the five SDK timeout classes, SandboxImagePullError, SandboxOOMError / a SandboxNotRunningError whose status is TERMINATED/ERROR/TIMEOUT — the SDK's own "gone" set) and falls back to sandbox_error. The nine mapping sites become raise _error("prime exec failed", e) from e; start() passes default=SandboxProvisioningError; the egress-policy 60 s expiry raises SandboxTimeoutError. runtimes/subprocess.py's read maps through sandbox_error (a missing file → SandboxNotFoundError). harness._check_result raises SandboxNotFoundError for "runtime died" (the failed alive() probe is structural evidence, not text) — flagging this one; happy to keep it bare if you'd rather.

trace.Error gains code: str | None = None, filled from the class in Trace.record_error and env._as_error (only for RolloutErrors — arbitrary exceptions' .code attributes, e.g. openai.APIError.code, are not read). utils.retries._retryable matches the recorded type's class family (its name and its RolloutError bases', looked up in verifiers.v1.errors), so include: ["SandboxError"] still matches every subclass and exclude: ["SandboxDeniedError"] is expressible; a type not defined there matches only itself, as before.

Consumer side, what replaces the prose matching:

try:
    result = await runtime.run(argv, {})
except vf.SandboxUnavailableError:   # hold and retry
    ...
except vf.SandboxNotFoundError:      # the box is gone: provision a fresh one
    ...
# [env.seat.retries] max_retries = 2  include = ["SandboxUnavailableError", "SandboxTimeoutError", "SandboxProvisioningError"]

What is preserved

• except SandboxError catches everything it did; every subclass is a SandboxError. Messages keep their "<what>: <cause>" shape (the one exception: the port-exposure message ends …instead: <cause> rather than …instead. (<cause>)).
• Failures with no typed evidence stay bare SandboxError and behave exactly as today; the text-only sites (_read's non-zero exit, the non-VM open_process refusal, docker/modal/base) are untouched.
• RetryConfig.include/exclude keep their meaning for every existing config; Error.code is optional, so old records load with None. No new dependency, no pyproject.toml change, no config surface.
• One visible change: trace.last_error.type for a mapped failure is now the subclass name ("SandboxUnavailableError") rather than "SandboxError" — the same convention TunnelError(InterceptionError) already follows. verifiers' own filter is family-aware; external code comparing type == "SandboxError" exactly needs to widen.

Tests

tests/v1/test_errors.py (deterministic, no network, no model): sandbox_error over the evidence table incl. the two SDK shapes (APIError … from ConnectError(UNAVAILABLE) — the flywheel case — and a bare APIError("HTTP 503: …") with the typed status only on __context__), and RuntimeError("No such file or directory") staying bare; prime's _error over the SDK types, a bare APIError staying bare, and the provisioning default in start() yielding to typed evidence; record_error → Error.type/code and the to_record → model_validate round trip, a ProviderError and an old record carrying code is None; _retryable family matching (include=["SandboxError"] matches a SandboxUnavailableError; exclude wins; an unknown type matches only itself); SubprocessRuntime.read("missing") → SandboxNotFoundError.

uv run pytest tests/v1 -m "not e2e", ruff, pre-commit run --all-files, ty check verifiers all pass.

Note

Add typed SandboxError subclasses with stable codes at runtime seam

• Introduces SandboxNotFoundError, SandboxTimeoutError, SandboxUnavailableError, SandboxDeniedError, SandboxDiskFullError, and SandboxProvisioningError in errors.py, each carrying a stable code on RolloutError.
• sandbox_error and Prime-specific _error factories scan exception chains (HTTP status, Connect RPC code, errno, timeout, SDK type) to pick the most specific subclass, falling back to a caller-provided default.
• RolloutError.code is persisted on trace.py Error records via _as_error in env.py; existing traces without a code remain valid.
• Retry matching in retries.py now resolves RolloutError names to their inheritance family, so a policy listing a base class retries its subclasses.
• PrimeRuntime and SubprocessRuntime replace generic SandboxError wrapping with the new factories across start, run, read, write, expose, and background-launch paths.
• Runtime.alive and Harness._check_result now treat only SandboxNotFoundError as a dead runtime, propagating other typed probe failures unchanged.
• Risk: Runtime.alive in base.py and Harness._check_result in harness.py change liveness semantics — any out-of-tree code that caught SandboxError broadly will now receive specific subclasses that may not match existing except clauses.

^{Macroscope summarized c93fdee.}

---

Note

Medium Risk
Changes how sandbox failures are typed and recorded on traces (last_error.type becomes subclass names), which can affect external code that matched "SandboxError" exactly or relied on message substrings; core rollout/retry paths are intentionally updated but downstream consumers need to widen handling.

Overview
Replaces flat SandboxError messages with six typed subclasses (SandboxNotFoundError, SandboxTimeoutError, SandboxUnavailableError, SandboxDeniedError, SandboxDiskFullError, SandboxProvisioningError), each with a stable class-level code recorded on traces.

Adds sandbox_error() to classify failures from exception chains (errno, httpx status/transport, Connect RPC codes, __cause__/__context__) without parsing message text. Prime runtime uses _error() to map prime-sandboxes SDK types first, then sandbox_error; provisioning defaults to SandboxProvisioningError. Subprocess read paths map OSError through the same helper (missing file → SandboxNotFoundError).

trace.Error gains optional code; episode capture copies it from RolloutError. Whole-rollout RetryConfig matching treats a listed base type (e.g. SandboxError) as matching subclasses, with exclude still winning.

Harness dead-runtime handling raises SandboxNotFoundError; Runtime.alive() re-raises typed faults like unavailable/timeout instead of treating them as “gone.” Adds tests/v1/test_errors.py for classification, trace round-trip, retries, subprocess, and harness probe behavior.

^{Reviewed by Cursor Bugbot for commit c93fdee. Bugbot is set up for automated code reviews on this repo. Configure here.}

[cursor]

Cursor Bugbot has reviewed your changes and found 1 potential issue.

^{❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.}

^{Reviewed by Cursor Bugbot for commit 37b3065. Configure here.}

[macroscopeapp]

Approvability

Verdict: Not approved

Macroscope's review found this PR not approvable — This PR introduces a public typed-error capability while changing Prime, subprocess, harness, trace, and opt-in retry behavior. The cross-cutting runtime effects and retry/reprovision implications warrant human review despite the strong test coverage.

^{You can add or adjust custom eligibility rules. Learn more.}