diff --git a/.agentenv/config.example.toml b/.agentenv/config.example.toml index deb3ec35b..d73a63705 100644 --- a/.agentenv/config.example.toml +++ b/.agentenv/config.example.toml @@ -39,6 +39,11 @@ agent_default = "local" # agents too # [sandbox.providers.boat.config] # api_key = "secret:boat_api_key" # or "env:BOAT_API_KEY" +# Optional Agent37 VMs (no extra needed): +# [sandbox.providers.agent37.config] +# api_key = "secret:agent37_api_key" # or "env:AGENT37_API_KEY" +# region = "us" # or "eu", "ap-south" + # Optional CoreWeave Sandboxes (install the 'coreweave' extra): # [sandbox.providers.coreweave.config] # api_key = "secret:cwsandbox_api_key" # or wandb_api_key, with optional wandb_entity and wandb_project diff --git a/AGENTS.md b/AGENTS.md index aa980e2e8..6969e367b 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -69,9 +69,9 @@ or one of them only where it is used. store, an OCI registry at `localhost:5000`, env-var secret store, `LocalRunner`, `local` Docker sandboxes for envs and agents. MongoDB, DynamoDB, Firestore, S3, Cloud Storage, ECR, AWS Secrets Manager, Google Cloud Secret Manager, and the Modal, E2B, Sail, SmolVM, Tensorlake, Tenki, Vercel, boxd, - CoreWeave, Daytona and boat sandboxes exist as implementations and are selected by config (the last nine - through their extras: `sail`, `smol`, `tensorlake`, `tenki`, `vercel`, `boxd`, `coreweave`, `daytona`, `boat`; - every extra but `smol` is in `dev`). + CoreWeave, Daytona, boat and Agent37 sandboxes exist as implementations and are selected by config (all but + Agent37 through their extras: `sail`, `smol`, `tensorlake`, `tenki`, `vercel`, `boxd`, `coreweave`, `daytona`, + `boat`; every extra but `smol` is in `dev`; Agent37 needs only `httpx`). - The model endpoint is unset until `[model] base_url` / `api_key` (or `LITELLM_BASE_URL` / `LITELLM_API_KEY`) is configured. A loopback URL handed into a `local` container (the model endpoint, an agent, peer or gateway on this machine) becomes `host.docker.internal`. On Linux that @@ -122,7 +122,7 @@ and deserialized through a registry. | `task/`, `task_step/` | A `Task` holds its `TaskStep`s inline; `Task.run()` executes them as a DAG. `depends_on` (None means all prior steps) gates a step, independent steps run concurrently, `fail_task_on_error` makes a failure fatal or tolerated, `retry_config` rolls a failed span back through the step journal and re-dispatches it. Built-in steps live in `task_step/task_steps/` (`deploy_env`, `deploy_agent`, `prompt_agent`, the verifiers under `verifiers/`, among them `website_state_verifier`, which reads each checked session of a deployed website env's sites, or a multi env's, at the site's `/go?sid=` through `/svc//` and checks one dot path of its `current_state`, `initial_state` or `state_diff` each, optionally reduced by `len`, `keys` or `bool`, with `eq`, `ne`, `ge`, `gt`, `le`, `lt`, `in` or `contains`, its sites checked before a run against the `env_version` it pins, `start_computer_recording` and `end_computer_recording`, and more). On an env whose card serves computer-exec/v1 (a computer), `run_code` with `env_id` and `run_container_unit_tests_verifier` with `env_id` run inside it as its session user through the driver: files staged with `data/add` and read with `data/get`, commands run with `run`; `env_outcome_verifier` hands a `verify` that takes an `env_url` keyword the env's address. A computer's framework-only routes (everything but MCP and the card) take the per-deploy control token its provider boots it with (`DeployedEnv.control_token`, `AGENTENV_CONTROL_TOKEN`); framework calls send `deployed.control_headers`, `Task.run` runs every step under `protocol_v1.authorized` with each deployed env's (a `verify` included), and a `run_code` script inside the computer gets `input["env"]["headers"]`. `teardown_run` and the `teardown_sandboxes` step share one core (`task/teardown.py` `tear_down`): the recordings the run left open end with fresh grants, each env still up closes (`Env.close`; a computer env ends any recording still open), what stayed open is registered from its namespace, then every sandbox is terminated by id. `task/episode/` runs a task with some of its agents played from outside, as a training loop does (`EpisodeSpec`). | | `store/` | Four store ABCs with local and cloud implementations: `DocumentStore` (SQLite, MongoDB, DynamoDB, Firestore with MongoDB compatibility), `ObjectStore` (filesystem, S3, Cloud Storage), `ImageStore` (local OCI registry, ECR), `SecretStore` (env vars or file, AWS Secrets Manager, Google Cloud Secret Manager). `VersionedEntityStore` implements the shared versioned get/put logic, `QueryBuilder` is the immutable chained query API, `store/base.py` holds the error types. `store/routing.py` is the local namespaces, `@local` and `@hf` (`is_local_id`), each with a documents file of its own: while the CLI or a `build_app` server runs, a local id and what a local task's run records go to local stores beside the configured ones, and that run's writes to the configured stores are refused. A new backend must pass the conformance kits in `tst/store/`. | | `config/` | The `Config` singleton (`get_config`, `configure`, `reset_config`) in `config/runtime.py`, file discovery in `config/loader.py`, and `load_impl`, which resolves `module:Class` pointers. `agent_env.store` re-exports the config names for compatibility. | -| `providers/` | `providers/sandbox_providers/` holds the sandbox providers `local`, `modal`, `modal_vm`, `e2b`, `sail_vm` (the `sail` extra), `smol_vm` (the `smol` extra), `tensorlake` (the `tensorlake` extra), `tenki` (the `tenki` extra), `vercel` (the `vercel` extra), `boxd` (the `boxd` extra), `coreweave` (the `coreweave` extra), `daytona` (the `daytona` extra), `boat` (the `boat` extra); `[sandbox] default` and `agent_default` accept a comma-separated fallback chain, whose deploys skip a provider that can't run them. A sandbox provider declares what it runs (`ON_THIS_MACHINE`, and for VMs on this machine the address their requests to it arrive at, `REACHES_HOST_AT`; `CONTAINER_ACCEPTS`, `PRIVATE_NETWORK`; `SUPPORTED_SANDBOX_MODES`, derived from whether it implements `create_sandbox` and `create_vm`; and `SANDBOX_ACCEPTS`, the images an agent's sandbox runs, which defaults to those a VM loads when it implements `create_vm` and to `CONTAINER_ACCEPTS` otherwise, so only a provider that implements `create_vm` yet gives an agent a container, like the local one, declares it), each sandbox carries the `platform` of the images it runs (linux/amd64 unless its provider sets another when it creates it; None, any, unchecked, for a local sandbox), and code outside `providers/sandbox_providers/` reads those, never which provider it has. `providers/env_providers/` holds the environment providers: `EnvironmentProvider` (an env's containers and state store), `EnvironmentGatewayProvider` (`gateway`, an env's default), which renders a docker-compose for the gateway and its MCP servers inside the sandbox, and `EnvironmentServerProvider` (`server`), one MCP server in its own container with no gateway in front and no state store; `providers/env_state/` holds env-state providers (`local_postgres` built in). | +| `providers/` | `providers/sandbox_providers/` holds the sandbox providers `local`, `modal`, `modal_vm`, `e2b`, `sail_vm` (the `sail` extra), `smol_vm` (the `smol` extra), `tensorlake` (the `tensorlake` extra), `tenki` (the `tenki` extra), `vercel` (the `vercel` extra), `boxd` (the `boxd` extra), `coreweave` (the `coreweave` extra), `daytona` (the `daytona` extra), `boat` (the `boat` extra), `agent37` (no extra); `[sandbox] default` and `agent_default` accept a comma-separated fallback chain, whose deploys skip a provider that can't run them. A sandbox provider declares what it runs (`ON_THIS_MACHINE`, and for VMs on this machine the address their requests to it arrive at, `REACHES_HOST_AT`; `CONTAINER_ACCEPTS`, `PRIVATE_NETWORK`; `SUPPORTED_SANDBOX_MODES`, derived from whether it implements `create_sandbox` and `create_vm`; and `SANDBOX_ACCEPTS`, the images an agent's sandbox runs, which defaults to those a VM loads when it implements `create_vm` and to `CONTAINER_ACCEPTS` otherwise, so only a provider that implements `create_vm` yet gives an agent a container, like the local one, declares it), each sandbox carries the `platform` of the images it runs (linux/amd64 unless its provider sets another when it creates it; None, any, unchecked, for a local sandbox), and code outside `providers/sandbox_providers/` reads those, never which provider it has. `providers/env_providers/` holds the environment providers: `EnvironmentProvider` (an env's containers and state store), `EnvironmentGatewayProvider` (`gateway`, an env's default), which renders a docker-compose for the gateway and its MCP servers inside the sandbox, and `EnvironmentServerProvider` (`server`), one MCP server in its own container with no gateway in front and no state store; `providers/env_state/` holds env-state providers (`local_postgres` built in). | | `a2a_agent/` | The `A2AAgent` entity (`a2a_agent`), its stores and the validator steps. The protocol package provides the agent-side framework. | | `openenv/` | `build_app(bundle)`: the [OpenEnv](https://github.com/openenv/OpenEnv) server for a bundle's tasks, each session playing one episode through the Episode API, with the envs' MCP tools, an added `end_turn` tool and the task's verifier score as the reward. It prepares bundles as `agent-env run` does (`prepare_bundle(played=True)`, without the agent the client plays) and keeps namespace routing on for the app's life. `openenv` comes with the `openenv` extra, a floor with no cap; this package and `training/` are the only ones that import it. Each task's description in the Task API names its `toolset` (the played agent's envs, role, and whether it gets `bash`). With `token`, every request and WebSocket but `/health` must carry `Authorization: Bearer `, for a server others reach. | | `training/` | One module per outside trainer, each behind its own extra, and the modules they share; [Training helpers](#training-helpers) lists them. | diff --git a/src/agent_env/cli/a2a_agent/deploy.py b/src/agent_env/cli/a2a_agent/deploy.py index 17a8f89df..d1b2298de 100644 --- a/src/agent_env/cli/a2a_agent/deploy.py +++ b/src/agent_env/cli/a2a_agent/deploy.py @@ -18,7 +18,7 @@ @click.option("--ttl-seconds", type=click.IntRange(min=MIN_TTL_SECONDS, max=MAX_TTL_SECONDS), default=DEFAULT_TTL_SECONDS, help=f"VM lifetime in seconds (default {DEFAULT_TTL_SECONDS})") @click.option("--sandbox", default=None, - help="Sandbox backend(s): a built-in (modal, modal_vm, e2b, tensorlake, tenki, boxd, coreweave, daytona, boat, local) " + help="Sandbox backend(s): a built-in (modal, modal_vm, e2b, tensorlake, tenki, boxd, coreweave, daytona, boat, agent37, local) " "or a name from [sandbox.providers] in .agentenv/config.toml; comma-separated for a " "fallback chain. Defaults to [sandbox].agent_default (else local) when omitted.") def deploy(agent_id: str, agent_version: int | None, env_var_pairs: tuple[str, ...], ttl_seconds: int, sandbox: str): diff --git a/src/agent_env/cli/env/deploy.py b/src/agent_env/cli/env/deploy.py index fe1984c28..284923533 100644 --- a/src/agent_env/cli/env/deploy.py +++ b/src/agent_env/cli/env/deploy.py @@ -19,7 +19,7 @@ @click.option("--gateway-mode", type=click.Choice([m.value for m in GatewayMode], case_sensitive=False), default=GatewayMode.PERFORMANCE.value, help="Gateway mode (performance or consistent)") @click.option("--sandbox", default=None, - help="Sandbox backend(s): a built-in (modal, modal_vm, e2b, tensorlake, tenki, boxd, coreweave, daytona, boat, local) " + help="Sandbox backend(s): a built-in (modal, modal_vm, e2b, tensorlake, tenki, boxd, coreweave, daytona, boat, agent37, local) " "or a name from [sandbox.providers] in .agentenv/config.toml; comma-separated for a " "fallback chain. Defaults to [sandbox].default (else local) when omitted.") @click.option("--service-db", "service_db_env_id", default=None, diff --git a/src/agent_env/providers/sandbox_providers/agent37/README.md b/src/agent_env/providers/sandbox_providers/agent37/README.md new file mode 100644 index 000000000..14643593f --- /dev/null +++ b/src/agent_env/providers/sandbox_providers/agent37/README.md @@ -0,0 +1,140 @@ +# `agent37` sandbox provider + +Runs agent-env sandboxes on [Agent37](https://www.agent37.com) VMs. Each environment and each agent gets its own +Linux virtual machine with its own kernel, Docker and Compose v2 running inside, and public HTTPS URLs for its ports. +It is a VM provider: agent-env's docker-in-VM flows run on it unchanged. That covers the gateway's docker-compose, an +agent's `docker run`, image loading and artifact collection. + +## Set up + +### 1. Nothing to install + +The provider talks to Agent37's REST API with `httpx`, which agent-env already depends on, so there is no extra. + +### 2. Get an Agent37 API key + +Sign up at [agent37.com](https://www.agent37.com), add funds (VMs need a funded workspace), and create a key under +**API keys** in the dashboard. A key belongs to one workspace, and VMs are created and billed there. For local work, +export it: + +```bash +export AGENT37_API_KEY=sk_live_... +``` + +For a shared deployment, put it in your secret store instead, for example as `agent37_api_key`, and reference it with +`secret:agent37_api_key`. Never put the key itself in `config.toml`. The provider reads the key only from its config, +never from the process environment. + +### 3. Configure + +A complete `.agentenv/config.toml` that runs every environment and agent on Agent37, with the local stores: + +```toml +[sandbox] +default = "agent37" # environments and sandboxes +agent_default = "agent37" # agents + +[sandbox.providers.agent37.config] +api_key = "env:AGENT37_API_KEY" # or "secret:agent37_api_key" + +[model] +base_url = "https://litellm.example.com" +api_key = "env:LITELLM_API_KEY" +``` + +To keep the local default and use Agent37 per run instead, add only the `[sandbox.providers.agent37.config]` table and +pass `--sandbox agent37`, or `--env-sandbox agent37` / `--agent-sandbox agent37` on `agent-env task run`. +`agent-env config show` prints the file in effect and masks the key. + +### 4. Run something + +The bundled `hello` task deploys a sandbox, loads a file into it and checks it, with no model needed: + +```bash +agent-env run hello --sandbox agent37 +``` + +`agent-env -v run …` also logs each VM as it starts +(`Agent37 instance started: id=… name=… region=… resources=… ports=… attribution=…`). + +## Configuration reference + +All keys go under `[sandbox.providers.agent37.config]`. Unknown keys are refused. + +| Key | Default | Meaning | +| --------- | -------- | ------------------------------------------------------------------------------------ | +| `api_key` | required | The Agent37 API key, as an `env:` or `secret:` reference. | +| `region` | `us` | Where VMs run: `us`, `eu` (France) or `ap-south` (India). The price is the same in all three. | + +## Resources and lifetime + +- **Sizes:** Agent37 VMs come in four shapes: 2 vCPU / 4 GiB, 4 vCPU / 8 GiB, 4 vCPU / 16 GiB and 4 vCPU / 32 GiB. + A request gets the smallest that covers its CPU, memory and disk, so agent-env's default (1 vCPU, 8 GiB, 10 GB) runs + on 4 vCPU / 8 GiB. +- **Disk:** every VM gets its shape's largest disk: 12, 20, 40 or 80 GB. Docker keeps its images and containers on it. +- **Refused requests:** a request beyond 4 vCPU, 32 GiB or 80 GB is refused before anything is created, not shrunk. +- **Ignored requests:** `boot_mode` has no Agent37 equivalent, so a `boot_mode` that is set logs a warning and has no + effect. +- **Lifetime:** Agent37 has no delete-after timer, so the sandbox's `timeout` is not enforced and a VM runs until it is + terminated. agent-env tears its sandboxes down when a run ends; a run that crashes leaves its VMs running. They are + named `agentenv-` and carry the run's attribution as metadata, so they are easy to find in the dashboard or + with `GET https://api.agent37.com/v1/instances`, and `DELETE /v1/instances/{id}` removes one. +- **Idle VMs:** auto-sleep is off, so a VM never sleeps on its own while a run waits on it. + +## Networking + +- **Ports:** each exposed port is an Agent37 [public port](https://www.agent37.com/docs/agents-api/public-ports), an + `https://.agent37.app` URL that fills `tunnel_urls`. It serves HTTP, streaming responses and WebSocket with + no platform authentication; the random hostname is the only credential. +- **`/health`:** Agent37's edge answers the path `/health` on every URL itself, so a service behind a public port never + sees a request to exactly `/health`. Every other path reaches it. +- **Egress:** allow-all only. Agent37 can restrict a VM to a list of IPv4 addresses and ranges but not to hostnames, so + an allowlist is refused with `NetworkPolicyUnsupportedError` before anything is created, and a fallback chain skips + to its next provider. +- **Agent37's own hosts:** `*.agent37.app` is added to every other provider's allowlists, so a restricted agent + elsewhere can reach an environment running here. +- **Image and object downloads:** with a store that signs URLs (S3, Cloud Storage), the VM fetches objects and image + tarballs itself. Everything in the local stores goes over exec from this machine instead, which runs at this + machine's upload speed. +- **Reconnect:** `get_sandbox` restores the ports and the applied egress policy. + +## Attribution and cost + +agent-env's attribution, including `[sandbox.attribution]` defaults, is stored as the instance's `metadata`, so the +dashboard and `GET /v1/instances` show which run each VM belongs to. VMs are billed per minute from the workspace's +prepaid balance while they run, and deleting one ends its billing. + +## Cleanup + +- A VM whose setup fails after it is created (Docker never readies) is deleted. +- If the caller is cancelled while the create is in flight, the provider deletes the VM that create makes once it + answers. +- If a create fails after Agent37 may have made the VM, the provider finds it by its unique name and deletes it. +- `terminate` is idempotent: a VM that is already gone is not an error. + +## Things to know + +- **Commands run as root,** so the `sudo` agent-env prefixes is dropped. +- **Commands read no input:** each runs with an empty standard input. +- **Long commands:** Agent37 stops a command after 15 minutes, with exit code 137. Start anything longer in the + background (`nohup … > log 2>&1 &`) and poll it. +- **Resends:** each command carries a random run name, and a connection lost mid-command is resent under it. Agent37 + answers a resend with the first run's result, waiting for it if it is still running, so a command never runs twice. +- **Lost connections:** when no resend gets an answer, `exec` returns exit code -1. `exec_script` retries idempotent + scripts on -1 only. +- **Output:** exec returns at most 512 KB of each of stdout and stderr, and logs a warning when it cuts one. Copying a + file off the VM to a store that cannot sign URLs reads it 4 MiB at a time as base64 through stdout, which is past that + cap, so that copy fails rather than return a truncated file; use a store that signs URLs (S3, Cloud Storage) to + collect snapshots and images built on the VM. +- **Images:** `create_vm(image=...)` is refused. VMs boot Agent37's `agent37-ubuntu` image (Ubuntu with Docker, Compose + and buildx, with `dockerd` keeping its data under `/root/docker`), and OCI images run under Docker inside them. +- **Platform:** linux/amd64. GPUs are not supported. +- **The model key enters the sandbox:** an agent's `LITELLM_API_KEY` is passed in, as on other providers. + +## Testing + +`tst/integration/providers/sandbox_providers/agent37_sandbox_smoke_test.py` runs against a real Agent37 workspace when +`[sandbox.providers.agent37.config]` resolves, for example with `make int-test-slow`. Otherwise it skips with +`agentenv-capability-missing: remote_sandbox`. The unit tests are +`tst/unit/providers/sandbox_providers/agent37_sandbox_provider_test.py` and `agent37_sandbox_test.py`, and run offline +against an in-memory Agent37 API behind an `httpx.MockTransport`. diff --git a/src/agent_env/providers/sandbox_providers/agent37/__init__.py b/src/agent_env/providers/sandbox_providers/agent37/__init__.py new file mode 100644 index 000000000..8f61b1242 --- /dev/null +++ b/src/agent_env/providers/sandbox_providers/agent37/__init__.py @@ -0,0 +1 @@ +"""Agent37 VM sandbox provider; see README.md.""" diff --git a/src/agent_env/providers/sandbox_providers/agent37/provider.py b/src/agent_env/providers/sandbox_providers/agent37/provider.py new file mode 100644 index 000000000..51b7e699e --- /dev/null +++ b/src/agent_env/providers/sandbox_providers/agent37/provider.py @@ -0,0 +1,224 @@ +"""Agent37 sandbox provider. + +Each sandbox is one Agent37 VM instance (``type: "vm"`` on the ``agent37-ubuntu`` template): a Linux machine with its +own kernel and Docker running inside. The gateway and agent containers run under that Docker through the inherited +``VmSandbox`` helpers. +""" + +from __future__ import annotations + +import asyncio +import logging +import uuid +from typing import Any, ClassVar, Self + +import httpx + +from agent_env.attribution import Attribution +from agent_env.config.errors import ConfigError +from agent_env.providers.sandbox_providers.agent37.sandbox import ( + Agent37Api, + Agent37Sandbox, + applied_policy, + public_urls, +) +from agent_env.providers.sandbox_providers.sandbox import NetworkPolicy +from agent_env.providers.sandbox_providers.sandbox_provider import ( + SandboxProvider, + apply_default_attribution, + refuse_unenforceable_policy, +) + +logger = logging.getLogger(__name__) + +TEMPLATE = "agent37-ubuntu" +REGIONS = ("us", "eu", "ap-south") +NAME_PREFIX = "agentenv" +NAME_SUFFIX_LENGTH = 16 +_SECTION = "[sandbox.providers.agent37.config]" +_CONFIG_KEYS = frozenset({"api_key", "region"}) + +# Agent37's VM shapes, (vCPU, GiB of memory, largest disk in GB), smallest first. +SHAPES = ((2, 4, 12), (4, 8, 20), (4, 16, 40), (4, 32, 80)) +_cleanups: set[asyncio.Task] = set() + + +def instance_resources(cpu: float, memory_mb: int, disk_size_gb: float) -> dict[str, int]: + """The smallest shape that covers the request, with that shape's largest disk. A request beyond the largest shape is + refused, not shrunk.""" + for vcpu, memory_gib, disk_gb in SHAPES: + if cpu <= vcpu and memory_mb <= memory_gib * 1024 and disk_size_gb <= disk_gb: + return {"cpu": vcpu, "memory": memory_gib, "disk": disk_gb} + vcpu, memory_gib, disk_gb = SHAPES[-1] + raise ValueError( + f"Agent37 cannot size a VM for cpu={cpu}, memory={memory_mb}MB, disk={disk_size_gb}GB " + f"(at most {vcpu} vCPU, {memory_gib} GiB memory, {disk_gb} GB disk)" + ) + + +def instance_name() -> str: + """``agentenv-``, unique, so the instance a create lost its answer for can be found again.""" + return f"{NAME_PREFIX}-{uuid.uuid4().hex[:NAME_SUFFIX_LENGTH]}" + + +async def _settle(cleanup: Any) -> None: + """Run ``cleanup`` to its end, also while the caller is being cancelled: ``asyncio.run`` waits for the task it + cancels, so it waits for the cleanup too, which no Agent37 call holds past ``API_TIMEOUT_SECONDS``. A second + cancellation leaves the cleanup running on its own.""" + task = asyncio.ensure_future(cleanup) + _cleanups.add(task) + task.add_done_callback(_cleanups.discard) + await asyncio.shield(task) + + +class Agent37SandboxProvider(SandboxProvider): + """Provision Docker-capable Agent37 VMs. + + ``api_key`` comes from resolved provider config, never from the process environment: configure it with an ``env:`` + or ``secret:`` reference in ``[sandbox.providers.agent37.config]``. Workloads never see it. + """ + + # Exposed ports get public URLs under this domain. + EGRESS_HOSTS: ClassVar[tuple[str, ...]] = ("*.agent37.app",) + + def __init__(self, *, api_key: str, region: str = "us", transport: httpx.AsyncBaseTransport | None = None): + if not isinstance(api_key, str) or not api_key.strip(): + raise ValueError("Agent37 api_key must be a non-empty resolved secret") + if region not in REGIONS: + raise ValueError(f"Agent37 region must be one of {list(REGIONS)}, not {region!r}") + self._region = region + self._api = Agent37Api(api_key, transport=transport) + + @classmethod + def from_config(cls, **config: Any) -> Self: + unknown = set(config) - _CONFIG_KEYS + if unknown: + raise ConfigError(f"{_SECTION} has unknown key(s): {sorted(unknown)}") + api_key = config.get("api_key") + if not isinstance(api_key, str) or not api_key.strip(): + raise ConfigError(f"{_SECTION} requires a non-empty 'api_key'") + if config.get("region", "us") not in REGIONS: + raise ConfigError(f"{_SECTION} 'region' must be one of {list(REGIONS)}") + return cls(**config) + + async def close(self) -> None: + """Close this event loop's client; a sandbox this provider made reopens its connection if it is used again.""" + await self._api.close() + + async def create_vm( + self, + *, + image: str | None = None, + boot_mode: str | None = None, + cpu: float = 1.0, + memory: int = 8192, + disk_size_gb: float = 10, + timeout: int = 3600 * 2, + exposed_ports: list[int] | None = None, + setup_for_gateway: bool = True, + attribution: Attribution | None = None, + network_policy: NetworkPolicy | None = None, + ) -> Agent37Sandbox: + """Create an Agent37 VM with a public URL for each of ``exposed_ports``. Agent37 has no delete-after timer, so + ``timeout`` is ignored and the VM runs until it is terminated. VMs boot Agent37's Docker-capable Ubuntu image, + so a per-call ``image`` is refused rather than ignored; ``boot_mode`` has no Agent37 equivalent and is + ignored.""" + refuse_unenforceable_policy(self, network_policy) + if image is not None: + raise ValueError(f"Agent37 VMs boot the {TEMPLATE} image, which runs Docker; a per-call image is unsupported") + if boot_mode is not None: + logger.warning("Ignoring boot_mode=%s: Agent37 VMs have no boot modes", boot_mode) + resources = instance_resources(cpu, memory, disk_size_gb) + ports = list(dict.fromkeys(exposed_ports or [])) + metadata = { + key: str(value) + for key, value in apply_default_attribution(dict(attribution or {})).items() + if value is not None + } + body = { + "template": TEMPLATE, + "type": "vm", + "region": self._region, + "resources": resources, + "name": instance_name(), + "metadata": metadata, + "public_ports": [{"port": port} for port in ports], + } + instance = await self._create(body) + sandbox = Agent37Sandbox(self._api, instance["id"], tunnel_urls=public_urls(instance), + network_policy=NetworkPolicy()) + try: + logger.info( + "Agent37 instance started: id=%s name=%s region=%s resources=%s ports=%s attribution=%s " + "(timeout=%ss not enforced)", + sandbox.sandbox_id, body["name"], self._region, resources, ports, metadata, timeout, + ) + if setup_for_gateway: + await sandbox.setup_vm_for_gateway(ports) + return sandbox + except BaseException: + await _settle(self._delete_quietly(sandbox.sandbox_id)) + raise + + async def _create(self, body: dict[str, Any]) -> dict[str, Any]: + """Await the create. When it fails or its caller is cancelled, the instance it may have made anyway is deleted + once it settles, found by its unique name if its answer was lost.""" + task = asyncio.ensure_future(self._api.create_instance(body)) + try: + return await asyncio.shield(task) + except BaseException: + await _settle(self._reclaim(task, body["name"])) + raise + + async def _reclaim(self, create: asyncio.Future, name: str) -> None: + try: + instance_id = (await create)["id"] + except BaseException: + try: + instance_id = next((i["id"] for i in await self._api.list_instances() if i.get("name") == name), None) + except Exception as exc: # noqa: BLE001 - logged; the instance, if any, is named in the message + logger.error("Could not look for an Agent37 instance %s left by a failed create: %s", name, exc) + return + if instance_id is not None: + await self._delete_quietly(instance_id) + + async def _delete_quietly(self, instance_id: str) -> None: + try: + await self._api.delete_instance(instance_id) + logger.info("Deleted Agent37 instance %s after its create failed or was cancelled", instance_id) + except Exception as exc: # noqa: BLE001 - logged; the caller's own error is the one raised + logger.error("Agent37 instance %s from a failed create is still running; delete it: %s", instance_id, exc) + + async def create_sandbox( + self, + *, + image_name: str, + port: int, + env: dict[str, str], + cpu: float = 1.0, + memory: int = 8192, + disk_size_gb: float = 10, + timeout: int = 3600 * 2, + attribution: Attribution | None = None, + network_policy: NetworkPolicy | None = None, + ) -> Agent37Sandbox: + # A VM backend: the gateway and agent paths start image_name inside the VM afterwards. + del image_name, env + return await self.create_vm( + cpu=cpu, + memory=memory, + disk_size_gb=disk_size_gb, + timeout=timeout, + exposed_ports=[port], + attribution=attribution, + network_policy=network_policy, + ) + + async def get_sandbox(self, sandbox_id: str) -> Agent37Sandbox: + """Reconnect, restoring the public URLs and the applied egress policy.""" + instance = await self._api.get_instance(sandbox_id) + return Agent37Sandbox(self._api, instance["id"], tunnel_urls=public_urls(instance), + network_policy=applied_policy(instance.get("network"))) + + async def terminate_sandbox(self, sandbox_id: str) -> None: + await self._api.delete_instance(sandbox_id) diff --git a/src/agent_env/providers/sandbox_providers/agent37/sandbox.py b/src/agent_env/providers/sandbox_providers/agent37/sandbox.py new file mode 100644 index 000000000..6a232a476 --- /dev/null +++ b/src/agent_env/providers/sandbox_providers/agent37/sandbox.py @@ -0,0 +1,255 @@ +"""An Agent37 VM presented through the :class:`VmSandbox` contract. + +Agent37's ``agent37-ubuntu`` VMs boot Ubuntu with their own kernel and with Docker and Compose v2 running, so the +gateway's docker-compose, an agent's ``docker run`` and image loading run under Docker inside the machine. Commands run +as root through Agent37's exec API, so the ``sudo`` agent-env prefixes is dropped. Each exposed port is an Agent37 +public port, an HTTPS URL under ``agent37.app`` that the edge routes to the VM, so the VM opens no firewall ports. +""" + +from __future__ import annotations + +import asyncio +import logging +import shlex +import socket +import ssl +import uuid +import weakref +from typing import Any + +import httpx + +from agent_env.providers.sandbox_providers.sandbox import NetworkMode, NetworkPolicy, VmSandbox +from agent_env.providers.sandbox_providers.sandbox_provider import SANDBOX_MODE_VM, SandboxNotFoundError + +logger = logging.getLogger(__name__) + +API_URL = "https://api.agent37.com" +# Exec goes straight to the host the instance runs on, a hop shorter than through API_URL. +SANDBOX_URL = "https://sandbox.agent37.app" +# The image starts dockerd as its entrypoint; a command that reaches a fresh VM can be ahead of it. +DOCKER_READY = r""" +for _ in $(seq 90); do + docker info >/dev/null 2>&1 && exec docker compose version + sleep 1 +done +echo "docker daemon did not become ready" >&2 +exit 1 +""" +# What exec returns when no answer arrives; exec_script retries only this. +UNKNOWN_EXIT_CODE = -1 +# A resend under the same name gets the first run's answer, waiting for it if it is still running, so an exec whose +# connection drops is resent; so is one refused before the command started (503 at the per-instance command cap). +RESENDABLE = frozenset({409, 429, 500, 502, 503, 504, 520, 521, 522, 523, 524, 525, 526, 530}) +RESEND_DELAYS = (1, 2, 4, 8, 16) +# Past the 900 s Agent37 lets a command run, so its answer arrives first. +_TIMEOUT = httpx.Timeout(30.0, read=1000.0) +# Instance calls answer in about a second, a create once its VM is running. A create that has not answered by then is +# given up on, and the VM it may still make is found by name and deleted. +API_TIMEOUT_SECONDS = 120 +KEEPALIVE_SECONDS = 15 + + +def _keepalive_transport() -> httpx.AsyncHTTPTransport: + """A transport whose connections send TCP keepalives, so a command that prints nothing for minutes keeps the + connection waiting on it from being dropped as idle.""" + options = [(socket.SOL_SOCKET, socket.SO_KEEPALIVE, 1)] + idle = getattr(socket, "TCP_KEEPIDLE", None) or getattr(socket, "TCP_KEEPALIVE", None) + if idle is not None: + options.append((socket.IPPROTO_TCP, idle, KEEPALIVE_SECONDS)) + if hasattr(socket, "TCP_KEEPINTVL"): + options.append((socket.IPPROTO_TCP, socket.TCP_KEEPINTVL, KEEPALIVE_SECONDS)) + return httpx.AsyncHTTPTransport(socket_options=options) + + +class Agent37Error(RuntimeError): + """An Agent37 API call that failed, with the status and error code it answered.""" + + def __init__(self, response: httpx.Response): + self.status_code = response.status_code + self.code, message = _error_of(response) + super().__init__(f"Agent37 {response.request.method} {response.request.url.path} answered " + f"{response.status_code} {self.code}: {message}") + + +def _error_of(response: httpx.Response) -> tuple[str, str]: + """``(code, message)`` of an error answer: the API's ``{"error": {"code", "message"}}`` envelope, or the flat + ``{"error": ""}`` the edge answers with before a request reaches an instance.""" + try: + error = response.json().get("error") + except ValueError: + return "unknown", response.text[:500] + if isinstance(error, dict): + return str(error.get("code", "unknown")), str(error.get("message", "")) + return str(error), "" + + +class Agent37Api: + """The Agent37 calls a sandbox makes: instances on the control plane, and exec on the host running one. + + The key comes from the provider's resolved config and is sent only to Agent37. ``transport`` is injectable so unit + tests never open a connection.""" + + def __init__(self, api_key: str, *, transport: httpx.AsyncBaseTransport | None = None): + self._api_key = api_key + self._transport = transport + # One client per event loop: a client's connections belong to the loop that opened them, and the CLI runs + # several loops in one process. + self._clients: weakref.WeakKeyDictionary[asyncio.AbstractEventLoop, httpx.AsyncClient] = ( + weakref.WeakKeyDictionary()) + + def _client(self) -> httpx.AsyncClient: + loop = asyncio.get_running_loop() + client = self._clients.get(loop) + if client is None: + client = self._clients[loop] = httpx.AsyncClient( + transport=self._transport or _keepalive_transport(), timeout=_TIMEOUT, + headers={"User-Agent": "agentenv-framework"}, + ) + return client + + async def close(self) -> None: + client = self._clients.pop(asyncio.get_running_loop(), None) + if client is not None: + await client.aclose() + + async def _call(self, method: str, path: str, *, json: Any = None) -> httpx.Response: + return await self._client().request(method, f"{API_URL}{path}", json=json, + headers={"Authorization": f"Bearer {self._api_key}"}, + timeout=httpx.Timeout(30.0, read=API_TIMEOUT_SECONDS)) + + async def create_instance(self, body: dict[str, Any]) -> dict[str, Any]: + """Create an instance; Agent37 answers once it is running.""" + response = await self._call("POST", "/v1/instances", json=body) + if response.status_code != 201: + raise Agent37Error(response) + return response.json() + + async def get_instance(self, instance_id: str) -> dict[str, Any]: + response = await self._call("GET", f"/v1/instances/{instance_id}") + if response.status_code == 404: + raise SandboxNotFoundError(f"Agent37 instance {instance_id!r} was not found") + if response.status_code != 200: + raise Agent37Error(response) + return response.json() + + async def list_instances(self) -> list[dict[str, Any]]: + response = await self._call("GET", "/v1/instances") + if response.status_code != 200: + raise Agent37Error(response) + return response.json()["data"] + + async def delete_instance(self, instance_id: str) -> None: + """Delete an instance; one that is already gone is not an error.""" + response = await self._call("DELETE", f"/v1/instances/{instance_id}") + if response.status_code == 404: + logger.info("Agent37 instance %s was already gone at terminate", instance_id) + elif response.status_code != 200: + raise Agent37Error(response) + + async def exec(self, instance_id: str, body: dict[str, Any]) -> httpx.Response: + return await self._client().post(f"{SANDBOX_URL}/v1/instances/{instance_id}/exec", json=body, + headers={"X-Agent37-Key": self._api_key}) + + +def applied_policy(network: Any) -> NetworkPolicy: + """The egress policy an instance's ``network`` field applies: ``allow-all``, ``deny-all`` or ``{"allow": [...]}``, + a list of IPv4 addresses and ranges.""" + if network == "deny-all": + return NetworkPolicy(mode=NetworkMode.ALLOWLIST) + if isinstance(network, dict): + return NetworkPolicy(mode=NetworkMode.ALLOWLIST, allow_cidrs=tuple(network.get("allow") or ())) + return NetworkPolicy() + + +def public_urls(instance: dict[str, Any]) -> dict[int, str]: + """``{port: url}`` of an instance's public ports.""" + return {int(entry["port"]): entry["url"] for entry in instance.get("public_ports") or []} + + +class _BytesReader: + """The small ``StreamReader`` subset consumed by ``Sandbox.exec_with_output``.""" + + def __init__(self, value: str): + self._value = value.encode() + + async def read(self) -> bytes: + return self._value + + +class _CompletedProcess: + """A finished Agent37 exec in the common process protocol.""" + + def __init__(self, stdout: str, stderr: str, exit_code: int): + self.stdout = _BytesReader(stdout) + self.stderr = _BytesReader(stderr) + self._exit_code = exit_code + + async def wait(self) -> int: + return self._exit_code + + +class Agent37Sandbox(VmSandbox): + """An Agent37 VM with Docker; its API is its provider's, shared with the provider's other sandboxes.""" + + type = "agent37" + + def __init__( + self, + api: Agent37Api, + instance_id: str, + *, + tunnel_urls: dict[int, str], + network_policy: NetworkPolicy | None, + ): + self._api = api + self.sandbox_id = instance_id + self.mode = SANDBOX_MODE_VM + self.vnc_url = None + self.tunnel_urls = tunnel_urls + self.network_policy = network_policy + + async def terminate(self) -> None: + await self._api.delete_instance(self.sandbox_id) + + async def exec(self, *command: str) -> _CompletedProcess: + """Run argv to completion as root, with no input; Agent37 stops a command at 900 s. Every resend carries the + first call's ``name``, so Agent37 answers it with that run's result instead of running the command again; when no + resend gets an answer, the exit code is -1, which ``exec_script`` retries.""" + if command[:1] == ("sudo",): + command = command[1:] + body = {"command": shlex.join(command), "user": "root", "name": uuid.uuid4().hex} + resends = iter(RESEND_DELAYS) + while True: + try: + response = await self._api.exec(self.sandbox_id, body) + # httpx passes some TLS failures through unwrapped, such as a peer's bad-record-MAC alert. + except (httpx.TransportError, ssl.SSLError) as exc: + failure = f"{type(exc).__name__}: {exc}" + else: + if response.status_code == 200: + result = response.json() + if result.get("truncated"): + logger.warning("Agent37 cut the output of a command in %s to 512 KB per stream: %.200s", + self.sandbox_id, body["command"]) + return _CompletedProcess(result["stdout"], result["stderr"], int(result["exit_code"])) + if response.status_code not in RESENDABLE: + raise Agent37Error(response) + failure = str(Agent37Error(response)) + delay = next(resends, None) + if delay is None: + return _CompletedProcess("", failure, UNKNOWN_EXIT_CODE) + logger.warning("Agent37 exec in %s got no answer (%s); resending in %ss", self.sandbox_id, failure, delay) + await asyncio.sleep(delay) + + async def wait_for_vm(self) -> None: + """Wait until the VM's Docker daemon answers, and check for Compose v2, which ``deploy_env`` needs.""" + try: + await self.exec_script(DOCKER_READY, max_retries=2) + except RuntimeError as exc: + raise RuntimeError(f"Docker not ready on Agent37 instance {self.sandbox_id}: {exc}") from exc + logger.info("Docker ready on Agent37 instance %s", self.sandbox_id) + + async def setup_vm_for_gateway(self, exposed_ports: list[int] | None = None) -> None: + """Ready Docker; the edge routes each public port to the VM, so no firewall.""" + await self.wait_for_vm() diff --git a/src/agent_env/providers/sandbox_providers/sandbox_provider.py b/src/agent_env/providers/sandbox_providers/sandbox_provider.py index 4246ad97d..628984426 100644 --- a/src/agent_env/providers/sandbox_providers/sandbox_provider.py +++ b/src/agent_env/providers/sandbox_providers/sandbox_provider.py @@ -356,6 +356,7 @@ def effective_network_policy(cls, policy: Optional[NetworkPolicy]) -> NetworkPol "daytona": "agent_env.providers.sandbox_providers.daytona.provider:DaytonaSandboxProvider", "boat": "agent_env.providers.sandbox_providers.boat.provider:BoatSandboxProvider", "coreweave": "agent_env.providers.sandbox_providers.coreweave.provider:CoreWeaveSandboxProvider", + "agent37": "agent_env.providers.sandbox_providers.agent37.provider:Agent37SandboxProvider", "local": "agent_env.providers.sandbox_providers.local_docker.provider:LocalSandboxProvider", } diff --git a/tst/integration/env/gateway/gateway_test.py b/tst/integration/env/gateway/gateway_test.py index ab4057715..255aea263 100644 --- a/tst/integration/env/gateway/gateway_test.py +++ b/tst/integration/env/gateway/gateway_test.py @@ -488,6 +488,11 @@ def agentenv_website_env() -> WebsiteEnv: id="coreweave", marks=skip_without_remote_sandbox("coreweave"), ), + pytest.param( + "agent37", + id="agent37", + marks=skip_without_remote_sandbox("agent37"), + ), ]) def sandbox_provider(request): from agent_env.providers import ( @@ -527,6 +532,9 @@ def sandbox_provider(request): elif request.param == "coreweave": set_sandbox_provider(build_sandbox_provider("coreweave")) set_env_sandbox_provider(build_sandbox_provider("coreweave")) + elif request.param == "agent37": + set_sandbox_provider(build_sandbox_provider("agent37")) + set_env_sandbox_provider(build_sandbox_provider("agent37")) try: yield request.param finally: diff --git a/tst/integration/env/task_step/task_steps_test.py b/tst/integration/env/task_step/task_steps_test.py index 7745747a4..21397dc3e 100644 --- a/tst/integration/env/task_step/task_steps_test.py +++ b/tst/integration/env/task_step/task_steps_test.py @@ -307,6 +307,11 @@ def echo_agent() -> A2AAgent: id="coreweave", marks=skip_without_remote_sandbox("coreweave"), ), + pytest.param( + "agent37", + id="agent37", + marks=skip_without_remote_sandbox("agent37"), + ), ]) def sandbox_provider(request): from agent_env.providers import ( @@ -357,6 +362,10 @@ def sandbox_provider(request): set_sandbox_provider(build_sandbox_provider("coreweave")) set_env_sandbox_provider(build_sandbox_provider("coreweave")) set_agent_sandbox_provider(build_sandbox_provider("coreweave")) + elif request.param == "agent37": + set_sandbox_provider(build_sandbox_provider("agent37")) + set_env_sandbox_provider(build_sandbox_provider("agent37")) + set_agent_sandbox_provider(build_sandbox_provider("agent37")) try: yield request.param finally: diff --git a/tst/integration/providers/sandbox_providers/agent37_sandbox_smoke_test.py b/tst/integration/providers/sandbox_providers/agent37_sandbox_smoke_test.py new file mode 100644 index 000000000..a3a22a46c --- /dev/null +++ b/tst/integration/providers/sandbox_providers/agent37_sandbox_smoke_test.py @@ -0,0 +1,105 @@ +"""Agent37 VM smoke test. + +Runs when the resolved config can build the ``agent37`` sandbox provider (``[sandbox.providers.agent37.config]`` with +``api_key = "secret:agent37_api_key"``), the ``remote_sandbox`` capability, and skips with the declared reason +otherwise. The test never reads or prints the API key. +""" + +from __future__ import annotations + +import asyncio +import hashlib +import os + +import httpx +import pytest +import pytest_asyncio + +from agent_env.providers.sandbox_providers.sandbox import NetworkPolicy, read_vm_file +from agent_env.providers.sandbox_providers.sandbox_provider import SandboxNotFoundError, build_sandbox_provider +from tst.util.capabilities import skip_without_remote_sandbox + +_PORT = 8080 + +pytestmark = [ + pytest.mark.integration, + pytest.mark.int_test_slow, + pytest.mark.asyncio, + skip_without_remote_sandbox("agent37"), +] + + +async def _wait_for_200(url: str) -> None: + async with httpx.AsyncClient(timeout=10) as client: + for _ in range(60): + try: + if (await client.get(url)).status_code == 200: + return + except httpx.HTTPError: + pass + await asyncio.sleep(2) + pytest.fail(f"{url} never answered 200") + + +@pytest.fixture(scope="module") +def agent37_provider(): + return build_sandbox_provider("agent37") + + +@pytest_asyncio.fixture(scope="module") +async def agent37_sandbox(agent37_provider): + """One Agent37 VM serving on ``_PORT`` from a container, always deleted after the module.""" + sandbox = await agent37_provider.create_vm(cpu=1, memory=4096, exposed_ports=[_PORT], timeout=900) + try: + await sandbox.exec_script( + f"docker run -d --name smoke -p {_PORT}:{_PORT} public.ecr.aws/docker/library/python:3.12-slim " + f"python -m http.server {_PORT}" + ) + await _wait_for_200(sandbox.tunnel_urls[_PORT]) + yield sandbox + finally: + await sandbox.terminate() + + +async def test_agent37_vm_docker_ports_files_and_reconnect(agent37_sandbox, tmp_path): + """Nested Docker, a public port end to end, a write to a root-owned path, a read back, and reconnect.""" + assert (agent37_sandbox.type, agent37_sandbox.mode) == ("agent37", "vm") + assert agent37_sandbox.network_policy == NetworkPolicy() + + exit_code, version, stderr = await agent37_sandbox.exec_with_output( + "sudo", "docker", "info", "--format", "{{.ServerVersion}}" + ) + assert exit_code == 0, stderr + assert version.strip() + + payload = os.urandom(1024 * 1024 + 17) + await agent37_sandbox.write_host_file(payload, "/opt/smoke/payload.bin") + digest = await agent37_sandbox.exec_script("sha256sum /opt/smoke/payload.bin") + assert digest.split()[0] == hashlib.sha256(payload).hexdigest() + + # One read range; exec output is capped at 512 KB, so a file has to fit in one as base64. + await agent37_sandbox.exec_script("head -c 300007 /dev/urandom > /opt/smoke/small.bin") + remote = (await agent37_sandbox.exec_script("sha256sum /opt/smoke/small.bin")).split()[0] + await read_vm_file(agent37_sandbox, "/opt/smoke/small.bin", str(tmp_path / "small.bin")) + assert hashlib.sha256((tmp_path / "small.bin").read_bytes()).hexdigest() == remote + + reconnected = await build_sandbox_provider("agent37").get_sandbox(agent37_sandbox.sandbox_id) + assert reconnected.tunnel_urls == agent37_sandbox.tunnel_urls + assert reconnected.network_policy == NetworkPolicy() + assert (await reconnected.exec_with_output("true"))[0] == 0 + + +async def test_agent37_concurrent_commands_each_get_their_own_answer(agent37_sandbox): + """Several execs at once each return their own output, and a command that reads input sees its end.""" + results = await asyncio.wait_for( + asyncio.gather(*(agent37_sandbox.exec_with_output("bash", "-c", f"cat; echo {i}") for i in range(12))), 60, + ) + assert [(code, out.strip()) for code, out, _ in results] == [(0, str(i)) for i in range(12)] + + +async def test_agent37_terminate_is_idempotent(agent37_provider): + sandbox = await agent37_provider.create_vm(cpu=1, memory=4096, timeout=600, setup_for_gateway=False) + await sandbox.terminate() + await sandbox.terminate() + with pytest.raises(SandboxNotFoundError): + await agent37_provider.get_sandbox(sandbox.sandbox_id) diff --git a/tst/unit/providers/sandbox_providers/agent37_fakes.py b/tst/unit/providers/sandbox_providers/agent37_fakes.py new file mode 100644 index 000000000..d1237b02e --- /dev/null +++ b/tst/unit/providers/sandbox_providers/agent37_fakes.py @@ -0,0 +1,92 @@ +"""An in-memory Agent37 API behind an ``httpx.MockTransport``, for the offline provider and sandbox tests.""" + +from __future__ import annotations + +import json +from collections.abc import Callable +from typing import Any + +import httpx + +API_KEY = "sk_live_test_secret" + +ExecAnswer = httpx.Response | Exception | dict[str, Any] + + +def _error(status: int, code: str) -> httpx.Response: + return httpx.Response(status, json={"error": {"code": code, "message": code}}) + + +class FakeAgent37: + """Instances on ``api.agent37.com`` and exec on ``sandbox.agent37.app``. Each exec answers with the next of + ``exec_answers`` (a result body, a response, or an exception to raise), then with ``exec_result``. A create answers + ``create_answer`` when it is set: a response refuses it, an exception is raised after the instance is made.""" + + def __init__(self) -> None: + self.instances: dict[str, dict[str, Any]] = {} + self.requests: list[httpx.Request] = [] + self.created: list[dict[str, Any]] = [] + self.deleted: list[str] = [] + self.execs: list[dict[str, Any]] = [] + self.exec_answers: list[ExecAnswer] = [] + self.exec_result: Callable[[dict[str, Any]], dict[str, Any]] = ( + lambda body: {"exit_code": 0, "stdout": "", "stderr": "", "truncated": False}) + self.create_answer: httpx.Response | Exception | None = None + + def transport(self) -> httpx.MockTransport: + return httpx.MockTransport(self._handle) + + def requests_to(self, method: str, path: str) -> list[httpx.Request]: + return [r for r in self.requests if r.method == method and r.url.path == path] + + def _handle(self, request: httpx.Request) -> httpx.Response: + self.requests.append(request) + parts = request.url.path.strip("/").split("/") + if request.url.host == "sandbox.agent37.app": + assert request.headers["X-Agent37-Key"] == API_KEY + return self._exec(parts[2], json.loads(request.content)) + assert request.url.host == "api.agent37.com" + assert request.headers["Authorization"] == f"Bearer {API_KEY}" + if parts == ["v1", "instances"] and request.method == "POST": + return self._create(json.loads(request.content)) + if parts == ["v1", "instances"]: + return httpx.Response(200, json={"data": list(self.instances.values())}) + instance = self.instances.get(parts[2]) + if instance is None: + return _error(404, "not_found") + if request.method == "DELETE": + del self.instances[parts[2]] + self.deleted.append(parts[2]) + return httpx.Response(200, json={"id": parts[2], "deleted": True}) + return httpx.Response(200, json=instance) + + def _create(self, body: dict[str, Any]) -> httpx.Response: + self.created.append(body) + if isinstance(self.create_answer, httpx.Response): + return self.create_answer + instance_id = f"i{len(self.created):09d}" + self.instances[instance_id] = { + "id": instance_id, + "status": "running", + "name": body.get("name"), + "metadata": body.get("metadata"), + "network": "allow-all", + "public_ports": [ + {"port": entry["port"], "url": f"https://slug{entry['port']}{instance_id}.agent37.app"} + for entry in body.get("public_ports", []) + ], + } + if self.create_answer is not None: + raise self.create_answer + return httpx.Response(201, json=self.instances[instance_id]) + + def _exec(self, instance_id: str, body: dict[str, Any]) -> httpx.Response: + self.execs.append(body) + if instance_id not in self.instances: + return httpx.Response(404, json={"error": "not_found"}) + answer = self.exec_answers.pop(0) if self.exec_answers else self.exec_result(body) + if isinstance(answer, Exception): + raise answer + if isinstance(answer, httpx.Response): + return answer + return httpx.Response(200, json=answer) diff --git a/tst/unit/providers/sandbox_providers/agent37_sandbox_provider_test.py b/tst/unit/providers/sandbox_providers/agent37_sandbox_provider_test.py new file mode 100644 index 000000000..01d91b5a7 --- /dev/null +++ b/tst/unit/providers/sandbox_providers/agent37_sandbox_provider_test.py @@ -0,0 +1,217 @@ +"""Offline unit tests for the Agent37 sandbox provider.""" + +from __future__ import annotations + +import asyncio + +import httpx +import pytest + +from agent_env.config.errors import ConfigError +from agent_env.providers.sandbox_providers.agent37 import sandbox as sandbox_module +from agent_env.providers.sandbox_providers.agent37.provider import ( + NAME_SUFFIX_LENGTH, + TEMPLATE, + Agent37SandboxProvider, + instance_name, + instance_resources, +) +from agent_env.providers.sandbox_providers.agent37.sandbox import Agent37Error, applied_policy +from agent_env.providers.sandbox_providers.sandbox import NetworkMode, NetworkPolicy, NetworkPolicyUnsupportedError +from agent_env.providers.sandbox_providers.sandbox_provider import SandboxNotFoundError +from tst.unit.providers.sandbox_providers.agent37_fakes import API_KEY, FakeAgent37 + +ALLOWLIST = NetworkMode.ALLOWLIST + + +@pytest.fixture +def api(): + return FakeAgent37() + + +@pytest.fixture(autouse=True) +def _no_resend_delays(monkeypatch): + monkeypatch.setattr(sandbox_module, "RESEND_DELAYS", (0, 0, 0, 0, 0)) + + +def _provider(api: FakeAgent37, **kwargs) -> Agent37SandboxProvider: + return Agent37SandboxProvider(api_key=API_KEY, transport=api.transport(), **kwargs) + + +@pytest.mark.parametrize(("cpu", "memory", "disk", "resources"), [ + (1.0, 8192, 10, {"cpu": 4, "memory": 8, "disk": 20}), + (0.5, 1024, 10, {"cpu": 2, "memory": 4, "disk": 12}), + (2, 4096, 12, {"cpu": 2, "memory": 4, "disk": 12}), + (2, 4096, 13, {"cpu": 4, "memory": 8, "disk": 20}), + (3, 2048, 10, {"cpu": 4, "memory": 8, "disk": 20}), + (1, 8193, 10, {"cpu": 4, "memory": 16, "disk": 40}), + (4, 32768, 80, {"cpu": 4, "memory": 32, "disk": 80}), +]) +def test_resources_are_the_smallest_shape_that_covers_the_request(cpu, memory, disk, resources): + assert instance_resources(cpu, memory, disk) == resources + + +@pytest.mark.parametrize(("cpu", "memory", "disk"), [(5, 1024, 10), (1, 32769, 10), (1, 1024, 81)]) +def test_resources_refuse_rather_than_shrink(cpu, memory, disk): + with pytest.raises(ValueError, match="Agent37 cannot size a VM"): + instance_resources(cpu, memory, disk) + + +def test_instance_names_are_unique_and_carry_no_attribution(): + name = instance_name() + assert name.startswith("agentenv-") and len(name) == len("agentenv-") + NAME_SUFFIX_LENGTH + assert instance_name() != name + + +def test_from_config_takes_the_known_keys(): + provider = Agent37SandboxProvider.from_config(api_key="sk_live_x", region="eu") + assert provider._region == "eu" + assert Agent37SandboxProvider.from_config(api_key="sk_live_x")._region == "us" + + +@pytest.mark.parametrize(("config", "message"), [ + ({"api_key": "k", "org": "x"}, "unknown key"), + ({}, "requires a non-empty 'api_key'"), + ({"api_key": " "}, "requires a non-empty 'api_key'"), + ({"api_key": "k", "region": "mars"}, "'region' must be one of"), +]) +def test_from_config_refuses_bad_config(config, message): + with pytest.raises(ConfigError, match=message): + Agent37SandboxProvider.from_config(**config) + + +@pytest.mark.parametrize("policy", [ + NetworkPolicy(mode=ALLOWLIST), + NetworkPolicy(mode=ALLOWLIST, allow_hosts=("pypi.org",)), + NetworkPolicy(mode=ALLOWLIST, allow_cidrs=("203.0.113.0/24",)), +]) +def test_supports_only_unrestricted_egress(policy): + assert Agent37SandboxProvider.supports_network_policy(NetworkPolicy()) + assert not Agent37SandboxProvider.supports_network_policy(policy) + + +@pytest.mark.asyncio +async def test_create_vm_makes_an_ubuntu_vm_with_a_public_url_per_port_and_its_attribution(api): + sandbox = await _provider(api, region="ap-south").create_vm( + cpu=1.5, memory=4096, exposed_ports=[8080, 8080, 9000], setup_for_gateway=False, attribution={"run_id": "r1"}, + ) + [body] = api.created + assert {key: body[key] for key in ("template", "type", "region", "resources", "public_ports")} == { + "template": TEMPLATE, + "type": "vm", + "region": "ap-south", + "resources": {"cpu": 2, "memory": 4, "disk": 12}, + "public_ports": [{"port": 8080}, {"port": 9000}], + } + assert body["metadata"]["run_id"] == "r1" + assert body["name"].startswith("agentenv-") + assert (sandbox.type, sandbox.mode, sandbox.sandbox_id) == ("agent37", "vm", "i000000001") + assert sandbox.tunnel_urls == {8080: "https://slug8080i000000001.agent37.app", + 9000: "https://slug9000i000000001.agent37.app"} + assert sandbox.network_policy == NetworkPolicy() + assert api.execs == [] + + +@pytest.mark.asyncio +async def test_create_vm_readies_docker_for_the_gateway(api): + await _provider(api).create_vm(exposed_ports=[8000]) + [check] = api.execs + assert check["user"] == "root" + assert check["command"] == f"bash -c {sandbox_module.shlex.quote(sandbox_module.DOCKER_READY)}" + + +@pytest.mark.asyncio +async def test_create_vm_refuses_what_it_cannot_honour_before_creating_anything(api): + provider = _provider(api) + with pytest.raises(ValueError, match="per-call image is unsupported"): + await provider.create_vm(image="ubuntu:24.04") + with pytest.raises(NetworkPolicyUnsupportedError): + await provider.create_vm(network_policy=NetworkPolicy(mode=ALLOWLIST, allow_hosts=("pypi.org",))) + with pytest.raises(ValueError, match="cannot size"): + await provider.create_vm(cpu=16) + assert api.requests == [] + + +@pytest.mark.asyncio +async def test_a_vm_whose_docker_never_readies_is_deleted(api): + api.exec_result = lambda body: {"exit_code": 1, "stdout": "", "stderr": "docker daemon did not become ready", + "truncated": False} + with pytest.raises(RuntimeError, match="Docker not ready on Agent37 instance i000000001"): + await _provider(api).create_vm(exposed_ports=[8000]) + assert api.deleted == ["i000000001"] + assert api.instances == {} + + +@pytest.mark.asyncio +async def test_a_create_whose_answer_is_lost_deletes_the_instance_it_made_by_name(api): + api.create_answer = httpx.ReadError("connection reset") + with pytest.raises(httpx.ReadError): + await _provider(api).create_vm(setup_for_gateway=False) + assert api.deleted == ["i000000001"] + + +@pytest.mark.asyncio +async def test_a_refused_create_raises_the_reason_and_deletes_nothing(api): + api.create_answer = httpx.Response(402, json={"error": {"code": "insufficient_balance", "message": "Top up."}}) + with pytest.raises(Agent37Error, match="402 insufficient_balance: Top up."): + await _provider(api).create_vm(setup_for_gateway=False) + assert api.instances == {} and api.deleted == [] + assert api.requests_to("GET", "/v1/instances") + + +@pytest.mark.asyncio +async def test_a_cancelled_create_deletes_the_instance_once_the_create_answers(api): + gate = asyncio.Event() + provider = _provider(api) + create = provider._api.create_instance + + async def slow_create(body): + await gate.wait() + return await create(body) + + provider._api.create_instance = slow_create + task = asyncio.ensure_future(provider.create_vm(setup_for_gateway=False)) + await asyncio.sleep(0) + task.cancel() + await asyncio.sleep(0) + gate.set() + with pytest.raises(asyncio.CancelledError): + await task + assert api.deleted == ["i000000001"] + + +@pytest.mark.asyncio +async def test_create_sandbox_is_a_vm_exposing_the_agent_port(api): + sandbox = await _provider(api).create_sandbox(image_name="agent:latest", port=8000, env={"A": "1"}, + attribution={"run_id": "r2"}) + [body] = api.created + assert body["public_ports"] == [{"port": 8000}] + assert list(sandbox.tunnel_urls) == [8000] + + +@pytest.mark.asyncio +async def test_get_sandbox_from_a_fresh_provider_restores_ports_and_policy(api): + created = await _provider(api).create_vm(exposed_ports=[8080], setup_for_gateway=False) + reconnected = await _provider(api).get_sandbox(created.sandbox_id) + assert (reconnected.sandbox_id, reconnected.tunnel_urls) == (created.sandbox_id, created.tunnel_urls) + assert reconnected.network_policy == NetworkPolicy() + + +@pytest.mark.parametrize(("network", "policy"), [ + ("allow-all", NetworkPolicy()), + ("deny-all", NetworkPolicy(mode=ALLOWLIST)), + ({"allow": ["203.0.113.0/24"]}, NetworkPolicy(mode=ALLOWLIST, allow_cidrs=("203.0.113.0/24",))), +]) +def test_applied_policy_reads_the_instance_network_field(network, policy): + assert applied_policy(network) == policy + + +@pytest.mark.asyncio +async def test_terminate_is_idempotent_and_a_gone_instance_is_not_found(api): + provider = _provider(api) + sandbox = await provider.create_vm(setup_for_gateway=False) + await sandbox.terminate() + await sandbox.terminate() + await provider.terminate_sandbox(sandbox.sandbox_id) + with pytest.raises(SandboxNotFoundError): + await provider.get_sandbox(sandbox.sandbox_id) diff --git a/tst/unit/providers/sandbox_providers/agent37_sandbox_test.py b/tst/unit/providers/sandbox_providers/agent37_sandbox_test.py new file mode 100644 index 000000000..f8f00de65 --- /dev/null +++ b/tst/unit/providers/sandbox_providers/agent37_sandbox_test.py @@ -0,0 +1,95 @@ +"""Offline unit tests for the Agent37 sandbox handle: exec and its resends.""" + +from __future__ import annotations + +import ssl + +import httpx +import pytest +import pytest_asyncio + +from agent_env.providers.sandbox_providers.agent37 import sandbox as sandbox_module +from agent_env.providers.sandbox_providers.agent37.provider import Agent37SandboxProvider +from agent_env.providers.sandbox_providers.agent37.sandbox import Agent37Error +from tst.unit.providers.sandbox_providers.agent37_fakes import API_KEY, FakeAgent37 + + +@pytest.fixture(autouse=True) +def _no_resend_delays(monkeypatch): + monkeypatch.setattr(sandbox_module, "RESEND_DELAYS", (0, 0, 0, 0, 0)) + + +@pytest.fixture +def api(): + return FakeAgent37() + + +@pytest_asyncio.fixture +async def sandbox(api): + provider = Agent37SandboxProvider(api_key=API_KEY, transport=api.transport()) + return await provider.create_vm(setup_for_gateway=False) + + +def _result(stdout: str = "", exit_code: int = 0) -> dict: + return {"exit_code": exit_code, "stdout": stdout, "stderr": "", "truncated": False} + + +def _answer(status: int, code: str) -> httpx.Response: + return httpx.Response(status, json={"error": {"code": code, "message": code}}) + + +@pytest.mark.asyncio +async def test_exec_runs_argv_as_root_with_the_sudo_prefix_dropped(api, sandbox): + api.exec_answers = [{"exit_code": 3, "stdout": "out\n", "stderr": "err\n", "truncated": False}] + assert await sandbox.exec_with_output("sudo", "bash", "-c", "echo 'a b'; exit 3") == (3, "out\n", "err\n") + [body] = api.execs + assert body["command"] == "bash -c 'echo '\"'\"'a b'\"'\"'; exit 3'" + assert body["user"] == "root" + assert len(body["name"]) == 32 + + +@pytest.mark.asyncio +async def test_a_command_whose_answer_is_lost_is_resent_under_the_same_name(api, sandbox): + api.exec_answers = [httpx.ReadError("reset"), _answer(503, "try_again"), ssl.SSLError(1, "bad record mac"), + _answer(504, "try_again"), _result("done\n")] + assert await sandbox.exec_with_output("sleep", "20") == (0, "done\n", "") + assert len(api.execs) == 5 + assert len({body["name"] for body in api.execs}) == 1 + + +@pytest.mark.asyncio +async def test_a_command_no_resend_reaches_is_exit_minus_one(api, sandbox): + api.exec_answers = [httpx.ConnectError("unreachable")] * 6 + exit_code, stdout, stderr = await sandbox.exec_with_output("true") + assert (exit_code, stdout) == (sandbox_module.UNKNOWN_EXIT_CODE, "") + assert "ConnectError" in stderr + assert len(api.execs) == 6 + + +@pytest.mark.asyncio +async def test_exec_script_retries_an_idempotent_script_after_exit_minus_one(api, sandbox): + api.exec_answers = [httpx.ConnectError("unreachable")] * 6 + [_result("ok\n")] + assert await sandbox.exec_script("echo ok", max_retries=1) == "ok\n" + assert len({body["name"] for body in api.execs}) == 2 + + +@pytest.mark.asyncio +async def test_an_exec_refusal_that_is_not_transient_raises(api, sandbox): + api.exec_answers = [_answer(401, "unauthorized")] + with pytest.raises(Agent37Error, match="401 unauthorized"): + await sandbox.exec("true") + assert len(api.execs) == 1 + + +@pytest.mark.asyncio +async def test_exec_in_a_deleted_instance_raises_the_edge_error(api, sandbox): + await sandbox.terminate() + with pytest.raises(Agent37Error, match="404 not_found"): + await sandbox.exec("true") + + +@pytest.mark.asyncio +async def test_truncated_output_is_returned_with_a_warning(api, sandbox, caplog): + api.exec_answers = [{"exit_code": 0, "stdout": "head…tail", "stderr": "", "truncated": True}] + assert (await sandbox.exec_with_output("cat", "big"))[1] == "head…tail" + assert "512 KB" in caplog.text diff --git a/tst/unit/providers/sandbox_providers/egress_hosts_test.py b/tst/unit/providers/sandbox_providers/egress_hosts_test.py index 113e14023..49aef5fbf 100644 --- a/tst/unit/providers/sandbox_providers/egress_hosts_test.py +++ b/tst/unit/providers/sandbox_providers/egress_hosts_test.py @@ -50,6 +50,7 @@ def test_union_covers_every_registered_platforms_tunnel_hosts(platform_registry) assert "*.sb.tenki.sh" in hosts assert "*.boxd.sh" in hosts # boxd's p. subdomain proxies assert "*.daytonaproxy01.net" in hosts # Daytona's - preview URLs + assert "*.agent37.app" in hosts # Agent37's public-port URLs def test_union_holds_only_platform_hosts(): @@ -102,5 +103,5 @@ def test_the_builtin_floor_is_pinned(builtin_registry): assert set(all_sandbox_egress_hosts()) == { "*.modal.host", "*.w.modal.host", "*.e2b.app", "*.sail.box", "*.sandbox.tensorlake.ai", "*.sb.tenki.sh", "*.us.sb.tenki.sh", "*.vercel.run", "*.boxd.sh", "*.daytonaproxy01.net", "*.daytonaproxy01.eu", - "*.on.boat.dev", + "*.on.boat.dev", "*.agent37.app", } diff --git a/tst/unit/providers/sandbox_providers/network_policy_stamping_test.py b/tst/unit/providers/sandbox_providers/network_policy_stamping_test.py index 616ea88f3..1ce66af25 100644 --- a/tst/unit/providers/sandbox_providers/network_policy_stamping_test.py +++ b/tst/unit/providers/sandbox_providers/network_policy_stamping_test.py @@ -13,6 +13,7 @@ import pytest +from agent_env.providers.sandbox_providers.agent37.provider import Agent37SandboxProvider from agent_env.providers.sandbox_providers.boat.provider import BoatSandboxProvider from agent_env.providers.sandbox_providers.boxd.provider import BoxdSandboxProvider from agent_env.providers.sandbox_providers.coreweave.provider import CoreWeaveSandboxProvider @@ -156,6 +157,12 @@ async def _boat() -> Sandbox: return await provider.create_vm(exposed_ports=[], setup_for_gateway=False) +async def _agent37() -> Sandbox: + from tst.unit.providers.sandbox_providers.agent37_fakes import API_KEY, FakeAgent37 + provider = Agent37SandboxProvider(api_key=API_KEY, transport=FakeAgent37().transport()) + return await provider.create_vm(exposed_ports=[], setup_for_gateway=False) + + async def _smol_vm() -> Sandbox: pytest.importorskip("smol") from agent_env.providers.sandbox_providers.smol_vm import provider as smol_module @@ -183,6 +190,7 @@ async def _smol_vm() -> Sandbox: "coreweave": _coreweave, "daytona": _daytona, "boat": _boat, + "agent37": _agent37, "smol_vm": _smol_vm, "local": _local, } diff --git a/tst/unit/providers/sandbox_providers/provider_capabilities_test.py b/tst/unit/providers/sandbox_providers/provider_capabilities_test.py index b18058004..8cf3d5744 100644 --- a/tst/unit/providers/sandbox_providers/provider_capabilities_test.py +++ b/tst/unit/providers/sandbox_providers/provider_capabilities_test.py @@ -11,6 +11,8 @@ import pytest from agent_env.artifact.artifacts.docker_image import DockerImageArtifact +from agent_env.providers.sandbox_providers.agent37.provider import Agent37SandboxProvider +from agent_env.providers.sandbox_providers.agent37.sandbox import Agent37Sandbox from agent_env.providers.sandbox_providers.boat.provider import BoatSandboxProvider from agent_env.providers.sandbox_providers.boat.sandbox import BoatSandbox from agent_env.providers.sandbox_providers.boxd.provider import BoxdSandboxProvider @@ -46,7 +48,8 @@ LOADABLE, NAME, NAME_OR_CONTEXT = Accepts.LOADABLE, Accepts.NAME, Accepts.NAME_OR_CONTEXT PROVIDER_CLASSES = [LocalSandboxProvider, ModalSandboxProvider, ModalVmSandboxProvider, E2BSandboxProvider, SailVmSandboxProvider, TensorlakeSandboxProvider, TenkiSandboxProvider, BoxdSandboxProvider, - CoreWeaveSandboxProvider, DaytonaSandboxProvider, BoatSandboxProvider, ChainedSandboxProvider] + CoreWeaveSandboxProvider, DaytonaSandboxProvider, BoatSandboxProvider, Agent37SandboxProvider, + ChainedSandboxProvider] @pytest.mark.parametrize("cls, on_this_machine, creates_vms, sandbox, container, private_network", [ @@ -61,7 +64,9 @@ (CoreWeaveSandboxProvider, False, True, NAME, NAME, False), (DaytonaSandboxProvider, False, True, LOADABLE, NAME, False), (BoatSandboxProvider, False, True, LOADABLE, NAME, False), -], ids=["local", "modal", "modal_vm", "e2b", "sail_vm", "tensorlake", "tenki", "boxd", "coreweave", "daytona", "boat"]) + (Agent37SandboxProvider, False, True, LOADABLE, NAME, False), +], ids=["local", "modal", "modal_vm", "e2b", "sail_vm", "tensorlake", "tenki", "boxd", "coreweave", "daytona", "boat", + "agent37"]) def test_each_provider_declares_where_it_runs_and_the_images_it_runs(cls, on_this_machine, creates_vms, sandbox, container, private_network): assert ((cls.ON_THIS_MACHINE, SANDBOX_MODE_VM in cls.SUPPORTED_SANDBOX_MODES, cls.PRIVATE_NETWORK) @@ -283,7 +288,8 @@ def test_an_image_that_records_no_platform_isnt_checked(): @pytest.mark.parametrize("sandbox", [Sandbox, ModalSandbox, ModalVmSandbox, E2BSandbox, SailVmSandbox, TensorlakeSandbox, - TenkiSandbox, BoxdSandbox, CoreWeaveSandbox, DaytonaSandbox, BoatSandbox]) + TenkiSandbox, BoxdSandbox, CoreWeaveSandbox, DaytonaSandbox, BoatSandbox, + Agent37Sandbox]) def test_a_sandbox_runs_linux_amd64_unless_its_provider_sets_another(sandbox): assert sandbox.platform == "linux/amd64" diff --git a/tst/util/capabilities.py b/tst/util/capabilities.py index 0e2592f48..eaf4675f6 100644 --- a/tst/util/capabilities.py +++ b/tst/util/capabilities.py @@ -21,8 +21,8 @@ MODEL_ENDPOINT = "model_endpoint_configured" #: The remote sandbox providers (``modal``, ``modal_vm``, ``e2b``, ``sail_vm``, ``tensorlake``, ``tenki``, -#: ``vercel``, ``boxd``, ``coreweave``, ``daytona``, ``boat``) need credentials the resolved config may not carry; the -#: local default never does. +#: ``vercel``, ``boxd``, ``coreweave``, ``daytona``, ``boat``, ``agent37``) need credentials the resolved config may not +#: carry; the local default never does. REMOTE_SANDBOX = "remote_sandbox" #: The A2A agent a ``deploy_agent`` step without an id resolves to (``default_a2a_agent_id``); the @@ -110,17 +110,17 @@ def skip_without_model_endpoint() -> pytest.MarkDecorator: def remote_sandbox_is_available(provider: str) -> bool: """Whether the resolved config can build the ``modal`` / ``modal_vm`` / ``e2b`` / ``sail_vm`` / - ``tensorlake`` / ``tenki`` / ``vercel`` / ``boxd`` / ``coreweave`` / ``daytona`` / ``boat`` sandbox provider, credentials included. + ``tensorlake`` / ``tenki`` / ``vercel`` / ``boxd`` / ``coreweave`` / ``daytona`` / ``boat`` / ``agent37`` sandbox provider, credentials included. Modal: any failure to resolve the credentials answers False. E2B, Sail, Tensorlake, Tenki, Vercel, boxd, - CoreWeave, Daytona and boat: building the provider resolves ``[sandbox.providers..config]`` and its ``secret:`` references; + CoreWeave, Daytona, boat and Agent37: building the provider resolves ``[sandbox.providers..config]`` and its ``secret:`` references; only a ``ConfigError`` (absent or incomplete config, or the provider's extra not installed) answers False, any other failure answers True and lets the test fail on the real problem, as ``model_endpoint_is_configured`` does. Vercel, which builds without credentials, also needs them in its config or the environment.""" - if provider not in ("modal", "modal_vm", "e2b", "sail_vm", "tensorlake", "tenki", "vercel", "boxd", "coreweave", "daytona", "boat"): + if provider not in ("modal", "modal_vm", "e2b", "sail_vm", "tensorlake", "tenki", "vercel", "boxd", "coreweave", "daytona", "boat", "agent37"): raise ValueError(f"unknown remote sandbox provider {provider!r}") - if provider in ("e2b", "sail_vm", "tensorlake", "tenki", "vercel", "boxd", "coreweave", "daytona", "boat"): + if provider in ("e2b", "sail_vm", "tensorlake", "tenki", "vercel", "boxd", "coreweave", "daytona", "boat", "agent37"): from agent_env.config.errors import ConfigError from agent_env.providers.sandbox_providers.sandbox_provider import build_sandbox_provider