Run Harbor Compose tasks on local Docker - #2547
Conversation
Codex Review SummaryThis comment shows the latest Codex review activity on this pull request.
ℹ️ About Codex in GitHubYour team has set up Codex to review pull requests in this repo. Reviews are triggered when you
Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings. |
![macroscopeapp[bot]](https://avatars.githubusercontent.com/in/900172?s=60&v=4){kind=small}
| port = parsed.port or (443 if parsed.scheme == "https" else 80) | ||
| if port not in self._forwarders: | ||
| listener = socket.socket() | ||
| listener.bind((self._gateway, 0)) |
There was a problem hiding this comment.
🟠 High harbor/runtime.py:140
On Linux, callback forwarders bind only to the Compose network gateway, but host.docker.internal resolves to Docker's configured host-gateway address, so Compose containers cannot connect to these listeners and model callbacks fail. Bind the listener to an address reachable through host-gateway (such as all host interfaces), or otherwise use the same gateway address for both.
- listener.bind((self._gateway, 0))
+ listener.bind(("0.0.0.0", 0))🚀 Reply "fix it for me" or copy this AI Prompt for your agent:
In file @verifiers/v1/tasksets/harbor/runtime.py around line 140:
On Linux, callback forwarders bind only to the Compose network gateway, but `host.docker.internal` resolves to Docker's configured `host-gateway` address, so Compose containers cannot connect to these listeners and model callbacks fail. Bind the listener to an address reachable through `host-gateway` (such as all host interfaces), or otherwise use the same gateway address for both.
| overlay["services"].setdefault(owner, {})["extra_hosts"] = { | ||
| "host.docker.internal": "host-gateway" | ||
| } |
There was a problem hiding this comment.
🟠 High harbor/runtime.py:69
When main uses network_mode: service:<sidecar>, the agent cannot resolve the host.docker.internal hostname in callback URLs, so model/interception requests fail. The mapping is added only to owner; add it to both main and the namespace owner.
- overlay["services"].setdefault(owner, {})["extra_hosts"] = {
- "host.docker.internal": "host-gateway"
- }
+ for service in {owner, "main"}:
+ overlay["services"].setdefault(service, {})["extra_hosts"] = {
+ "host.docker.internal": "host-gateway"
+ }🚀 Reply "fix it for me" or copy this AI Prompt for your agent:
In file @verifiers/v1/tasksets/harbor/runtime.py around lines 69-71:
When `main` uses `network_mode: service:<sidecar>`, the agent cannot resolve the `host.docker.internal` hostname in callback URLs, so model/interception requests fail. The mapping is added only to `owner`; add it to both `main` and the namespace owner.
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 28dffe7255
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| main: dict[str, object] = { | ||
| "image": self.config.image, | ||
| "working_dir": self.config.workdir, | ||
| } |
There was a problem hiding this comment.
When docker-compose.yaml is the sole source of main.image or main.working_dir, this override replaces those values with the default DockerConfig values (python:3.11-slim and /app). Such tasks therefore run the agent in the wrong image and directory rather than in the authored Compose environment, potentially invalidating both the solution and grading; only explicitly configured task/runtime overrides should replace these Compose fields.
Useful? React with 👍 / 👎.
| overlay = {"services": {"main": main}} | ||
| overlay["services"].setdefault(owner, {})["extra_hosts"] = { | ||
| "host.docker.internal": "host-gateway" | ||
| } |
There was a problem hiding this comment.
When main uses network_mode: service:<sidecar>, owner becomes that sidecar and this installs host.docker.internal only in the sidecar. Although the containers share a network namespace, their /etc/hosts files are separate, so the agent running in main cannot resolve the hostname returned by host_url() and consequently cannot reach the interception/model or host-local MCP endpoints. Keep any owner-specific setup needed for forwarding, but add the alias to main as well.
Useful? React with 👍 / 👎.
![cursor[bot]](https://avatars.githubusercontent.com/in/1210556?s=60&v=4){kind=small}
ApprovabilityVerdict: Not approved Macroscope's review found this PR not approvable — This PR adds a substantial Docker Compose runtime path with new container lifecycle, networking, callback, and borrowed-rollout behavior. Unresolved concerns about startup deadlines, callback reachability, Compose configuration preservation, and retries leave material runtime risk requiring human review. Not approved because:
No code changes detected at Adjust the Minimum Blocking Severity for this repo — including turning it Off — in Settings. You can add or adjust custom eligibility rules. Learn more. |
28dffe7 to
6bff76b
Compare
| overlay["services"].setdefault(owner, {})["ports"] = [ | ||
| f"127.0.0.1::{SERVICE_PORT}" | ||
| ] |
There was a problem hiding this comment.
🟠 High harbor/runtime.py:66
A Compose task whose main service uses network_mode: host fails to start because this code adds a ports mapping to main, which Docker Compose forbids with host networking. Skip the port mapping when the owner uses host networking.
- overlay["services"].setdefault(owner, {})["ports"] = [
- f"127.0.0.1::{SERVICE_PORT}"
- ]
+ if services.get(owner, {}).get("network_mode") != "host":
+ overlay["services"].setdefault(owner, {})["ports"] = [
+ f"127.0.0.1::{SERVICE_PORT}"
+ ]🚀 Reply "fix it for me" or copy this AI Prompt for your agent:
In file @verifiers/v1/tasksets/harbor/runtime.py around lines 66-68:
A Compose task whose `main` service uses `network_mode: host` fails to start because this code adds a `ports` mapping to `main`, which Docker Compose forbids with host networking. Skip the port mapping when the owner uses host networking.
| self._container = ( | ||
| await self._compose("ps", "--all", "--quiet", "main") | ||
| ).strip() | ||
| self.info.id = self._container |
There was a problem hiding this comment.
🟠 High harbor/runtime.py:106
Scaled main services cause setup to pass a newline-separated list of container IDs as one _container value to docker inspect, so Compose startup fails instead of handling replicas deterministically. Split the output into IDs and reject anything other than exactly one container (or select a replica explicitly).
- self._container = (
- await self._compose("ps", "--all", "--quiet", "main")
- ).strip()
+ container_ids = (
+ await self._compose("ps", "--all", "--quiet", "main")
+ ).splitlines()
+ if len(container_ids) != 1:
+ raise SandboxError(
+ f"Expected exactly one Compose main container, got {len(container_ids)}"
+ )
+ self._container = container_ids[0]🚀 Reply "fix it for me" or copy this AI Prompt for your agent:
In file @verifiers/v1/tasksets/harbor/runtime.py around lines 106-109:
Scaled `main` services cause setup to pass a newline-separated list of container IDs as one `_container` value to `docker inspect`, so Compose startup fails instead of handling replicas deterministically. Split the output into IDs and reject anything other than exactly one container (or select a replica explicitly).
| await self._compose("config", "--quiet") | ||
| self._created = True | ||
| # Finish an interrupted Compose launch before deleting its partial project. | ||
| await run_shielded(self._compose("up", "--detach", "--wait")) |
There was a problem hiding this comment.
🟠 High harbor/runtime.py:105
A Compose service that remains in its starting or health-check state makes docker compose up --wait run indefinitely, so run_shielded() prevents asyncio.timeout(timeouts.setup) from stopping setup and the runtime never tears down. Pass a --wait-timeout matching the configured setup timeout (or otherwise enforce that deadline) before waiting for Compose readiness.
🚀 Reply "fix it for me" or copy this AI Prompt for your agent:
In file @verifiers/v1/tasksets/harbor/runtime.py around line 105:
A Compose service that remains in its starting or health-check state makes `docker compose up --wait` run indefinitely, so `run_shielded()` prevents `asyncio.timeout(timeouts.setup)` from stopping setup and the runtime never tears down. Pass a `--wait-timeout` matching the configured setup timeout (or otherwise enforce that deadline) before waiting for Compose readiness.
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 6bff76b193
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| await self._compose("config", "--quiet") | ||
| self._created = True | ||
| # Finish an interrupted Compose launch before deleting its partial project. | ||
| await run_shielded(self._compose("up", "--detach", "--wait")) |
There was a problem hiding this comment.
When image pulling or a service health transition lasts beyond the configured setup timeout, run_shielded absorbs the cancellation from asyncio.timeout(timeouts.setup) and continues awaiting this command, so the timeout cannot actually stop startup and a worker may remain stuck indefinitely. Compose exposes --wait-timeout specifically to bound --wait (docker compose up); pass the remaining setup budget there or otherwise terminate the command before shielding cleanup.
Useful? React with 👍 / 👎.
| ) | ||
| await agents.agent.run( | ||
| task.defer_scoring() if separate else task, | ||
| runtime=runtime, |
There was a problem hiding this comment.
When a Compose rollout ends with a retryable provider or harness error and agent retries are configured, supplying this runtime makes Agent.run treat the box as borrowed; its retry loop explicitly stops after the first attempt because a borrowed box is dirty. Consequently every Compose task silently gets only one attempt while equivalent non-Compose tasks receive the configured retries; each retry needs a freshly provisioned Compose project or equivalent ownership-aware retry handling.
Useful? React with 👍 / 👎.
145f890 to
f589839
Compare
|
You have reached your Codex usage limits for code reviews. You can see your limits in the Codex usage dashboard. |
1 similar comment
|
You have reached your Codex usage limits for code reviews. You can see your limits in the Codex usage dashboard. |
There was a problem hiding this comment.
Cursor Bugbot has reviewed your changes and found 1 potential issue.
❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.
Reviewed by Cursor Bugbot for commit f589839. Configure here.
| main: dict[str, object] = { | ||
| "image": self.config.image, | ||
| "working_dir": self.config.workdir, | ||
| } |
There was a problem hiding this comment.
Overlay overwrites authored main image
High Severity
The compose overlay always sets main.image and main.working_dir from the resolved Docker config. When the task has no docker_image, that image is the runtime default (python:3.11-slim), so a later -f file replaces the authored compose image or build. An existing default image also makes Compose skip sidecar/main builds. The same overlay replaces an authored working_dir, so a preserved entrypoint can start in the wrong directory.
Reviewed by Cursor Bugbot for commit f589839. Configure here.
1 participant
Adds support for Harbor tasks that define Docker Compose environments. The Harbor env owns the project and lends its main container to the existing agent rollout.
Depends on #2528 for the shared container runtime and Docker callback proxy.
Compose preserves authored commands, entrypoints, dependencies, health checks and networking. Harbor supplies the base template and startup environment override; the Docker runtime supplies execution, live processes, file access and host callbacks. Main receives the resolved CPU and memory limits, and its runtime service port is published on host loopback through the service that owns its network namespace.
The project is removed with a checked Compose teardown, including volumes and orphan containers. Separate graders retain their own fresh runtime. This adapter supports CPU tasks with unrestricted local Docker networking.
Note
Add
HarborComposeRuntimeto run Harbor tasks on local Docker ComposeHarborComposeRuntimein runtime.py, aDockerRuntimesubclass that manages the Compose project lifecycle and exposes the main service as the agent runtime.HarborEnv.runin env.py so tasks with a Compose file start the project before the agent rollout, pass the runtime to the agent, and stop it on exit; setup is bounded by the resolved setup timeout.HarborComposeRuntime.startvalidates the Compose topology, generates overlay files, launches the project with detached wait semantics, selects the main container, and starts an egress proxy.HarborComposeRuntime.__init__rejects restricted-network and GPU configurations with a configuration error; tasks requiring those features are not supported under Compose.Macroscope summarized f589839.
Note
Medium Risk
New sandbox lifecycle path (Compose up/down, port publishing, egress proxy) affects task isolation and cleanup; scope is limited to CPU tasks with public Docker networking and local Docker only.
Overview
Adds Docker Compose support for Harbor tasks that ship
environment/docker-compose.yaml, so multi-service topologies run on local Docker while the agent still executes in themainservice.HarborComposeRuntime(new) owns the Compose project lifecycle: it merges Harbor’s prebuilt compose stack with overlays formain(resolved image, workdir, CPU/memory), publishes the runtime service port on the service that ownsmain’s network namespace, runsdocker compose up --detach --wait, and tears the project down with volumes on exit. GPU and restricted-network agent configs are rejected for this path.HarborEnv.runnow starts that runtime inside anAsyncExitStackwhen a compose file exists (setup bounded by the agent setup timeout), passes it intoagents.agent.run(..., runtime=...), and keeps separate-verifier behavior (defer scoring, artifact collection) unchanged. Separate graders still use the normal verifier runtime, not Compose.The shared
cli/_communicatehelpers accept an optionalenvdict merged over the host environment so Compose commands get Harbor infra variables.Docs document Compose behavior, limits (no GPU/restricted net, no sidecar collect on Compose), and clarify that ENTRYPOINT replacement only applies outside Compose.
Reviewed by Cursor Bugbot for commit f589839. Bugbot is set up for automated code reviews on this repo. Configure here.