Skip to content

feat: environments can inherit the deployment compute and source-control providers for nested Roomote - #2181

Open
mrubens wants to merge 5 commits into
developfrom
feat/environment-inherit-compute
Open

feat: environments can inherit the deployment compute and source-control providers for nested Roomote#2181
mrubens wants to merge 5 commits into
developfrom
feat/environment-inherit-compute

Conversation

@mrubens

@mrubens mrubens commented Sep 4, 2026

Copy link
Copy Markdown
Contributor

What changed

Environment definitions gain two opt-in flags:

  • inherit_compute: true forwards the deployment's default compute provider and that provider's setup-catalog values.
  • inherit_source_control: true forwards every fully configured source-control provider's setup-catalog values (for GitHub: App slug, App ID, private key, OAuth client id and secret, webhook secret). Partially configured providers are skipped.

Both are resolved at dequeue the way the control plane resolves them (validated Env first, then the encrypted deployment env) and travel under a single launcher-only name, R_NESTED_DEPLOYMENT_ENV. The worker expands that value into the real names for the nested environment env only, and drops the raw value from the worker runtime env, the harness process env, and every shell file, including the snapshot bootstrap path.

Local Docker is never forwarded (it needs the host socket). R_NESTED_DEPLOYMENT_ENV is added to the control-plane reserved set so operators cannot store it by hand, and it is stripped from ordinary repository tasks.

The repo's own .roomote/environments/roomote.yaml opts into both flags and exposes the nested API on an unproxied api port (13001). The controller command in that definition already reads ROOMOTE_API_HOST for TRPC_URL, so sandboxes spawned by the nested controller can reach the nested API directly.

Docs, the YAML serializer, and the environment preview cover both fields.

Why this change was made

Every compute credential and source-control app secret is in CONTROL_PLANE_ENV_VAR_NAMES, so redactControlPlaneEnvVars strips it before any sandbox sees it, and the controller-side worker env filter drops every *_SECRET. That is the right default, but it means a Roomote instance running inside a Roomote environment (Roomote-on-Roomote) can run Fast sessions yet cannot launch a coding task: its controller boots with no provider credentials, and even once it can spawn a sandbox, it cannot mint a repo token for the nested worker.

The GitHub App fields cannot be entered in the nested instance either, short of re-running its setup wizard on every outer task. PR #2154 and #2155 built the equivalent pass-through for model settings; this is the compute and source-control counterpart, made explicit per environment rather than blanket.

Impact

  • Environments that set neither flag are unchanged.
  • In an opted-in environment, every task can read the forwarded values from its shell, so both flags are documented as trust-scoped. For Roomote Cloud tenants the compute credential is the tenant's own broker credential, so nested sandboxes count against the same tenant. A forwarded GitHub App makes the nested instance act as the same app on the same repositories; webhooks keep arriving at the outer deployment.
  • Only the expanded names reach the nested app, and they are tracked as environment-owned so they survive deployment env reloads.

Verification

  • pnpm lint, pnpm check-types:fast, pnpm knip pass.
  • Tests in @roomote/types (compute and source-control builders, merge, parser, schema, reserved-name wiring), @roomote/sdk (compute forward, source-control forward with a partial provider skipped, both flags merged, opt-out, non-nestable provider, operator-stored value stripped), @roomote/worker (expansion into the nested env only, runtime-env and harness-env cleanup, shell omission, snapshot bootstrap), @roomote/web (YAML round-trip), and the @roomote/db declarative-environments test that validates roomote.yaml.
  • Compute forwarding was exercised end to end locally (see the verification comment below): the nested controller spawned a Modal sandbox with the local worker tarball and the nested worker reached the nested API through the unproxied port. The nested run then failed on repo token minting, which is exactly what inherit_source_control addresses.

@roomote-community

roomote-community Bot commented Sep 4, 2026

Copy link
Copy Markdown
Contributor

2 issues outstanding. See task

  • Snapshot setup writes the raw nested compute credential blob to env.sh (packages/sdk/src/server/lib/task-runs/fetch-snapshot-env.ts:48).
  • Nested compute forwarding omits Modal endpoint, registry, and ECR launch configuration (packages/types/src/nested-deployment-env.ts:60).
  • Azure DevOps delegated source control cannot work from the nested deployment because its account tokens remain in the outer database (packages/types/src/nested-deployment-env.ts:100).

Reviewed 1849406

Comment thread packages/sdk/src/server/lib/task-runs/fetch-snapshot-env.ts Outdated
@mrubens

mrubens commented Sep 4, 2026

Copy link
Copy Markdown
Contributor Author

Local end-to-end verification

Ran Roomote-on-Roomote locally against this branch: outer deployment on the managed provider (Modal backend), an environment definition equal to .roomote/environments/roomote.yaml with inherit_compute: true and the unproxied api port.

What the outer sandbox saw:

  • DEFAULT_COMPUTE_PROVIDER, the Roomote Cloud token pair, slug, and MODAL_BASE_IMAGE_REF present in the task env and exported by env.sh; R_NESTED_COMPUTE_ENV absent from env.sh (0 occurrences).
  • ROOMOTE_API_HOST set to the direct machine URL for port 13001; reachable from outside the sandbox (/ 200, /trpc/health 401). A curl to it from inside the sandbox returns 000, which is hairpin routing, not the port.
  • The local worker release tarball built inside the sandbox.

What the nested Roomote did:

  • Controller resolved the managed provider from the forwarded values, created a Modal sandbox in ~57s, uploaded the local-dev worker tarball, and the nested worker booted and called the nested API through the tunnel. Its run then failed on Failed to create source control token, which is expected for the seeded demo deployment (no source-control app configured) and unrelated to compute.

Two fixes came out of the run:

  • resolveComputeProviderEnvValues now receives the validated Env (4f8f260^): the raw process env has no NODE_ENV/APP_ENV, so the derived base image ref was null and the required-field check silently dropped the whole forward.
  • The raw R_NESTED_COMPUTE_ENV value was still reaching the harness process env via run-task.ts; it is now filtered there alongside the sandbox OpenRouter key, with a test.

@mrubens

mrubens commented Sep 4, 2026

Copy link
Copy Markdown
Contributor Author

@roomote-community the last sync review posted "No new code issues found" but the check run ended as "review result unavailable". Please re-review the current head.

@roomote-community

Copy link
Copy Markdown
Contributor

I started a PR review task for this request. See task

Comment thread packages/types/src/nested-compute.ts Outdated
DEFAULT_COMPUTE_PROVIDER: input.provider,
};

for (const field of descriptor.fields) {

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

descriptor.fields is not the full set of runtime values used to launch a provider. For Modal, the nested controller also reads MODAL_ENDPOINT, MODAL_ENVIRONMENT, MODAL_REGISTRY_USERNAME/MODAL_REGISTRY_PASSWORD, and ECR settings directly from Env (see RoomoteController.spawnFreshWorker). None are setup-catalog fields, so an opted-in deployment using a custom endpoint or private/ECR-backed worker image loses required launch configuration in the nested controller. Forward the complete provider launch configuration, including the paired registry/ECR values, rather than only catalog fields.

@mrubens mrubens changed the title feat: environments can inherit the deployment compute provider for nested Roomote feat: environments can inherit the deployment compute and source-control providers for nested Roomote Sep 4, 2026
): Record<string, string> | null {
const env: Record<string, string> = {};

for (const descriptor of SETUP_SOURCE_CONTROL_PROVIDER_CATALOG) {

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This treats Azure DevOps delegated configuration as forwardable, but ADO_LINKED_ACCOUNT_ID only identifies an authAccounts row in the outer deployment database. The nested Roomote has a different database, so resolveAdoDelegatedToken() finds no access/refresh token and createTaskRunAdoCredentials() fails. Skip delegated ADO from this inheritance mode (or pass a scoped credential through a mechanism that can actually be refreshed); copying its env fields cannot make the provider usable.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant