Skip to content

sandbox-store: move the Direct substrate's restore-lease and receipt writes into a direct module - #1342

Merged
ac-pplx merged 1 commit into
ac/lifecycle-move-restore-leasefrom
ac/lifecycle-move-direct
Nov 3, 2026
Merged

ac-pplx merged 1 commit into
ac/lifecycle-move-restore-leasefrom
ac/lifecycle-move-direct

Conversation

@ac-pplx

@ac-pplx ac-pplx commented Nov 2, 2026 •

Copy link
Copy Markdown
Contributor

Mechanical move, no functional change. The sandboxes row writes listed below move into pplx-space-sandbox-store with their SQL byte-for-byte; callers keep their locks, transactions and error handling. Everything else is doc comments. Part of the wave-2 stack (5 of 10, on #1339); it lands with the stack.

Summary

The Direct substrate wrote the control-plane sandboxes row from inline SQL in crates/libs/api/src/controllers/substrate/direct.rs. Five of its six statements move into a new direct module of pplx-space-sandbox-store; the adoption update stays in the substrate (below). There is one function per transition, and each runs on the executor its caller passes:

  • claim_direct_restore_lease → direct::claim_restore_lease (operation step)
  • release_direct_restore_lease → direct::release_restore_lease (operation step)
  • renew_direct_restore_lease → direct::renew_restore_lease (operation step)
  • complete_direct_restore_lease → direct::complete_restore_lease (operation step)
  • persist_owned_restore_route_identity_receipt (placement UPDATE) → direct::place_owned_restore_receipt (observed projection)

The SQL text is the same byte for byte (checked by extracting the literals from both files). The binds carry the same values; SandboxId and NodeId encode as the same UUIDs. These stay in the substrate: the restore-ownership SELECT ... FOR UPDATE, the route-identity writes, the owned-restore transaction and the lease durations. Each function returns whether its guarded write matched, and the substrate maps a miss to the same error as before.

The doc comments note the guards a reader might not expect:

  • Renewal and completion check only the token. That is enough because the sandboxes_clear_restore_lease trigger clears the token whenever a row leaves restoring.
  • Releasing a retry lease leaves the lease in place.
  • The receipt placement has no lifecycle predicate. It relies on the caller's row lock.

sql! locators change: the moved statements now report pplx_space_sandbox_store::direct, which shows in pg_stat_activity and in query monitoring.

pattern_ratchet.sandbox_writes for substrate/direct.rs drops from 6 to 1. The one left is adopt_live_execution's update: its conditions join two Direct-only tables (sandbox_direct_route_identities, sandbox_direct_route_provenance), and moving it into the store would add an exception to the direct_tables rule. Exceptions there are not allowed to grow, so it stays in the substrate until those tables have an owning store. ARCHITECTURE.toml and substrate/AGENTS.md are unchanged.

Validation

  • cargo nextest run -p pplx-space-sandbox-store -p pplx-space-api: 1583 of 1584 passed, Postgres-backed tests ran with no dependency skips. The one failure, routes::e2b::tests::detail_reports_authoritative_federated_ownership_not_metadata (a JSON key-order snapshot), fails the same way on unmodified origin/main.
  • No test files changed.
  • Mutation checks, reverted afterwards:
    • Widening the claim's lifecycle guard to include 'started' fails non_restore_lifecycle_without_a_lease_reports_conflict and restore_lease_claim_and_release_move_only_restore_lifecycles.
  • cargo clippy --workspace --locked --all-targets --all-features -- -D warnings: clean. cargo fmt --check: clean.
  • cargo arch: 0 required findings (110 advisory, also when run as CI does, against the merge base's architecture-baseline.toml, renames and ARCHITECTURE.toml). The direct_tables no_crossing finding on the store's direct.rs is gone: the same run with the old ARCHITECTURE.toml reports it, at 111 findings. In this PR's files the only finding left is the existing file_size.listed_no_growth on substrate/direct.rs (grew from 3907 to 4424 code lines; its cap was already below the file's size on main).
  • Kind, Kubernetes lane (fresh cluster, up.sh 555 s, smoke-kind-typed.sh all 3159 s):
    • ECW pass: 82 passed, 1 failed, 5 inconclusive. Native pass: 35/35.
    • local.gentle_drain_leaves_sandbox_resident failed once and passed on rerun.
    • Two inconclusives were residue from a leftover sandbox and from that drain failure; both passed on rerun.
    • The other three inconclusives are bench conditions: no admin token file or receipt directory (2), and the Datadog telemetry mode (1).
  • Kind, Direct lane (fresh cluster with SPACE_KIND_SUBSTRATE=direct, up.sh 122 s, all 1023 s):
    • ECW pass: 51/51, including pause/connect, the pause race, wake-from-idle-suspend and restore scenarios.
    • Native pass: 26 passed, 1 failed. native.stopped_restart_preserves_rootfs failed at its cleanup step, after both stop/start cycles passed. On this branch's base, the fixture's cleanup looked up the sandbox's execution pod, which Direct doesn't create; kind-ci: run the Direct substrate lane nightly and on Direct-path PRs (not required) #1225 on main makes it use the API's node placement on Direct. None of the moved statements is on that step. This PR's Kind Direct CI run, on the merge with main, passed the full typed suite on Direct (ECW 51/51, native 27/27), this scenario included.

Severity and risk

Severity: does not apply (pure move). Risk: the restore and adoption paths of the Direct substrate. Because the statements are unchanged, old and new pods issue the same SQL during a rollout. Rollback: revert the commit.

Open PRs that will need to rebase

AI assistance: Claude Code (an AI agent) wrote the code change, ran the validation, and drafted this description.

After the wave-2 restack

This PR is one layer of a 10-PR stack, restacked onto main c5fdf0c7 (which includes #826) and then rebased mechanically onto 42a99002, which changed only baseline context lines, in order #1329 → #1336 → #1338 → #1339 → #1342 → #1347 → #1333 → #1335 → #1328 → #1332. The validation earlier in this description ran at pre-restack heads; this layer's head is 50153846.

@github-actions github-actions Bot added the triage:pending Waiting for a maintainer to triage label Nov 2, 2026
@github-actions

github-actions Bot commented Nov 2, 2026

Copy link
Copy Markdown

Thanks for opening this pull request! A SPACE maintainer will take a look.

CONTRIBUTING.md explains how we review outside contributions.

@ac-pplx
ac-pplx force-pushed the ac/lifecycle-move-direct branch from 3d05f85 to 3cb6431 Compare November 2, 2026 21:39
@ac-pplx
ac-pplx force-pushed the ac/lifecycle-move-direct branch from 3cb6431 to 01e4227 Compare November 2, 2026 23:25
@ac-pplx ac-pplx changed the title sandbox-store: move the Direct substrate's sandbox row writes into a direct module sandbox-store: move the Direct substrate's restore-lease and receipt writes into a direct module Nov 3, 2026
@ac-pplx
ac-pplx changed the base branch from main to ac/lifecycle-move-restore-lease November 3, 2026 03:50
@ac-pplx
ac-pplx added this pull request to stack #1382 November 3, 2026 03:50
@ac-pplx
ac-pplx marked this pull request as ready for review November 3, 2026 03:51
@ac-pplx
ac-pplx marked this pull request as draft November 3, 2026 03:52
@ac-pplx
ac-pplx force-pushed the ac/lifecycle-move-direct branch from 01e4227 to 14685f3 Compare November 3, 2026 06:00
@ac-pplx
ac-pplx marked this pull request as ready for review November 3, 2026 06:29
…direct module

The Direct substrate wrote the control-plane `sandboxes` row from inline
SQL in `controllers/substrate/direct.rs`. Five of those six statements
move into a new `direct` module of `pplx-space-sandbox-store`, one
function per transition that runs on the executor its caller passes:

- the restore lease's claim, release, renewal and completion
  (operation steps);
- the placement an owned restore receipt records (observed projection).

Statement text is unchanged byte for byte; binds carry the same values,
with `SandboxId`/`NodeId` encoding as the UUIDs bound before. The
restore-ownership `SELECT ... FOR UPDATE`, the route-identity writes, the
owned-restore transaction and the lease durations stay in the substrate.
Each function returns whether its guarded write matched, and the substrate
maps a miss to the same error as before.

The moved statements' `sql!` locators now name
`pplx_space_sandbox_store::direct`. The `sandbox_writes` ratchet entry for
`controllers/substrate/direct.rs` drops from 6 to 1.

The adoption of a live node record by a row with no active node stays in
the substrate. Its update joins `sandbox_direct_route_identities` and
`sandbox_direct_route_provenance`, Direct-only tables that
`pattern_ratchet.direct_tables` confines to the Direct substrate, and that
rule allows no new exception. The module's doc comments now say that a
guarded write that matches no row may also mean the row never met the
guard, and how the restore-lease trigger and `insert_unplaced` keep a
token-holding row `restoring`.
@ac-pplx
ac-pplx force-pushed the ac/lifecycle-move-direct branch from 14685f3 to 5015384 Compare November 3, 2026 21:24
@ac-pplx
ac-pplx added this pull request to the merge queue Nov 3, 2026
Merged via the queue into main with commit b020ecd Nov 3, 2026
27 checks passed
@ac-pplx
ac-pplx deleted the ac/lifecycle-move-direct branch November 3, 2026 22:03
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

triage:pending Waiting for a maintainer to triage

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants