Repository navigation
sbx-mgr: add the persisted state-transition table and update_state_from (no callers) - #388
Merged
ac-pplx merged 1 commit intoOct 14, 2026
Conversation
ac-pplx
force-pushed
the
ac/sbxmgr-state-write-pr1-helper
branch
2 times, most recently
from
October 13, 2026 19:47
f27c281 to
2fc6457
Compare
ac-pplx
marked this pull request as ready for review
October 13, 2026 19:56
kumarpallav-pplx
requested review from
shrikant-pplx and
zibi-pplx
and removed request for
zibi-pplx
October 13, 2026 19:56
kumarpallav-pplx
requested review from
nikhilgupta-pplx and
the-gigi-pplx
and removed request for
shrikant-pplx
October 13, 2026 21:18
Collaborator
Review — 3 findingsMedium
Low
Verified against I also checked and dropped two candidates:
I didn't re-verify the edge inventory's per-writer from-sets or the count tables in the description against every call site. |
the-gigi-pplx
approved these changes
Oct 13, 2026
the-gigi-pplx
left a comment
the-gigi-pplx
left a comment
Collaborator
There was a problem hiding this comment.
Approved. 3 non-blocking findings in the review comment above.
This was referenced Oct 13, 2026
…om (no callers)
Add db/state_write.rs for the node-local SQLite sandboxes.state column:
- StateSet, a bitset over the 18 lifecycle states.
- EDGES: 73 data-only edge constants (from-set, target, kind) built from
every production writer at this head, each naming its writer functions.
Kinds are Request, Completion, Compensation, Recovery and KnownRace (the
stale or unvalidated writes generic update_state callers can make).
SandboxState::can_transition_to stays the RPC-admission machine.
- update_state_from(conn, id, &'static Edge) -> Applied { from } |
Refused { found, refusal } | Missing. Call sites name a table edge;
KnownRace and insert edges are refused. It applies update_state's
quarantine and cleanup-ownership guards, now generic_state_write_refusal
in this module, then writes with the edge's from-set predicate in the
UPDATE itself. A missed write from a source state is an invariant error.
It runs in a caller's transaction or opens its own IMMEDIATE one in
autocommit, never a nested one. It has no production callers yet.
update_state calls the moved guard; its behavior is unchanged, and
state_transitions.rs shrinks below its archcheck cap, which is lowered.
Tests:
- With every state expected, the writer matches update_state on every
cell of the existing outcome matrix and its whole-database diffs,
standalone and inside a transaction, by running the matrix's own grids
and sweeps against both writers.
- A from-set without the row's state refuses with every row unchanged, for
every (from, to) pair, with accepting controls; every writable edge
writes only from its own sources; refused edge kinds change nothing.
- The invariant error and the autocommit atomicity of a deleting write and
its teardown-job failure are pinned with temporary triggers.
- Every disagreement between the edge table and can_transition_to (85
pairs) is in a checked-in allowlist with a reason, so a new one fails.
The sbx-mgr AGENTS.md writer rule now points new transitions at
update_state_from with an edge, or a specialized compare-and-set writer.
ac-pplx
force-pushed
the
ac/sbxmgr-state-write-pr1-helper
branch
from
October 14, 2026 03:11
2fc6457 to
8646f74
Compare
ac-pplx
added a commit
that referenced
this pull request
Oct 14, 2026
Every write of sbx-mgr's node-local SQLite `sandboxes.state` now goes
through `update_state_from` with an explicit edge, or through a named
specialized writer. Nothing here touches the control-plane Postgres
`public.sandboxes.lifecycle`.
The three remaining db-internal `update_state` callers:
- The `deleting` re-entry inside `update_state` had no production caller:
Stop and Delete admission always write inside their own transaction.
`update_state_from` already opens an `IMMEDIATE` transaction in
autocommit for every target, so the deletion and the teardown jobs it
fails still commit together.
- `begin_resident_resume_with_policy` writes `edges::RESUME_BEGIN`
(`paused` -> `resuming`) inside its policy transaction. The edge now
refuses a non-paused row, so the separate state pre-check is gone; the
generation fence, the lock and the transaction are unchanged.
- Stop and Delete admission
(`update_state_superseding_activation_in_transaction`) cannot use an
exact edge without changing behavior. Admission re-admits a row that is
already in its target: a `stopping` row for a repeated Stop of a live
runtime, and a `deleting` row for a repeated Delete. No edge lists its
own target. The write therefore moves to `write_cleanup_admission_state`,
a private writer for only those two targets. It writes over every state
the generic guards admit, using `update_state_from`'s statement, guards
and warnings, as admission always has. `STOP_ADMISSION` and
`DELETE_ADMISSION` keep listing the transitions it makes.
`update_state`, `SandboxStateWriteOutcome`, the legacy
`require_state_write` and `CleanupCompletionError::StateWriteNotApplied`
are deleted. The guard types collapse: `db::StateWriteGuard { Quarantined,
CleanupOwned }`, `generic_state_write_refusal` returns
`Option<StateWriteGuard>`, and `StateWriteRefusal` is
`FromState | Guard(StateWriteGuard)`. That removes the unreachable
"guards never answer FromState" arm. `SandboxStateWriteNotApplied` becomes
`Guard(StateWriteGuard) | Missing` and renders the same client-visible text
byte for byte. `into_status`, checkpoint admission and cleanup admission
all render through it.
Tests:
- The `update_state` outcome matrix already ran through `update_state_from`
with every state expected, standalone and in a transaction (#388). Only
its `update_state` arm is dropped, so every guard cell stays pinned. The
guard tests in `db/tests.rs` now target production edges, and the #438
status tests keep their literal texts without the legacy comparison.
- New: Delete admission from every source state. It admits every plain row,
a `deleting` row included, except a `suspending` row whose suspend has
not reached the cluster. Nothing pinned Delete re-admission before.
- Test-only callers use an edge where production has one, `force_state`
where only the state matters, and an explicit-source
`update_state_from_states` for the two `started` -> `restoring` seeds.
- The five copies of the Stop fixture (superseding write, `mark_stopped`
intent, row-intent completion) are now `admit_stop`, `complete_stop` and
`stop` in `db/tests/state_write_fixtures.rs`.
Review comments from the stack's approval, folded in:
- `update_state_from_best_effort` logs a missing row with its own message,
instead of "superseded".
- `require_state_write_from` takes `context: &str`, as `into_status` does.
- Pause's comment on `mark_quarantined_best_effort` now says it also
answers false on a poisoned connection or a database error.
- A refused startup recovery write names the edge, the found state and the
refusal, instead of printing the outcome's `Debug` form. A missing row
has its own error.
- Auto-lifecycle Stop gains a case for a Delete admitted mid-write: a
`deleting` row with a current `delete_row` intent, which the
cleanup-ownership guard refuses and the Stop skips.
- Suspend's guest-prep rollback refusal and auto-pause refusal still answer
`Internal`. Both are unreachable under suspend's fence and locks, and a
comment now says so.
- Warm start's doc says why a guarded `suspended` row stays `Aborted`, and
that answering a transient row with `FailedPrecondition` is deliberate.
- Checkpoint admission renders through `SandboxStateWriteNotApplied`, not
the legacy outcome. A missing-row case pins its text alongside the
existing guard literals.
The sbx-mgr AGENTS.md writer rule is updated, and the archcheck cap for
`db/state_transitions.rs` is lowered to its new size.
ac-pplx
added a commit
that referenced
this pull request
Oct 14, 2026
Every write of sbx-mgr's node-local SQLite `sandboxes.state` now goes
through `update_state_from` with an explicit edge, or through a named
specialized writer. Nothing here touches the control-plane Postgres
`public.sandboxes.lifecycle`.
The three remaining db-internal `update_state` callers:
- The `deleting` re-entry inside `update_state` had no production caller:
Stop and Delete admission always write inside their own transaction.
`update_state_from` already opens an `IMMEDIATE` transaction in
autocommit for every target, so the deletion and the teardown jobs it
fails still commit together.
- `begin_resident_resume_with_policy` writes `edges::RESUME_BEGIN`
(`paused` -> `resuming`) inside its policy transaction. The edge now
refuses a non-paused row, so the separate state pre-check is gone; the
generation fence, the lock and the transaction are unchanged.
- Stop and Delete admission
(`update_state_superseding_activation_in_transaction`) cannot use an
exact edge without changing behavior. Admission re-admits a row that is
already in its target: a `stopping` row for a repeated Stop of a live
runtime, and a `deleting` row for a repeated Delete. No edge lists its
own target. The write therefore moves to `write_cleanup_admission_state`,
a private writer for only those two targets. It writes over every state
the generic guards admit, using `update_state_from`'s statement, guards
and warnings, as admission always has. `STOP_ADMISSION` and
`DELETE_ADMISSION` keep listing the transitions it makes.
`update_state`, `SandboxStateWriteOutcome`, the legacy
`require_state_write` and `CleanupCompletionError::StateWriteNotApplied`
are deleted. The guard types collapse: `db::StateWriteGuard { Quarantined,
CleanupOwned }`, `generic_state_write_refusal` returns
`Option<StateWriteGuard>`, and `StateWriteRefusal` is
`FromState | Guard(StateWriteGuard)`. That removes the unreachable
"guards never answer FromState" arm. `SandboxStateWriteNotApplied` becomes
`Guard(StateWriteGuard) | Missing` and renders the same client-visible text
byte for byte. `into_status`, checkpoint admission and cleanup admission
all render through it.
Tests:
- The `update_state` outcome matrix already ran through `update_state_from`
with every state expected, standalone and in a transaction (#388). Only
its `update_state` arm is dropped, so every guard cell stays pinned. The
guard tests in `db/tests.rs` now target production edges, and the #438
status tests keep their literal texts without the legacy comparison.
- New: Delete admission from every source state. It admits every plain row,
a `deleting` row included, except a `suspending` row whose suspend has
not reached the cluster. Nothing pinned Delete re-admission before.
- Test-only callers use an edge where production has one, `force_state`
where only the state matters, and an explicit-source
`update_state_from_states` for the two `started` -> `restoring` seeds.
- The five copies of the Stop fixture (superseding write, `mark_stopped`
intent, row-intent completion) are now `admit_stop`, `complete_stop` and
`stop` in `db/tests/state_write_fixtures.rs`.
Review comments from the stack's approval, folded in:
- `update_state_from_best_effort` logs a missing row with its own message,
instead of "superseded".
- `require_state_write_from` takes `context: &str`, as `into_status` does.
- Pause's comment on `mark_quarantined_best_effort` now says it also
answers false on a poisoned connection or a database error.
- A refused startup recovery write names the edge, the found state and the
refusal, instead of printing the outcome's `Debug` form. A missing row
has its own error.
- Auto-lifecycle Stop gains a case for a Delete admitted mid-write: a
`deleting` row with a current `delete_row` intent, which the
cleanup-ownership guard refuses and the Stop skips.
- Suspend's guest-prep rollback refusal and auto-pause refusal still answer
`Internal`. Both are unreachable under suspend's fence and locks, and a
comment now says so.
- Warm start's doc says why a guarded `suspended` row stays `Aborted`, and
that answering a transient row with `FailedPrecondition` is deliberate.
- Checkpoint admission renders through `SandboxStateWriteNotApplied`, not
the legacy outcome. A missing-row case pins its text alongside the
existing guard literals.
The sbx-mgr AGENTS.md writer rule is updated, and the archcheck cap for
`db/state_transitions.rs` is lowered to its new size.
ac-pplx
added a commit
that referenced
this pull request
Oct 14, 2026
Every write of sbx-mgr's node-local SQLite `sandboxes.state` now goes
through `update_state_from` with an explicit edge, or through a named
specialized writer. Nothing here touches the control-plane Postgres
`public.sandboxes.lifecycle`.
The three remaining db-internal `update_state` callers:
- The `deleting` re-entry inside `update_state` had no production caller:
Stop and Delete admission always write inside their own transaction.
`update_state_from` already opens an `IMMEDIATE` transaction in
autocommit for every target, so the deletion and the teardown jobs it
fails still commit together.
- `begin_resident_resume_with_policy` writes `edges::RESUME_BEGIN`
(`paused` -> `resuming`) inside its policy transaction. The edge now
refuses a non-paused row, so the separate state pre-check is gone; the
generation fence, the lock and the transaction are unchanged.
- Stop and Delete admission
(`update_state_superseding_activation_in_transaction`) cannot use an
exact edge without changing behavior. Admission re-admits a row that is
already in its target: a `stopping` row for a repeated Stop of a live
runtime, and a `deleting` row for a repeated Delete. No edge lists its
own target. The write therefore moves to `write_cleanup_admission_state`,
a private writer for only those two targets. It writes over every state
the generic guards admit, as admission always has, using
`update_state_from`'s statement and guards. `update_state`'s guard
warnings move with it; their text and fields are unchanged, but the log
target is now `db::state_write`. `STOP_ADMISSION` and `DELETE_ADMISSION`
keep listing the transitions it makes. One behavior change: a row whose
state label this binary cannot parse, reachable only after a downgrade or
with a corrupt row, now fails Stop admission instead of being overwritten
with `stopping`. Delete already failed earlier, in `get_sandbox`.
`update_state`, `SandboxStateWriteOutcome`, the legacy
`require_state_write` and `CleanupCompletionError::StateWriteNotApplied`
are deleted. The guard types collapse: `db::StateWriteGuard { Quarantined,
CleanupOwned }`, `generic_state_write_refusal` returns
`Option<StateWriteGuard>`, and `StateWriteRefusal` is
`FromState | Guard(StateWriteGuard)`. That removes the unreachable
"guards never answer FromState" arm. `SandboxStateWriteNotApplied` becomes
`Guard(StateWriteGuard) | Missing` and renders the same client-visible text
byte for byte. `into_status`, checkpoint admission and cleanup admission
all render through it.
Tests:
- The `update_state` outcome matrix already ran through `update_state_from`
with every state expected, standalone and in a transaction (#388). Only
its `update_state` arm is dropped, so every guard cell stays pinned. The
guard tests in `db/tests.rs` now target production edges, and the #438
status tests keep their literal texts without the legacy comparison.
- New: admission's same-state write. Re-admitting a `stopping` or
`deleting` row clears the failure reason and another target's receipt and
advances `updated_at`. For `deleting`, it also fails the active teardown
jobs. The Stop sweep now pins the full client text for `deleting` and
`quarantined` rows.
- New: Delete admission from every source state. It admits every plain row,
a `deleting` row included, except a `suspending` row whose suspend has
not reached the cluster. Nothing pinned Delete re-admission before.
- Test-only callers use an edge where production has one, `force_state`
where only the state matters, and an explicit-source
`update_state_from_states` for the two `started` -> `restoring` seeds.
- The five copies of the Stop fixture (superseding write, `mark_stopped`
intent, row-intent completion) are now `admit_stop`, `complete_stop` and
`stop` in `db/tests/state_write_fixtures.rs`.
Review comments from the stack's approval, folded in:
- `update_state_from_best_effort` logs a missing row with its own message,
instead of "superseded".
- `require_state_write_from` takes `context: &str`, as `into_status` does.
- Pause's comment on `mark_quarantined_best_effort` now says it also
answers false on a poisoned connection or a database error.
- A refused startup recovery write names the edge, the found state and the
refusal, instead of printing the outcome's `Debug` form. A missing row
has its own error.
- Auto-lifecycle Stop gains a case for a Delete admitted mid-write: a
`deleting` row with a current `delete_row` intent, which the
cleanup-ownership guard refuses and the Stop skips.
- Suspend's guest-prep rollback refusal and auto-pause refusal still answer
`Internal`. Both are unreachable under suspend's fence and locks, and a
comment now says so.
- Warm start's doc says why a guarded `suspended` row stays `Aborted`, and
that answering a transient row with `FailedPrecondition` is deliberate.
- Checkpoint admission renders through `SandboxStateWriteNotApplied`, not
the legacy outcome. A missing-row case pins its text alongside the
existing guard literals.
The sbx-mgr AGENTS.md writer rule is updated, and the archcheck cap for
`db/state_transitions.rs` is lowered to its new size.
ac-pplx
added a commit
that referenced
this pull request
Oct 14, 2026
Every write of sbx-mgr's node-local SQLite `sandboxes.state` now goes
through `update_state_from` with an explicit edge, or through a named
specialized writer. Nothing here touches the control-plane Postgres
`public.sandboxes.lifecycle`.
The three remaining db-internal `update_state` callers:
- The `deleting` re-entry inside `update_state` had no production caller:
Stop and Delete admission always write inside their own transaction.
`update_state_from` already opens an `IMMEDIATE` transaction in
autocommit for every target, so the deletion and the teardown jobs it
fails still commit together.
- `begin_resident_resume_with_policy` writes `edges::RESUME_BEGIN`
(`paused` -> `resuming`) inside its policy transaction. The edge now
refuses a non-paused row, so the separate state pre-check is gone; the
generation fence, the lock and the transaction are unchanged.
- Stop and Delete admission
(`update_state_superseding_activation_in_transaction`) cannot use an
exact edge without changing behavior. Admission re-admits a row that is
already in its target: a `stopping` row for a repeated Stop of a live
runtime, and a `deleting` row for a repeated Delete. No edge lists its
own target. The write therefore moves to `write_cleanup_admission_state`,
a private writer for only those two targets. It writes over every state
the generic guards admit, as admission always has, using
`update_state_from`'s statement and guards. `update_state`'s guard
warnings move with it; their text and fields are unchanged, but the log
target is now `db::state_write`. `STOP_ADMISSION` and `DELETE_ADMISSION`
keep listing the transitions it makes. One behavior change: a row whose
state label this binary cannot parse, reachable only after a downgrade or
with a corrupt row, now fails Stop admission instead of being overwritten
with `stopping`. Delete already failed earlier, in `get_sandbox`.
`update_state`, `SandboxStateWriteOutcome`, the legacy
`require_state_write` and `CleanupCompletionError::StateWriteNotApplied`
are deleted. The guard types collapse: `db::StateWriteGuard { Quarantined,
CleanupOwned }`, `generic_state_write_refusal` returns
`Option<StateWriteGuard>`, and `StateWriteRefusal` is
`FromState | Guard(StateWriteGuard)`. That removes the unreachable
"guards never answer FromState" arm. `SandboxStateWriteNotApplied` becomes
`Guard(StateWriteGuard) | Missing` and renders the same client-visible text
byte for byte. `into_status`, checkpoint admission and cleanup admission
all render through it.
Tests:
- The `update_state` outcome matrix already ran through `update_state_from`
with every state expected, standalone and in a transaction (#388). Only
its `update_state` arm is dropped, so every guard cell stays pinned. The
guard tests in `db/tests.rs` now target production edges, and the #438
status tests keep their literal texts without the legacy comparison.
- New: admission's same-state write. Re-admitting a `stopping` or
`deleting` row clears the failure reason and another target's receipt and
advances `updated_at`. For `deleting`, it also fails the active teardown
jobs. The Stop sweep now pins the full client text for `deleting` and
`quarantined` rows.
- New: Delete admission from every source state. It admits every plain row,
a `deleting` row included, except a `suspending` row whose suspend has
not reached the cluster. Nothing pinned Delete re-admission before.
- Test-only callers use an edge where production has one, `force_state`
where only the state matters, and an explicit-source
`update_state_from_states` for the two `started` -> `restoring` seeds.
- The five copies of the Stop fixture (superseding write, `mark_stopped`
intent, row-intent completion) are now `admit_stop`, `complete_stop` and
`stop` in `db/tests/state_write_fixtures.rs`.
Review comments from the stack's approval, folded in:
- `update_state_from_best_effort` logs a missing row with its own message,
instead of "superseded".
- `require_state_write_from` takes `context: &str`, as `into_status` does.
- Pause's comment on `mark_quarantined_best_effort` now says it also
answers false on a poisoned connection or a database error.
- A refused startup recovery write names the edge, the found state and the
refusal, instead of printing the outcome's `Debug` form. A missing row
has its own error.
- Auto-lifecycle Stop gains a case for a Delete admitted mid-write: a
`deleting` row with a current `delete_row` intent, which the
cleanup-ownership guard refuses and the Stop skips.
- Suspend's guest-prep rollback refusal and auto-pause refusal still answer
`Internal`. Both are unreachable under suspend's fence and locks, and a
comment now says so.
- Warm start's doc says why a guarded `suspended` row stays `Aborted`, and
that answering a transient row with `FailedPrecondition` is deliberate.
- Checkpoint admission renders through `SandboxStateWriteNotApplied`, not
the legacy outcome. A missing-row case pins its text alongside the
existing guard literals.
The sbx-mgr AGENTS.md writer rule is updated, and the archcheck cap for
`db/state_transitions.rs` is lowered to its new size.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Stacked on #372 (the PR 0 characterization tests). Base:
ac/sbxmgr-state-write-pr0-tests. Review #372 first; this PR's diff is one commit on top of it.All of this concerns sbx-manager's node-local SQLite
sandboxes.state, not the control-plane Postgrespublic.sandboxes.lifecycle.Coordination. The D2 Stop/Kill change (#396) was closed, so nothing touches
state.rsor this allowlist's Stop entries. This PR does not editstate.rs.What merging does
Adds
db/state_write.rs.StateSet: a bitset over the 18 lifecycle states.EMPTY,ALL,of,with,without,contains,is_empty,iter.Edge/EdgeKind/EDGES: 73 data-only edge constants, each with afrom: StateSet, ato: SandboxState, akindand a doc comment naming its writer functions and callers. Fields are private.SandboxState::can_transition_tostays the RPC-admission machine (D1).update_state_from(conn, id, edge: &'static Edge) -> StateWriteOutcome(plan §4.1:&edges::PAUSE_ROLLBACK). A call site names a table edge and cannot invent one. It returns one of:Applied { from }Refused { found, refusal: FromState | Quarantined | CleanupOwned }MissingIts behavior:
KnownRaceedges and inserts withInvalidQuery, since a known race disappears when its caller is converted. Targetsupdate_staterejects (quarantined,failed) areInvalidQuerytoo.update_state's quarantine and cleanup-ownership guards (§1.4) throughgeneric_state_write_refusal. That function now lives instate_write.rsand returnsStateWriteRefusaldirectly;update_statecalls it with no behavior change.AND state IN (<edge.from>)in theUPDATEitself.update_staterefuses is refused for the same reason. Converting anupdate_statecaller (PRs 2, 3, 5) therefore changes only rows outside the edge's from-set.starting/restoringrows with a current activation episode answerCleanupOwned. PR 4 must branch onedge.from().contains(found), not on the refusal tag.UPDATEmatches no row although the row was read in a source state, it returnsStatementChangedRows(0), notFromState(plan §4).update_state's: it clearsfailure_reason, the restore columns and non-matching receipts, and adeletingwrite fails active teardown jobs.IMMEDIATEtransaction, so the guard reads, the write and the job failure commit together. It never nests one.#[cfg(test)]update_state_from_states(conn, id, from, to)lets tests sweep arbitrary from-sets.Updates the sbx-mgr
AGENTS.mdwriter rule. A new transition callsupdate_state_fromwith its edge, or adds a specialized CAS writer; no newupdate_statecallers. Record the edge insrc/db/state_write.rs, and add any new disagreement withcan_transition_to, with a reason, toALLOWLISTinsrc/db/tests/state_write_table.rs. Nothing checks the table against the writers; the&'static Edgeargument does that at call sites.Edge inventory
The list was built from the code at this head: every
SET state/INSERT INTO sandboxesand everyupdate_state/update_state_best_effort/superseding caller (24 call sites), not from the plan's §2 table. These rules shaped it:update_statecaller's edge lists the state(s) the caller relies on. Its stale-state overwrites are separateKnownRaceedges.publish_stopped_artifactoverstopped,record_cleanup_episode_failureovercleanup_failed) and row deletion are not edges. A test enforces that no edge lists its own target as a source.mark_quarantined,mark_starting_quarantined,mark_restoring_quarantined,begin_activation,mark_stopping_superseding_activationandmark_deleting_superseding_activation.Request (14)
insert_sandbox_with_cleanup_episode_and_route_evidence(create, pool clone),transfer::reserve_importbegin_activation_with_route_evidence(ColdStart)begin_activation_with_route_evidence(Restore)fromstart_from_checkpointtry_start_from_warm_snapshot:update_state(Suspended)insert_restoring_sandbox_row_from_snapshot→restore_adopt_orphan_or_insert_sandbox_in_transactionpause, suspend auto-pause:update_state(Pausing)begin_resident_resume_with_policy;resume_after_wake_lock(no policy):update_state(Resuming)do_checkpoint,create_template_base_snapshot:update_state(Checkpointing)begin_suspend,begin_teardown_suspendadmit_lifecycle_cleanup_fenced→update_state_superseding_activation_in_transaction(Stop, Kill, auto-lifecycle Stop). Stop resolves its target only whilefc.pidis live, and admission does not re-check it. Socreated/stopped/suspendedare reached only in the gate-to-fence window or with a leaked runtime, andcreatingnever is. Kill takes only started/pausedrun_due_auto_lifecycle:update_state(Stopped); bypasses every RPC (kept as Request, and its allowlist reason says so)admit_lifecycle_cleanup_fenced→ superseding writer (Delete, auto-lifecycle Delete)claim_*_sandbox_for_evictionwritersrearm_prepared_clone_disposal(Delete of a quarantined pool clone)Completion (16)
create_with_id:update_state(Created)create_with_idbase-snapshot mode:update_state(Suspended)complete_pool_clone,complete_importcomplete_importmark_start_succeededmark_restore_succeededpublish_restored_suspended_artifact,publish_restored_suspended_without_artifactpublish_restored_disk_as_newpause, suspend auto-pause:update_state(Paused)restore_resident_resume_state(Started)→rollback_activation_staterestore_checkpoint_state(finish_checkpoint_success); template snapshotupdate_state(Paused)complete_row_intent_cleanup_for_job(MarkSuspended)update_state(Suspended)complete_row_intent_cleanup_for_job(MarkStopped),publish_stopped_artifactCompensation (9)
rollback_starting_episoderollback_restoring_episodepauseand suspend auto-pauseupdate_state(Started);update_state_best_effort(Started)(connect failure,pause_with_reconciliation)restore_resident_resume_state(Paused)restore_checkpoint_state_best_effort; template snapshotupdate_state_best_effort(Paused)rollback_suspending_to_paused(suspend snapshot path)Recovery (28)
request_cleanup_episode(activation cleanup guard, startup, cleanup recovery);created/suspendedwhen create's guard fires after its own writemark_cleanup_required,mark_cleanup_required_if_no_intent[_in_state]pin_suspended_artifact_for_cleanupmark_cleanup_failed,record_cleanup_episode_failurecomplete_cleanup_episode_for_job,complete_row_intent_cleanup_for_job→apply_cleanup_state;publish_stopped_artifactquarantine_cleanup_episode_for_job;apply_cleanup_state(Quarantine, or MarkSuspended without artifact)mark_quarantined_best_effort,mark_quarantined_from_statemark_suspending_quarantined; row-intent MarkSuspended without artifactsettle_startup_resident(Quarantined)insert_quarantined_sandbox_in_transactionrecover_pause_failed_quarantineadopt_quarantine_cleanup_episoderestore_adopt_orphan_or_insert_sandbox_in_transaction(conflict branch)recover_interrupted_create:update_state(Created)recover_unlaunched_start:update_staterecover_interrupted_pause_statesettle_startup_resident(Started)settle_startup_resident(Paused)rollback_suspending_to_paused(startup suspend recovery)recover_interrupted_startup_states:update_state(Suspended);publish_orphan_quarantine_recovered_artifactinsert_sandbox,insert_recovered_suspended_sandbox_in_transaction,restore_adopt_orphan_or_insert_sandbox_in_transactionKnownRace (6)
Start { from_warm }rejects onlyfailedbeforetry_start_from_warm_snapshot, so the reach is every state except failed and suspended that the generic guards admit. No concurrency is needed. Rows with a current activation episode answerCleanupOwnedFinding 4 (create completion over
stopping) is unreachable. Stop requires a live runtime, a row stillcreatingorcreatedduring its create has none, and Delete'sdeletingrow is refused by the generic guard. Its two edges are dropped.What the counts say
Convention. An edge enters or leaves a transient state when its target or any of its sources is one of the nine
SandboxState::is_transientstates. A pair qualifies when either end is transient.In-flight edges. 55 of the 73 edges qualify: 9 of 9 compensations, 14 of 16 completions, 18 recoveries, 11 requests and 3 of 6 known races. At pair level, 108 of 138 pairs qualify. Under the stricter convention (target transient, or every source transient), 50 edges qualify, including 2 known races.
Episode-bound edges. 17 edges create, require or complete an activation cleanup episode:
Five more cleanup edges (CLEANUP_FAILURE, CLEANUP_COMPLETE_* ×3, CLEANUP_QUARANTINE) serve both episodes and row intents.
Recovery is the largest kind. 28 edges: 47 pairs plus 4 inserts.
Targets.
failedis the only state no writer targets. It is only a source, of Stop and Delete admission. Every other state is targeted.Admission-machine edges no writer implements.
can_transition_toallows 56 state-changing pairs, and 8 have no writer:started → suspending(suspend pauses first), and→ quarantinedfromcreating,created,starting,stopped,deleting,suspendedandrestoring.Admission-machine edges only non-request writers implement. 21 allowed pairs are written only by non-request edges:
27 allowed pairs are written by a request edge.
Written pairs the machine forbids. The table writes 56 pairs that
can_transition_toforbids.creating → deleting.created/stopped → suspended.suspended → stopped.stopped → started/paused/resuming, and→ suspendedfromstarted,stopping,pausing,paused,resumingandcheckpointing.Differences from the plan's §2 table, found in code
settle_startup_residentdoes not writepaused → startedorstarted → pausedat this head. It writesresuming → started/paused,checkpointing → startedand quarantines.quarantined → quarantined(reconfirm) has no production caller: every production quarantine writer passes an expected non-quarantined state.Parity allowlist
db/tests/state_write_table.rsclassifies every state-changing pair on which the edge table andcan_transition_todisagree. It asserts that the classification equals the checked-inALLOWLISTof 85 entries, each with a one-line reason:RequestForbiddencan_transition_toforbids itWriteForbiddenAllowedNotRequestedAllowedUnwrittenThe Stop entries carry the runtime-gate reasons above;
suspended → stoppedsays that auto-lifecycle bypasses every RPC.A new writer edge, a changed from-set, or a changed state machine fails CI until the allowlist records it. On failure, the test prints the unlisted entries in allowlist syntax and the stale ones.
Tests
Equivalence (
update_state_from.rs).update_state_matrix.rsnow runs its sweeps over aWriter.update_state_from_states(ALL), both standalone and inside a caller's transaction, gives the same outcome and the same diff asupdate_stateon every cell. That covers all 18×18 pairs, the five cleanup shapes, both receipt kinds, the quarantine episode and the missing row.Applied { from }andRefused { found }must name the seeded state.From-set refusal. It covers every
(from, to)pair on a plain row, with both receipt kinds, standalone and in a transaction, using four from-sets:Refused { found: from, FromState }with every row unchanged;Applied { from }withupdate_state's exact diff.Guard refusals must not depend on the from-set. Rejected targets (
quarantined,failed) must returnInvalidQuerywith nothing changed.Edge binding. For every writable edge × every source state on a plain row,
update_state_from(&edge)applies exactly when the edge's from-set contains the row's state and the generic guards (sbx-mgr: characterize the state writers before the state-write helper (tests only) #372's PLAIN grid) admit it, with the exact row diff. Otherwise it refuses with every row unchanged.Refused edges. Every
KnownRace, insert and quarantine/failed-target edge (16 in all) isInvalidQuerywith every row unchanged. The 6 known races are tried over a source state the generic guards admit. The 6 inserts have no source, and every PLAIN cell for the 4 quarantine-target edges isX, so those 10 are tried overstarted.Invariant. A temp
RAISE(IGNORE)trigger stands in for SQL that drifted from the from-set: a source-state row whose write matches nothing returnsStatementChangedRows(0), row unchanged.Autocommit atomicity. A temp trigger makes the
jobsupdate fail. A standalonedeletingwrite then returns the error, and the state write is rolled back too.Table. The parity test above, plus a test that no edge lists its own target as a source.
Mutation proof
The mutations ran one at a time, each from a single backup, and every SQL
?Nstayed bound. Each was restored and touched before the next.AND ?4 IS NOT NULL)delete_row-intent guardIMMEDIATEeven inside a caller's transactionfail_activeondeletingsuspended → stoppedallowlist entry(Created, Paused)tocan_transition_toStateSet::ALLinstead ofedge.from()KnownRaceedgesFromStateIMMEDIATEwrapperGuards 2–5 live in the shared
generic_state_write_refusal, so each mutation fails both writers' tests.Instructions for the converting PRs (2–5)
KnownRaceedge, a pair that is still written by another edge keeps its class, so the test stays green while its reason goes stale. Re-read the mixed reasons that name a race:creating,starting,restoring,cleanup_requiredandcleanup_failed→suspended(they name the warm start);stopping → suspended(warm start and process-snapshot rollback);stopped → paused(pause and template-snapshot completion).known_race_insert_and_rejected_target_edges_are_refusedasserts that at least oneKnownRaceedge exists. It fails when the last one is deleted, which forces the PR that does so to retireEdgeKind::KnownRaceand the test's race branch.Decisions made without an operator
Requestincludes node-initiated admissions: the checkpoint scheduler, auto-lifecycle and eviction act as clients.to. When the same pair serves two purposes, each gets its own edge, e.g.CHECKPOINT_COMPLETE_PAUSED(Completion) andCHECKPOINT_ROLLBACK_PAUSED(Compensation).update_state's reason (see Precedence above).state IN (SELECT value FROM json_each(?4)). This has precedent incleanup_jobs.rs.update_state's SQL quarantine predicate is not duplicated: the shared Rust guard answers first, and a duplicate predicate would be untestable.WARM_START_UNVALIDATEDlists the full reach. I tracedstart_sandbox_with_authority's early returns for an empty-checkpointfrom_warmStart withrestore_artifact_as_new = None: target lookup, interrupted-restore action, the suspend-checkpoint and restore-as-new branches, and the non-empty-checkpoint refusal. None of them gates on state beforetry_start_from_warm_snapshotexcept thefailedcheck. Withrestore_artifact_as_newset,interrupted_restore_actionbranches onrestoring, and the reattach branch can answer a livestartedsandbox before warm staging. The 12-state inventory, which covers theNonecase, is unchanged.AUTO_LIFECYCLE_STOP_SUSPENDEDstaysRequest, matching the module's documented convention that node-initiated admissions are requests. Its allowlist reason says that it bypasses every RPC.#[rustfmt::skip], which has precedent inroutes/public_docs/compatibility/tests.rs, so it stays one line per entry.Validation
Review rounds 1 and 2 are folded into this one commit. The final round changed only a test comment and this body; that pass re-ran fmt, workspace clippy (0 warnings) and one
SPACE_CI_REQUIRE_TEST_DEPS=1sbx-mgr nextest run (1622/1622, 0 dependency skips).cargo fmt --all -- --check,scripts/check-rust-error-policy.shcargo clippy --workspace --locked --all-targets --all-features -- -D warningscargo archstate_transitions.rswas 1163 code lines at 8f12bd6 (already over its 1156 cap), 1182 at d9162b7, and is 1140 at this head; the cap is lowered from 1156 to 1140.state_write.rsis under the 1000-line limitpre-commit(Gitleaks) on the staged changeSPACE_CI_REQUIRE_TEST_DEPS=1 cargo nextest run --locked -p pplx-space-sbx-mgr --retries 0× 3#[ignore]privilegedtransfer_directory::tests::distinct_service_uids_can_exchange_downloads, unrelated🤖 Generated with Claude Code