Repository navigation
sbx-mgr: retire the generic update_state - #498
Merged
Merged
Conversation
ac-pplx
force-pushed
the
ac/sbxmgr-sw-pr6-retire-update-state
branch
from
October 14, 2026 22:19
ab56b47 to
3434db0
Compare
Every write of sbx-mgr's node-local SQLite `sandboxes.state` now goes
through `update_state_from` with an explicit edge, or through a named
specialized writer. Nothing here touches the control-plane Postgres
`public.sandboxes.lifecycle`.
The three remaining db-internal `update_state` callers:
- The `deleting` re-entry inside `update_state` had no production caller:
Stop and Delete admission always write inside their own transaction.
`update_state_from` already opens an `IMMEDIATE` transaction in
autocommit for every target, so the deletion and the teardown jobs it
fails still commit together.
- `begin_resident_resume_with_policy` writes `edges::RESUME_BEGIN`
(`paused` -> `resuming`) inside its policy transaction. The edge now
refuses a non-paused row, so the separate state pre-check is gone; the
generation fence, the lock and the transaction are unchanged.
- Stop and Delete admission
(`update_state_superseding_activation_in_transaction`) cannot use an
exact edge without changing behavior. Admission re-admits a row that is
already in its target: a `stopping` row for a repeated Stop of a live
runtime, and a `deleting` row for a repeated Delete. No edge lists its
own target. The write therefore moves to `write_cleanup_admission_state`,
a private writer for only those two targets. It writes over every state
the generic guards admit, as admission always has, using
`update_state_from`'s statement and guards. `update_state`'s guard
warnings move with it; their text and fields are unchanged, but the log
target is now `db::state_write`. `STOP_ADMISSION` and `DELETE_ADMISSION`
keep listing the transitions it makes. One behavior change: a row whose
state label this binary cannot parse, reachable only after a downgrade or
with a corrupt row, now fails Stop admission instead of being overwritten
with `stopping`. Delete already failed earlier, in `get_sandbox`.
`update_state`, `SandboxStateWriteOutcome`, the legacy
`require_state_write` and `CleanupCompletionError::StateWriteNotApplied`
are deleted. The guard types collapse: `db::StateWriteGuard { Quarantined,
CleanupOwned }`, `generic_state_write_refusal` returns
`Option<StateWriteGuard>`, and `StateWriteRefusal` is
`FromState | Guard(StateWriteGuard)`. That removes the unreachable
"guards never answer FromState" arm. `SandboxStateWriteNotApplied` becomes
`Guard(StateWriteGuard) | Missing` and renders the same client-visible text
byte for byte. `into_status`, checkpoint admission and cleanup admission
all render through it.
Tests:
- The `update_state` outcome matrix already ran through `update_state_from`
with every state expected, standalone and in a transaction (#388). Only
its `update_state` arm is dropped, so every guard cell stays pinned. The
guard tests in `db/tests.rs` now target production edges, and the #438
status tests keep their literal texts without the legacy comparison.
- New: admission's same-state write. Re-admitting a `stopping` or
`deleting` row clears the failure reason and another target's receipt and
advances `updated_at`. For `deleting`, it also fails the active teardown
jobs. The Stop sweep now pins the full client text for `deleting` and
`quarantined` rows.
- New: Delete admission from every source state. It admits every plain row,
a `deleting` row included, except a `suspending` row whose suspend has
not reached the cluster. Nothing pinned Delete re-admission before.
- Test-only callers use an edge where production has one, `force_state`
where only the state matters, and an explicit-source
`update_state_from_states` for the two `started` -> `restoring` seeds.
- The five copies of the Stop fixture (superseding write, `mark_stopped`
intent, row-intent completion) are now `admit_stop`, `complete_stop` and
`stop` in `db/tests/state_write_fixtures.rs`.
Review comments from the stack's approval, folded in:
- `update_state_from_best_effort` logs a missing row with its own message,
instead of "superseded".
- `require_state_write_from` takes `context: &str`, as `into_status` does.
- Pause's comment on `mark_quarantined_best_effort` now says it also
answers false on a poisoned connection or a database error.
- A refused startup recovery write names the edge, the found state and the
refusal, instead of printing the outcome's `Debug` form. A missing row
has its own error.
- Auto-lifecycle Stop gains a case for a Delete admitted mid-write: a
`deleting` row with a current `delete_row` intent, which the
cleanup-ownership guard refuses and the Stop skips.
- Suspend's guest-prep rollback refusal and auto-pause refusal still answer
`Internal`. Both are unreachable under suspend's fence and locks, and a
comment now says so.
- Warm start's doc says why a guarded `suspended` row stays `Aborted`, and
that answering a transient row with `FailedPrecondition` is deliberate.
- Checkpoint admission renders through `SandboxStateWriteNotApplied`, not
the legacy outcome. A missing-row case pins its text alongside the
existing guard literals.
The sbx-mgr AGENTS.md writer rule is updated, and the archcheck cap for
`db/state_transitions.rs` is lowered to its new size.
ac-pplx
force-pushed
the
ac/sbxmgr-sw-pr6-retire-update-state
branch
from
October 14, 2026 22:30
3434db0 to
cd14056
Compare
ac-pplx
marked this pull request as ready for review
October 14, 2026 22:54
Collaborator
Review: 1 findingLow
Checked and not reported
Verified against |
the-gigi-pplx
approved these changes
Oct 15, 2026
the-gigi-pplx
left a comment
Collaborator
There was a problem hiding this comment.
Approved. 1 non-blocking finding in the review comment above.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Follows the merged sbx-mgr state-write stack (#438 → #418 → #413 → #416 → #421 → #417 → #414). This is PR 6 of the state-write plan (
reports/state-write-helper-plan.mdonac/state-write-helper-plan, §5 row 6).All state here is sbx-mgr's node-local SQLite
sandboxes.state. Nothing touches the control-plane Postgrespublic.sandboxes.lifecycle.What changes
Every persisted write of the node-local SQLite
sandboxes.statenow goes throughupdate_state_fromwith an explicit edge, or through a named specialized writer. The genericupdate_stateis deleted. So areSandboxStateWriteOutcome, the legacyrequire_state_write, andCleanupCompletionError::StateWriteNotApplied, which nothing constructed. Net production change: 9 files, +203 / −273.Stop and Delete admission is not on an edge. Owner rule: if (c) is not behavior-preserving with any exact edge, leave it on a private, clearly named non-generic helper and report why.
update_state_superseding_activation_in_transaction) legitimately re-admits a row already in its target:stoppingrow, for a repeated Stop of a live runtime (the sbx-mgr: characterize the state writers before the state-write helper (tests only) #372 characterization classifiesstoppingas reachable from the RPC);deletingrow, for a repeated Delete (the Delete RPC has no runtime gate).no_edge_lists_its_own_target_as_a_source). It also says a same-state write "changes no state and has no edge".write_cleanup_admission_state, a private writer forstoppinganddeletingonly. It writes over every state the generic guards admit, as admission always has. It usesupdate_state_from's statement, guards andIMMEDIATEhandling. The guard warnings areupdate_state's, moved into this writer:update_state_fromemits none.STOP_ADMISSIONandDELETE_ADMISSIONkeep listing the transitions it makes.Scope
deletingbranch insideupdate_stateupdate_state.update_state_fromalready opensIMMEDIATEin autocommit for every target, so the deletion and itsteardown_jobs::fail_activestill commit together (pinned bya_standalone_deletion_rolls_back_when_its_job_failure_does). Test callers that wrotedeletingnow useedges::DELETE_ADMISSION.begin_resident_resume_with_policyupdate_state_from(&tx, id, &edges::RESUME_BEGIN)(Request,{paused}→resuming) inside the sameIMMEDIATEpolicy transaction. The edge refuses a non-paused row, so the duplicatestate != Pausedpre-check is deleted. The generation fence, the transaction and theOk(false)on any refusal are unchanged. Edge doc updated. Log-only change:update_statewarned when a guard refused apausedrow, andupdate_state_fromis silent. That case is likely unreachable here.write_cleanup_admission_state, as described above. Lock, transaction, client messages and side columns are unchanged. The writer also returnsInvalidQueryfor a target other thanstoppingordeleting. That duplicates thedebug_assertstill inupdate_state_superseding_activation_in_transaction, only matters in release builds, and no production caller passes another target.update_staterg -P '(?<![\w])update_state\('finds nothing in any file. Of the #372 matrix, only theupdate_statewriter arm and its two tests are deleted. The same grids, row diffs and shapes already run throughupdate_state_fromwith every state expected, standalone and in a transaction (#388), so every guard cell stays pinned. Indb/tests.rs, the guard tests now target production edges. Thefailed-target assertion is dropped because the matrix'sXcolumn pins it. Seed-only test callers use an edge where production has one,force_statewhere only the state matters, and an explicit-sourceupdate_state_from_statesfor the twostarted→restoringseeds.SandboxStateWriteNotAppliedis nowGuard(StateWriteGuard) | Missing. ItsDisplayholds the three client-visible literals, byte for byte.into_status, checkpoint admission and cleanup admission render through it. In the #438 parity tests, the legacy comparison andupdate_state_outcomes_keep_their_statusesare dropped; the literal-text assertions stay.db::StateWriteGuard { Quarantined, CleanupOwned }.generic_state_write_refusalreturnsOption<StateWriteGuard>.StateWriteRefusal::{FromState, Guard(StateWriteGuard)}. Every match is wildcard-free, and the "guards never answerFromState" arm is gone.admit_stop,complete_stopandstopindb/tests/state_write_fixtures.rsreplace the copies inpause_resume/supersession_tests.rs,auth_lookup/resume_begin/tests.rs,create_support/state_write_tests.rs,grpc_state_snapshot/tests.rsandgrpc_spaced_process_snapshots/tests.rs.reconcile/tests.rs's blocking Stop (no intent) is a different fixture, so it stays.AGENTS.mdupdate_state_fromwith an explicit edge, or through a named specialized writer. A new edge needs a kind and a doc comment naming its writer. Map outcomes withrequire_state_write_from/state_write_result.delete_admission_answers_every_source_state. Delete re-admission of adeletingrow was unpinned: narrowing the Delete from-set bydeletingfailed no test.update_state_from_best_effortlogsMissingwith its own message ("found no sandbox row").require_state_write_fromtakescontext: &str, asinto_statusdoes.mark_quarantined_best_effortalso answers false on a poisoned connection or a DB error. No behavior change.StateWriteRefused { edge, found, refusal }, rendered explicitly. A missing row isStateWriteMissing { edge }. No{outcome:?}.delete_admitted_during_auto_lifecycle_stop_keeps_its_row: adeletingrow with a currentdelete_rowintent, which theCleanupOwnedguard refuses, is skipped and kept.Internal. No behavior change.write_warm_start_restageddoc says whyGuard { found: Suspended }staysAbortedwhileFromState { Suspended }succeeds. It also saysFailedPreconditionfor a transient row, versus cold Start'sUnavailable, is deliberate (plan §5).activation_supersession_retires_the_episode_before_the_generic_writenow re-admits a decoratedstoppingand a decorateddeletingrow: failure reason, astartedreceipt, backdated timestamps, and an active teardown job fordeleting. It asserts the whole-database diff.Unavailablemessage fordeleting(superseded_by_cleanup_episode) andquarantined(blocked_by_quarantine).update_state_matrix's writer adapter matches every outcome explicitly and asserts the seeded state separately.create_support.rs. The three test docs that namedupdate_statenow say "the generic guards' message" or "a same-state generic write".checkpoint_admission_of_a_missing_row_reports_itpinsmissing.cargo archdb/state_transitions.rscap lowered 1140 → 1018. No cap raised. My change shrinksservice/cleanup.rsby 2 lines; its advisory growth over the cap is already on main.Changes beyond the refactor
statelabel this binary cannot parse now fails Stop admission withFromSqlConversionFailure, where it used to be overwritten withstopping. Delete is unchanged: it already failed earlier, inget_sandbox. This is reachable only after a downgrade or with a corrupt row. No tolerance was added.pplx_space_sbx_mgr::db::state_writeinstead of…::db::state_transitions. The old target is not pinned.Out of scope, as instructed: finding 8's dead and test-only writers (including
mark_stopping_superseding_activation,mark_deleting_superseding_activationandupdate_state_superseding_activation), the unreachable cleanup-admission branches, and D2/D4. Side columns are untouched.Mutation proof
Each mutant was applied alone, run against the full sbx-mgr suite with
--retries 0, then reverted.RESUME_BEGINto all statesresident_policy_resume_is_generation_fenced_and_arms_only_after_thaw,policy_free_resume_writes_resuming_only_over_paused, and two edge-table testsRESUME_BEGINbypaused(empty)cleanup_intent_cannot_be_replaced_by_suspend_or_resident_completion,start_arms_access_idle_only_after_guest_readinessand an edge-table teststoppingstop_admission_answers_every_source_statedeletingdelete_admission_answers_every_source_state(new). Before that test existed, nothing failed.IMMEDIATEwrappera_standalone_deletion_rolls_back_when_its_job_failure_doesFromStatedelete_admitted_during_auto_lifecycle_stop_keeps_its_row,auto_lifecycle_stop_writes_only_over_a_still_suspended_rowif found == target { return Ok(Ok(())) })activation_supersession_retires_the_episode_before_the_generic_writeQuarantinedstop_admission_answers_every_source_stateValidation
cargo fmt --all --checkcargo clippy --workspace --all-targets --all-features -- -D warningscargo archSPACE_CI_REQUIRE_TEST_DEPS=1 cargo nextest run --locked -p pplx-space-sbx-mgr --retries 0×3 (pre-rebase head)#[ignore]d root-onlydistinct_service_uids_can_exchange_downloads.#[ignore]skip#[ignore]skip099710bgit_sha=099710b2e779, and the deployedspace-sbxmgr-svccontains strings only this change adds. Tilt was quiet (46 ok, 4 not applicable, none pending).smoke-kind-e2e.sh: PASS.smoke-kind-typed.sh allwithSPACE_SMOKE_KIND_ADMIN_TOKEN_FILEandSPACE_SMOKE_KIND_COLD_RECEIPTSset as CI does: ECW 78/78 and native 23/23, 0 failed, 0 inconclusive, 0 quarantined. Torn down withinfra/space/kind/scripts/teardown.sh.Kind ran on the pre-rebase commit. The rebase onto main applied without conflicts, and this change's diff is the same on both bases. The round-1 fixes touch only tests, comments and this text, so Kind was not rerun.
🤖 Generated with Claude Code