Skip to content

sbx-mgr: make encrypted suspend capture resumable - #1761

Draft
NancyKwongPerplexity wants to merge 1 commit into
mainfrom
nancy/encrypted-suspend-resumable-capture
Draft

NancyKwongPerplexity wants to merge 1 commit into
mainfrom
nancy/encrypted-suspend-resumable-capture

Conversation

@NancyKwongPerplexity

Copy link
Copy Markdown
Collaborator

Port of #837 (by @mabdelraouf-pplx) onto current main. This is the durable fix for the encrypted-suspend quarantine race, and the companion to #1760.

Change (from #837)

  • Adds the node-manager CreateEncryptedSuspendSnapshot RPC. It captures an encrypted runtime and stages its ciphertext as one operation, keyed by the capture's local artifact id.
  • Each step records its outcome in the staging directory. A retry after a lost reply resumes staging a confirmed capture instead of rolling back to paused, and it never captures over a capture whose outcome is unknown.
  • sbx-mgr keeps the row and staging for a capture that node-manager durably owns, so recovery can complete it.

Rebase notes

  • crates/libs/* → crates/l40_api/*; pplx_space_metrics::{byok,snapshot} → crate::metrics::*.
  • The Cloud Hypervisor encrypted capture now passes the recorded boot source (snapshot_boot_source), because main added that parameter.
  • Kept main's btrfs_send_encode_falls_back_to_full. Dropped failed_encrypted_capture_may_release_volume, as sbx-mgr: make encrypted suspend capture resumable #837 did.
  • Tests now compare SandboxStatus. Cleanup-required expectations are SandboxStatus::owned(Teardown::Suspending, CleanupState::Required).

Testing

  • cargo check -p pplx-space-node-mgr -p pplx-space-sbx-mgr --tests is clean.
  • The node-mgr encrypt tests pass (36).
  • I couldn't build the sbx-mgr test binary locally because it ran out of memory, so this relies on CI.
  • Not validated on Kind/Firecracker. Left as a draft until CI and an e2e suspend/resume pass.

Supersedes #837.

Created with Perplexity

Port of #837 (by @mabdelraouf-pplx) onto current main.

Adds the node-manager CreateEncryptedSuspendSnapshot RPC, which captures
an encrypted runtime and stages its ciphertext as one operation keyed by
the capture's local artifact id. Each step records its outcome in the
staging directory, so a retry after a lost reply (e.g. the 120 s gRPC
request timeout firing while the detached capture keeps running) resumes
staging a confirmed capture instead of rolling the sandbox back to paused
and later quarantining it.

Rebase notes: paths moved from crates/libs to crates/l40_api, metrics
moved to crate::metrics, the Cloud Hypervisor capture now threads the
recorded boot source, and test row states use SandboxStatus.

Co-authored-by: Perplexity <noreply@perplexity.ai>
@coderabbitai

coderabbitai Bot commented Nov 12, 2026

Copy link
Copy Markdown

Important

Draft PR not reviewed

Draft PRs are not automatically reviewed by default.

  • Trigger a manual review

To automatically review draft PRs, update your CodeRabbit configuration:

reviews:
  auto_review:
    drafts: true

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant