You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
metrics: split worker lifecycle populations by runtime version - #1581
Extend the existing sandbox.worker.lifecycle.count gauge with runtime and runtime_version so worker-local lifecycle populations can be broken down by their recorded execution runtime. This does not add a competing population metric.
The SQLite census groups durable rows using their persisted runtime identity, normalizes runtime aliases, and preserves rows with absent or malformed metadata as unknown rather than assuming a version. The sampler tracks the complete state/runtime/version bucket when emitting a final zero for a disappearing population. No sandbox, template, or execution identifier becomes a metric dimension.
Validation
Affected-package cargo check --all-targets and cargo clippy --all-targets -- -D warnings passed for pplx-space-sbx-mgr-store, pplx-space-sbx-mgr, and pplx-space-metrics.
All 673 storage library tests and 11 metrics library tests passed. Storage coverage exercises mixed runtime versions, aliases, suspended rows, malformed and missing metadata, count preservation, and removal of a runtime bucket.
The sandbox-manager library suite reported 1700 passed, 2 failed, and 1 ignored. Both failures also reproduce on the unchanged baseline 3458cc6ac7eebcddab6e68a5dc47571eb44d3db6: a_restart_suspended_teardown_capture_never_registers_unfenced and lapsed_supersession_registers_the_capture_again. A baseline run of the retired-teardown group reported 53 passed and those same 2 failures. The full suite is not green.
Formatting, whitespace checks, and the Gitleaks pre-commit scan passed.
Live metric ingestion and deployment have not been validated.
Severity and risk
Severity is low: the existing total remains available, but it cannot currently distinguish runtime versions. This changes observability, not sandbox placement or execution policy.
Risk is increased series cardinality and incorrect grouping if persisted runtime metadata is not decoded correctly. Legacy rows remain explicitly unknown, and summing runtime buckets preserves the per-state population. The census represents locally recorded lifecycle state, not an independent VM process-liveness check.
Rollout
Metric backends with explicit tag-retention allowlists must retain runtime and runtime_version alongside state. During a mixed-version worker rollout, old emitters lack these dimensions, so complete version coverage requires updating the intended worker cohort. Reverting the emitter restores the previous state-only metric.
AI assistance: Perplexity Computer contributed the code, tests, documentation, validation, and this description.
Why: The lifecycle metric now depends on a changed SQLite population query. Its metadata format matches the writer, but the query change warrants validation before merge under the repository’s risk guidance.
Commit: 74a750f139e4.
No actionable comments were generated in the recent review.
Walkthrough
Sandbox lifecycle counts now include optional runtime kind and runtime version. The storage query derives runtime identity from execution metadata, falls back to the stored runtime kind, and retains rows with unavailable metadata. The metrics pipeline emits runtime and version dimensions, using "unknown" for missing values. Tests cover grouping, metadata cases, and deletion. The observability documentation describes the metric and its sampling behavior.
Possibly related PRs
perplexityai/SPACE#1555: Changes the lifecycle state column used by the count query that this change extends with runtime and version grouping.
Merge Risk: Moderate · up to 74a75
The lifecycle metric now depends on a changed SQLite population query. Its metadata format matches the writer, but the query change warrants validation before merge under the repository’s risk guidance.
Architecture risk: Low · up to 74a75
The change affects 2 systems.
Changed systems:crates, docs
Architecture concerns
No architecture-level concerns identified.
Review details
Systems and components
observed — crates (service) was modified; 7 changed files map to changed impact.
observed — docs (service) was modified; 1 changed file maps to changed impact.
Before / after behavior
observed — Modified behavior in crates/l30_storage/sbx/mgr/store/src/db.rs: The db module’s common-core type imports now include RuntimeIdentity and RuntimeVersion.
observed — Modified behavior in crates/l40_api/sbx/mgr/server/src/lifecycle_metrics.rs: Imports RuntimeKind and RuntimeVersion for lifecycle bucket metadata.
observed — Modified behavior in crates/l40_api/sbx/mgr/server/src/lifecycle_metrics.rs: LifecycleBucket adds optional runtime kind and runtime version fields, populated from each SandboxLifecycleCount; buckets therefore distinguish lifecycle counts by state and runtime metadata.
observed — Modified behavior in crates/l40_api/sbx/mgr/server/src/lifecycle_metrics.rs: Lifecycle metric emission now passes the bucket’s optional runtime kind and runtime version, in addition to count and state.
Pre-merge checks: 5
Check name
Status
Explanation
Title check
The title describes the lifecycle metric change and meets the specified format.
Description Follows The Pr Template
The description includes a populated Summary, Validation, and Severity and risk section. It also includes an AI assistance line that names the tool and its contributions.
Bug Fixes Name A Regression Test
This PR adds runtime dimensions to an existing metric. The base census grouped only by state. The metadata decoding fix corrects code introduced within this PR, so the PR is not an application bug fix…
Weakened Safeguards Are Justified
The diff does not remove or weaken a test, assertion, lint, CI check, or repository guard. Existing test assertions retain their expected counts and include the new fields.
Public-Safe Content
The reviewed diff, title, and description contain no content that matches the specified failure conditions. The added query is an example, not production output.
Full details: Bug Fixes Name A Regression Test
Explanation
This PR adds runtime dimensions to an existing metric. The base census grouped only by state. The metadata decoding fix corrects code introduced within this PR, so the PR is not an application bug fix against the base.
Recent review info
Automatic approval policy
Commit: 74a750f139e4.
Setting
Effective value
Source
Automatic approvals
Enabled
Global configuration
Merge risk
Low
Override 2
Security risk
Not set
Global configuration
Matching overrides (numbered as in the resolved configuration):
Any matching disabled override withholds approval, regardless of risk. Otherwise, the strictest explicitly set limit wins for each score. Limits omitted by all matching overrides inherit the global configuration.
Not set means no limit was configured after inheritance. Use @coderabbitai configuration to inspect the resolved settings and their sources.
Included review availability: This review used your included allowance. Your plan provides up to 12 included reviews per hour; 6 remain after this review.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Extend the existing
sandbox.worker.lifecycle.countgauge withruntimeandruntime_versionso worker-local lifecycle populations can be broken down by their recorded execution runtime. This does not add a competing population metric.The SQLite census groups durable rows using their persisted runtime identity, normalizes runtime aliases, and preserves rows with absent or malformed metadata as unknown rather than assuming a version. The sampler tracks the complete state/runtime/version bucket when emitting a final zero for a disappearing population. No sandbox, template, or execution identifier becomes a metric dimension.
Validation
cargo check --all-targetsandcargo clippy --all-targets -- -D warningspassed forpplx-space-sbx-mgr-store,pplx-space-sbx-mgr, andpplx-space-metrics.3458cc6ac7eebcddab6e68a5dc47571eb44d3db6:a_restart_suspended_teardown_capture_never_registers_unfencedandlapsed_supersession_registers_the_capture_again. A baseline run of the retired-teardown group reported 53 passed and those same 2 failures. The full suite is not green.Severity and risk
Severity is low: the existing total remains available, but it cannot currently distinguish runtime versions. This changes observability, not sandbox placement or execution policy.
Risk is increased series cardinality and incorrect grouping if persisted runtime metadata is not decoded correctly. Legacy rows remain explicitly unknown, and summing runtime buckets preserves the per-state population. The census represents locally recorded lifecycle state, not an independent VM process-liveness check.
Rollout
Metric backends with explicit tag-retention allowlists must retain
runtimeandruntime_versionalongsidestate. During a mixed-version worker rollout, old emitters lack these dimensions, so complete version coverage requires updating the intended worker cohort. Reverting the emitter restores the previous state-only metric.AI assistance: Perplexity Computer contributed the code, tests, documentation, validation, and this description.