Repository navigation
api: bind the substrate value instead of writing it in SQL - #506
Conversation
877ebdd to
8891dcf
Compare
Control-plane Postgres `sandboxes.managed_by` and `nodes.managed_by` hold the substrate that owns a row, `'k8s'` or `'space'`. The api crate scoped and wrote that value as a raw literal in 22 production SQL statements and compared it as a string in two places. Those statements now bind `SubstrateKind` (`managed_by = $n` with `.bind(SubstrateKind::Kubernetes)` or `::Space`), each still one whole statement, and the comparisons parse the label and match the variant. `SubstrateKind` encodes as the column's text, so every statement matches and writes exactly the rows it did. No schema, wire format, metric or configuration changes. - Direct substrate (`direct.rs`, 12 statements): node selection for create, pooled adoption, pull-on-create, placement lookups, restore candidates and builder lookup bind `Space`. Only the SQL text and its binds change in this file. - Direct node registry (`routes/admin/node_maintenance.rs`, 4): node reactivation, insert, update and hard delete bind `Space`; the reactivation statement uses one parameter for its filter and its `SET managed_by`. - Kubernetes node inventory (`kubernetes/store/node_inventory.rs`, 3): demotion of unobserved nodes and the node upsert use one parameter for every use in the statement; the ownership re-check binds its own. - Public sandbox read and list (`routes/public/sandboxes.rs`): the failure-episode projection binds `Kubernetes` and `Space` as `$6` and `$7` in the shared select, so the by-id and page suffixes renumber their parameters from `$8`, with binds in the same order. - Template reconcile targets (`routes/admin/templates.rs`) and template delete inventory (`public_template_operations.rs`) bind `Space`. - `spaced.rs`'s dormant-wake fallback and the console's `/me` substrate (`session_auth.rs`) parse the label as `SubstrateKind` and match `Kubernetes`; any other value, including an unparseable one, behaves as before. Tests: each converted statement now has a Postgres-backed test in which a row of the statement's substrate is selected or written and an otherwise identical row of the other substrate is not. New modules: `direct/tests/substrate_scope.rs`, `k8s_store_postgres_tests/substrate_scope_tests.rs` (including a concurrent relabel the demotion UPDATE must re-check, committed once the writer's pid blocks it), `node_maintenance/tests/substrate_scope.rs`, `templates/tests/substrate_scope.rs` and `routes/public/sandboxes/tests/substrate_scope.rs`; the template delete inventory test gains a Kubernetes twin of its retired Direct node, `session_auth` gains a label table, and `kubernetes/tests/substrate_scope.rs` shows that only a `'k8s'` dormant row takes `control_runtime_client`'s wake fallback. Test fixtures keep writing literal `managed_by` values. Left as is: the Direct route identity's `substrate_kind` match in `direct.rs` (a Rust match on a column read in Direct-owned code outside the SQL statements), `DirectSubstrate::managed_by()`'s `"space"`, the Kubernetes adapter's literals (`substrate_adapter.rs`) and the shared `routable_execution_or_non_k8s` fragment. Converted statements' `/*module_path:line*/` locators, sqlx statement-cache keys and query signatures change with their text.
8891dcf to
e400a90
Compare
nickdujay-pplx
left a comment
There was a problem hiding this comment.
Approved at e400a90. Independently ran 160 targeted tests with real PostgreSQL; all passed, including substrate ownership, concurrent relabel/demotion, public read/list behavior, and Direct selection/registry cases. Checked the SQL bind order, shared-select suffix renumbering, enum text encoding, and existing index predicates. CI is green; no blocking correctness findings.
Deployment and rollback: no schema, persisted-format, configuration, or inter-service protocol changes. The enum binds the same text values as the old literals, and the Rust comparisons preserve the existing fallback behavior. Old and new processes can coexist, and reverting the binaries does not require a data conversion or migration rollback. No new service deployment ordering is required.
Both this patch and #505 apply cleanly on top of the other. They were tested separately on their exact reviewed heads; I did not rerun the Kind suites or the author's synthetic plan-cache benchmark.
Control-plane Postgres `sandboxes.managed_by` and `nodes.managed_by` hold the substrate that owns a row, 'k8s' or 'space'. After #373, #383 and #506 the remaining production literals were the shared routability fragment, the Kubernetes adapter's builder lookup and the Direct route-identity match. They now bind `SubstrateKind` or match its parsed variant, and a new archcheck instance keeps new literals out of production code. No schema, wire format, metric or configuration changes, and no behavior change. - common-svc: `routable_execution_or_non_k8s(alias, kubernetes_placeholder)` writes `{alias}.managed_by <> {placeholder}` instead of `<> 'k8s'`. Each caller binds `SubstrateKind::Kubernetes` at the placeholder it numbers after its own parameters: the transient sweeper's aged-transient list and restoring converge (`$5`), internal-api's restored-state convergence (`$4`) and the api's wake-completion CAS (`$2`). - api Kubernetes adapter: `builder_client_for_node` binds `n.managed_by = $2`. - api Direct: the route identity's `substrate_kind` text is parsed as `SubstrateKind` in `route_identity_substrate_kind`, which maps exactly 'space' and 'k8s' as before and rejects any other text with the same error. - ARCHITECTURE.toml: `pattern_ratchet.substrate_literals` matches SQL comparisons of `managed_by` with a 'k8s'/'space' literal (=, <>, !=, IS [NOT] DISTINCT FROM, [NOT] IN; any case and alias) and Rust comparisons or match arms on "k8s"/"space". It allows test-support modules and fixtures that archcheck's filename-based test rule counts as production. On the parent commit it matches exactly the four converted lines; on this commit it matches nothing, so it has no baseline table and every behaviour is required. - Tests: Postgres-backed twins for each converted statement. A new Kubernetes builder-lookup test, a unit table for the route-identity labels, and an otherwise identical 'space' row added to the sweeper's converge, internal-api's restored convergence and the wake-completion redrive tests. Test callers of the sweeper's shared builders bind the extra parameter. - Docs: `docs/rust/space-conventions.md`, the substrate AGENTS.md and the space-rust-development skill say every crate's production SQL binds the value; Rust parses it and matches the variant (or calls the `SubstrateKind` method that names the question) and never compares the label string; and the archcheck rule enforces it. Plugin version 0.2.33 -> 0.2.34. Converted statements' text changes, so their sqlx statement-cache keys and query signatures change; statements below an added `.bind` move to new `sql!` locators.
Control-plane Postgres `sandboxes.managed_by` and `nodes.managed_by` hold the substrate that owns a row, 'k8s' or 'space'. After #373, #383 and #506 the remaining production literals were the shared routability fragment, the Kubernetes adapter's builder lookup and the Direct route-identity match, plus three statements that landed on main since: internal-api's publisher-node lookup and Direct's create-target claim and release. They now bind `SubstrateKind` or match its parsed variant, and a new archcheck instance keeps new literals out of production code. No schema, wire format, metric or configuration changes, and no behavior change. - common-svc: `routable_execution_or_non_k8s(alias, kubernetes_placeholder)` writes `{alias}.managed_by <> {placeholder}` instead of `<> 'k8s'`. Each caller binds `SubstrateKind::Kubernetes` at the placeholder it numbers after its own parameters: the transient sweeper's aged-transient list and restoring converge (`$5`), internal-api's restored-state convergence (`$4`) and the api's wake-completion CAS (`$2`). - api Kubernetes adapter: `builder_client_for_node` binds `n.managed_by = $2`. - internal-api: `publisher_node_for_identity` binds `n.managed_by = $3` (`Kubernetes`). - api Direct: `claim_create_target` and `release_create_target` bind `managed_by = $3` (`Space`). - api Direct: the route identity's `substrate_kind` text is parsed as `SubstrateKind` in `route_identity_substrate_kind`, which maps exactly 'space' and 'k8s' as before and rejects any other text with the same error. - ARCHITECTURE.toml: `pattern_ratchet.substrate_literals` matches SQL comparisons of `managed_by` with a 'k8s'/'space' literal (=, <>, !=, IS [NOT] DISTINCT FROM, [NOT] IN; any case and alias) and Rust comparisons or match arms on "k8s"/"space". It allows test-support modules and fixtures that archcheck's filename-based test rule counts as production. On the parent commit it matches exactly the seven converted lines; on this commit it matches nothing, so it has no baseline table and every behaviour is required. - Tests: Postgres-backed twins for each converted statement. New Kubernetes builder-lookup and Direct create-target tests, a unit table for the route-identity labels, a relabelled-builder case in the tenant publication table, and an otherwise identical 'space' row added to the sweeper's converge, internal-api's restored convergence and the wake-completion redrive tests. Test callers of the sweeper's shared builders bind the extra parameter. - Docs: `docs/rust/space-conventions.md`, the substrate AGENTS.md and the space-rust-development skill say every crate's production SQL binds the value; Rust parses it and matches the variant (or calls the `SubstrateKind` method that names the question) and never compares the label string; and the archcheck rule enforces it. Plugin version 0.2.33 -> 0.2.34. Converted statements' text changes, so their sqlx statement-cache keys and query signatures change; statements below an added `.bind` move to new `sql!` locators.
Control-plane Postgres `sandboxes.managed_by` and `nodes.managed_by` hold the substrate that owns a row, 'k8s' or 'space'. After #373, #383 and #506 the remaining production literals were the shared routability fragment, the Kubernetes adapter's builder lookup and the Direct route-identity match, plus Direct's create-target claim and release, which landed on main since. They now bind `SubstrateKind` or match its parsed variant, and a new archcheck instance keeps new literals out of production code. No schema, wire format, metric or configuration changes, and no behavior change. - common-svc: `routable_execution_or_non_k8s(alias, kubernetes_placeholder)` writes `{alias}.managed_by <> {placeholder}` instead of `<> 'k8s'`. Each caller binds `SubstrateKind::Kubernetes` at the placeholder it numbers after its own parameters: the transient sweeper's aged-transient list and restoring converge (`$5`), internal-api's restored-state convergence (`$4`) and the api's wake-completion CAS (`$2`). - api Kubernetes adapter: `builder_client_for_node` binds `n.managed_by = $2`. - api Direct: `claim_create_target` and `release_create_target` bind `managed_by = $3` (`Space`). - api Direct: the route identity's `substrate_kind` text is parsed as `SubstrateKind` in `route_identity_substrate_kind`, which maps exactly 'space' and 'k8s' as before and rejects any other text with the same error. - ARCHITECTURE.toml: `pattern_ratchet.substrate_literals` matches SQL comparisons of `managed_by` with a 'k8s'/'space' literal (=, <>, !=, IS [NOT] DISTINCT FROM, [NOT] IN; any case and alias) and Rust comparisons or match arms on "k8s"/"space". It allows test-support modules and fixtures that archcheck's filename-based test rule counts as production. On the parent commit it matches exactly the six converted lines; on this commit it matches nothing, so it has no baseline table and every behaviour is required. - Tests: Postgres-backed twins for each converted statement. New Kubernetes builder-lookup and Direct create-target tests, a unit table for the route-identity labels, and an otherwise identical 'space' row added to the sweeper's converge, internal-api's restored convergence and the wake-completion redrive tests. Test callers of the sweeper's shared builders bind the extra parameter. - Docs: `docs/rust/space-conventions.md`, the substrate AGENTS.md and the space-rust-development skill say every crate's production SQL binds the value; Rust parses it and matches the variant (or calls the `SubstrateKind` method that names the question) and never compares the label string; and the archcheck rule enforces it. Plugin version 0.2.33 -> 0.2.34. Converted statements' text changes, so their sqlx statement-cache keys and query signatures change; statements below an added `.bind` move to new `sql!` locators.
Control-plane Postgres `sandboxes.managed_by` and `nodes.managed_by` hold the substrate that owns a row, 'k8s' or 'space'. After #373, #383 and #506 the remaining production literals were the shared routability fragment, the Kubernetes adapter's builder lookup and the Direct route-identity match, plus Direct's create-target claim and release, which landed on main since. They now bind `SubstrateKind` or match its parsed variant, and a new archcheck instance keeps new literals out of production code. No schema, wire format, metric or configuration changes, and no behavior change. - common-svc: `routable_execution_or_non_k8s(alias, kubernetes_placeholder)` writes `{alias}.managed_by <> {placeholder}` instead of `<> 'k8s'`. Each caller binds `SubstrateKind::Kubernetes` at the placeholder it numbers after its own parameters: the transient sweeper's aged-transient list and restoring converge (`$5`), internal-api's restored-state convergence (`$4`) and the api's wake-completion CAS (`$2`). - api Kubernetes adapter: `builder_client_for_node` binds `n.managed_by = $2`. - api Direct: `claim_create_target` and `release_create_target` bind `managed_by = $3` (`Space`). - api Direct: the route identity's `substrate_kind` text is parsed as `SubstrateKind` in `route_identity_substrate_kind`, which maps exactly 'space' and 'k8s' as before and rejects any other text with the same error. - ARCHITECTURE.toml: `pattern_ratchet.substrate_literals` matches SQL comparisons of `managed_by` with a 'k8s'/'space' literal (=, <>, !=, IS [NOT] DISTINCT FROM, [NOT] IN; any case and alias) and Rust comparisons or match arms on "k8s"/"space". It allows test-support modules and fixtures that archcheck's filename-based test rule counts as production. On the parent commit it matches exactly the six converted lines; on this commit it matches nothing, so it has no baseline table and every behaviour is required. - Tests: Postgres-backed twins for each converted statement. New Kubernetes builder-lookup and Direct create-target tests, a unit table for the route-identity labels, and an otherwise identical 'space' row added to the sweeper's converge, internal-api's restored convergence and the wake-completion redrive tests. Test callers of the sweeper's shared builders bind the extra parameter. - Docs: `docs/rust/space-conventions.md`, the substrate AGENTS.md and the space-rust-development skill say every crate's production SQL binds the value; Rust parses it and matches the variant (or calls the `SubstrateKind` method that names the question) and never compares the label string; and the archcheck rule enforces it. Plugin version 0.2.33 -> 0.2.34. Converted statements' text changes, so their sqlx statement-cache keys and query signatures change; statements below an added `.bind` move to new `sql!` locators.
Control-plane Postgres `sandboxes.managed_by` and `nodes.managed_by` hold the substrate that owns a row, 'k8s' or 'space'. After #373, #383 and #506 the remaining production literals were the shared routability fragment, the Kubernetes adapter's builder lookup and the Direct route-identity match, plus Direct's create-target claim and release, which landed on main since. They now bind `SubstrateKind` or match its parsed variant, and a new archcheck instance keeps new literals out of production code. No schema, wire format, metric or configuration changes, and no behavior change. - common-svc: `routable_execution_or_non_k8s(alias, kubernetes_placeholder)` writes `{alias}.managed_by <> {placeholder}` instead of `<> 'k8s'`. Each caller binds `SubstrateKind::Kubernetes` at the placeholder it numbers after its own parameters: the transient sweeper's aged-transient list and restoring converge (`$5`), internal-api's restored-state convergence (`$4`) and the api's wake-completion CAS (`$2`). - api Kubernetes adapter: `builder_client_for_node` binds `n.managed_by = $2`. - api Direct: `claim_create_target` and `release_create_target` bind `managed_by = $3` (`Space`). - api Direct: the route identity's `substrate_kind` text is parsed as `SubstrateKind` in `route_identity_substrate_kind`, which maps exactly 'space' and 'k8s' as before and rejects any other text with the same error. - ARCHITECTURE.toml: `pattern_ratchet.substrate_literals` matches SQL comparisons of `managed_by` with a 'k8s'/'space' literal (=, <>, !=, IS [NOT] DISTINCT FROM, [NOT] IN; any case and alias) and Rust comparisons or match arms on "k8s"/"space". It allows test-support modules and fixtures that archcheck's filename-based test rule counts as production. On the parent commit it matches exactly the six converted lines; on this commit it matches nothing, so it has no baseline table and every behaviour is required. - Tests: Postgres-backed twins for each converted statement. New Kubernetes builder-lookup and Direct create-target tests, a unit table for the route-identity labels, and an otherwise identical 'space' row added to the sweeper's converge, internal-api's restored convergence and the wake-completion redrive tests. Test callers of the sweeper's shared builders bind the extra parameter. - Docs: `docs/rust/space-conventions.md`, the substrate AGENTS.md and the space-rust-development skill say every crate's production SQL binds the value; Rust parses it and matches the variant (or calls the `SubstrateKind` method that names the question) and never compares the label string; and the archcheck rule enforces it. Plugin version 0.2.33 -> 0.2.34. Converted statements' text changes, so their sqlx statement-cache keys and query signatures change; statements below an added `.bind` move to new `sql!` locators.
Control-plane Postgres `sandboxes.managed_by` and `nodes.managed_by` hold the substrate that owns a row, 'k8s' or 'space'. After #373, #383 and #506 the remaining production literals were the shared routability fragment, the Kubernetes adapter's builder lookup and the Direct route-identity match, plus Direct's create-target claim and release, which landed on main since. They now bind `SubstrateKind` or match its parsed variant, and a new archcheck instance keeps new literals out of production code. No schema, wire format, metric or configuration changes, and no behavior change. - common-svc: `routable_execution_or_non_k8s(alias, kubernetes_placeholder)` writes `{alias}.managed_by <> {placeholder}` instead of `<> 'k8s'`. Each caller binds `SubstrateKind::Kubernetes` at the placeholder it numbers after its own parameters: the transient sweeper's aged-transient list and restoring converge (`$5`), internal-api's restored-state convergence (`$4`) and the api's wake-completion CAS (`$2`). - api Kubernetes adapter: `builder_client_for_node` binds `n.managed_by = $2`. - api Direct: `claim_create_target` and `release_create_target` bind `managed_by = $3` (`Space`). - api Direct: the route identity's `substrate_kind` text is parsed as `SubstrateKind` in `route_identity_substrate_kind`, which maps exactly 'space' and 'k8s' as before and rejects any other text with the same error. - ARCHITECTURE.toml: `pattern_ratchet.substrate_literals` catches the common literal forms in production code: SQL comparisons of `managed_by` with 'k8s'/'space' (=, <>, !=, IS [NOT] DISTINCT FROM, [NOT] IN, = ANY, same-line CASE ... WHEN; any case, alias, prefix, cast or wrapping call), and Rust comparisons, label-string matches and match arms on "k8s"/"space" beside a substrate name. Its comment lists what it misses and its false positives. It allows test-support modules and fixtures that archcheck's filename-based test rule counts as production. On the parent commit it matches the seven lines this commit converts; on this commit it matches nothing, so it has no baseline table. no_crossing, listed_no_growth and stale_entries are required; no_new_entries and no_raised_caps are advisory, so a deliberate exception is a reviewed baseline entry. - Tests: Postgres-backed twins for each converted statement. New Kubernetes builder-lookup and Direct create-target tests, a unit table for the route-identity labels, and an otherwise identical 'space' row added to the sweeper's converge, internal-api's restored convergence and the wake-completion redrive tests. Test callers of the sweeper's shared builders bind the extra parameter. - Docs: `docs/rust/space-conventions.md`, the substrate AGENTS.md and the space-rust-development skill say every crate's production SQL binds the value; in Rust a decision about a row's substrate calls a `SubstrateKind` method named for the question, a fact about the running substrate reads `capabilities()`, and code never compares the label string; and the archcheck rule catches the common literal forms. Plugin version 0.2.33 -> 0.2.34. Converted statements' text changes, so their sqlx statement-cache keys and query signatures change; statements below an added `.bind` move to new `sql!` locators.
Control-plane Postgres `sandboxes.managed_by` and `nodes.managed_by` hold the substrate that owns a row, 'k8s' or 'space'. After #373, #383 and #506 the remaining production literals were the shared routability fragment, the Kubernetes adapter's builder lookup and the Direct route-identity match, plus Direct's create-target claim and release, which landed on main since. They now bind `SubstrateKind` or match its parsed variant, and a new archcheck instance keeps new literals out of production code. No schema, wire format, metric or configuration changes, and no behavior change. - common-svc: `routable_execution_or_non_k8s(alias, kubernetes_placeholder)` writes `{alias}.managed_by <> {placeholder}` instead of `<> 'k8s'`. Each caller binds `SubstrateKind::Kubernetes` at the placeholder it numbers after its own parameters: the transient sweeper's aged-transient list and restoring converge (`$5`), internal-api's restored-state convergence (`$4`) and the api's wake-completion CAS (`$2`). - api Kubernetes adapter: `builder_client_for_node` binds `n.managed_by = $2`. - api Direct: `claim_create_target` and `release_create_target` bind `managed_by = $3` (`Space`). - api Direct: the route identity's `substrate_kind` text is parsed as `SubstrateKind` in `route_identity_substrate_kind`, which maps exactly 'space' and 'k8s' as before and rejects any other text with the same error. - ARCHITECTURE.toml: `pattern_ratchet.substrate_literals` catches the common literal forms in production code: SQL comparisons of `managed_by` with 'k8s'/'space' (=, <>, !=, IS [NOT] DISTINCT FROM, [NOT] IN, = ANY, same-line CASE ... WHEN; any case, alias, prefix, quoting, cast or wrapping call), and Rust comparisons with the label (raw strings included), a `NODE_MANAGED_BY_*` const or a variant's `.as_str()`/`.to_string()`, matches on the label string, and match arms on "k8s"/"space" beside a substrate name. Its comment lists what it misses and its false positives. It allows test-support modules and fixtures that archcheck's filename-based test rule counts as production. On the parent commit it matches the seven lines this commit converts; on this commit it matches nothing, so it has no baseline table. no_crossing, listed_no_growth and stale_entries are required; no_new_entries and no_raised_caps are advisory, so a deliberate exception is a reviewed baseline entry. - Tests: Postgres-backed twins for each converted statement. New Kubernetes builder-lookup and Direct create-target tests, a unit table for the route-identity labels, and an otherwise identical 'space' row added to the sweeper's converge, internal-api's restored convergence and the wake-completion redrive tests. Test callers of the sweeper's shared builders bind the extra parameter. - Docs: `docs/rust/space-conventions.md`, the substrate AGENTS.md and the space-rust-development skill say every crate's production SQL binds the value; in Rust a decision about a row's substrate calls a `SubstrateKind` method named for the question, a fact about the running substrate reads `capabilities()`, and code never compares the label string; and the archcheck rule catches the common literal forms. Plugin version 0.2.33 -> 0.2.34. Converted statements' text changes, so their sqlx statement-cache keys and query signatures change; statements below an added `.bind` move to new `sql!` locators.
Control-plane Postgres `sandboxes.managed_by` and `nodes.managed_by` hold the substrate that owns a row, 'k8s' or 'space'. After #373, #383 and #506 the remaining production literals were the shared routability fragment, the Kubernetes adapter's builder lookup and the Direct route-identity match, plus Direct's create-target claim and release, which landed on main since. They now bind `SubstrateKind` or match its parsed variant, and a new archcheck instance keeps new literals out of production code. No schema, wire format, metric or configuration changes, and no behavior change. - common-svc: `routable_execution_or_non_k8s(alias, kubernetes_placeholder)` writes `{alias}.managed_by <> {placeholder}` instead of `<> 'k8s'`. Each caller binds `SubstrateKind::Kubernetes` at the placeholder it numbers after its own parameters: the transient sweeper's aged-transient list and restoring converge (`$5`), internal-api's restored-state convergence (`$4`) and the api's wake-completion CAS (`$2`). - api Kubernetes adapter: `builder_client_for_node` binds `n.managed_by = $2`. - api Direct: `claim_create_target` and `release_create_target` bind `managed_by = $3` (`Space`). - api Direct: the route identity's `substrate_kind` text is parsed as `SubstrateKind` in `route_identity_substrate_kind`, which maps exactly 'space' and 'k8s' as before and rejects any other text with the same error. - ARCHITECTURE.toml: `pattern_ratchet.substrate_literals` catches the common literal forms in production code: SQL comparisons of `managed_by` with 'k8s'/'space' (=, <>, !=, IS [NOT] DISTINCT FROM, [NOT] IN, = ANY, same-line CASE ... WHEN; any case, alias, prefix, quoting, cast or wrapping call), and Rust comparisons with the label (raw strings included), a `NODE_MANAGED_BY_*` const or a variant's `.as_str()`/`.to_string()`, matches on the label string, and match arms on "k8s"/"space" beside a substrate name. Its comment lists what it misses and its false positives. It allows test-support modules and fixtures that archcheck's filename-based test rule counts as production. On the parent commit it matches the seven lines this commit converts; on this commit it matches nothing, so it has no baseline table. no_crossing, listed_no_growth and stale_entries are required; no_new_entries and no_raised_caps are advisory, so a deliberate exception is a reviewed baseline entry. - Tests: Postgres-backed twins for each converted statement. New Kubernetes builder-lookup and Direct create-target tests, a unit table for the route-identity labels, and an otherwise identical 'space' row added to the sweeper's converge, internal-api's restored convergence and the wake-completion redrive tests. Test callers of the sweeper's shared builders bind the extra parameter. - Docs: `docs/rust/space-conventions.md`, the substrate AGENTS.md and the space-rust-development skill say every crate's production SQL binds the value; in Rust a decision about a row's substrate calls a `SubstrateKind` method named for the question, a fact about the running substrate reads `capabilities()`, and code never compares the label string; and the archcheck rule catches the common literal forms. Plugin version 0.2.33 -> 0.2.34. Converted statements' text changes, so their sqlx statement-cache keys and query signatures change; statements below an added `.bind` move to new `sql!` locators.
…erplexityai#611) Control-plane Postgres `sandboxes.managed_by` and `nodes.managed_by` hold the substrate that owns a row, 'k8s' or 'space'. After perplexityai#373, perplexityai#383 and perplexityai#506 the remaining production literals were the shared routability fragment, the Kubernetes adapter's builder lookup and the Direct route-identity match, plus Direct's create-target claim and release, which landed on main since. They now bind `SubstrateKind` or match its parsed variant, and a new archcheck instance keeps new literals out of production code. No schema, wire format, metric or configuration changes, and no behavior change. - common-svc: `routable_execution_or_non_k8s(alias, kubernetes_placeholder)` writes `{alias}.managed_by <> {placeholder}` instead of `<> 'k8s'`. Each caller binds `SubstrateKind::Kubernetes` at the placeholder it numbers after its own parameters: the transient sweeper's aged-transient list and restoring converge (`$5`), internal-api's restored-state convergence (`$4`) and the api's wake-completion CAS (`$2`). - api Kubernetes adapter: `builder_client_for_node` binds `n.managed_by = $2`. - api Direct: `claim_create_target` and `release_create_target` bind `managed_by = $3` (`Space`). - api Direct: the route identity's `substrate_kind` text is parsed as `SubstrateKind` in `route_identity_substrate_kind`, which maps exactly 'space' and 'k8s' as before and rejects any other text with the same error. - ARCHITECTURE.toml: `pattern_ratchet.substrate_literals` catches the common literal forms in production code: SQL comparisons of `managed_by` with 'k8s'/'space' (=, <>, !=, IS [NOT] DISTINCT FROM, [NOT] IN, = ANY, same-line CASE ... WHEN; any case, alias, prefix, quoting, cast or wrapping call), and Rust comparisons with the label (raw strings included), a `NODE_MANAGED_BY_*` const or a variant's `.as_str()`/`.to_string()`, matches on the label string, and match arms on "k8s"/"space" beside a substrate name. Its comment lists what it misses and its false positives. It allows test-support modules and fixtures that archcheck's filename-based test rule counts as production. On the parent commit it matches the seven lines this commit converts; on this commit it matches nothing, so it has no baseline table. no_crossing, listed_no_growth and stale_entries are required; no_new_entries and no_raised_caps are advisory, so a deliberate exception is a reviewed baseline entry. - Tests: Postgres-backed twins for each converted statement. New Kubernetes builder-lookup and Direct create-target tests, a unit table for the route-identity labels, and an otherwise identical 'space' row added to the sweeper's converge, internal-api's restored convergence and the wake-completion redrive tests. Test callers of the sweeper's shared builders bind the extra parameter. - Docs: `docs/rust/space-conventions.md`, the substrate AGENTS.md and the space-rust-development skill say every crate's production SQL binds the value; in Rust a decision about a row's substrate calls a `SubstrateKind` method named for the question, a fact about the running substrate reads `capabilities()`, and code never compares the label string; and the archcheck rule catches the common literal forms. Plugin version 0.2.33 -> 0.2.34. Converted statements' text changes, so their sqlx statement-cache keys and query signatures change; statements below an added `.bind` move to new `sql!` locators.
Based on
main(232cea1). This PR is one commit. Phase 2 of containing the Direct/Kubernetes substrate split, api crate: the same method as #373 (cluster controller) and #383 (internal-api, trigger queue, sandbox store).What merging does
Control-plane Postgres
sandboxes.managed_byandnodes.managed_byhold the substrate that owns a row,'k8s'or'space'. The api crate scoped on, and wrote, that value with raw literals in 22 production SQL statements, and compared it as a string in two places. After this PR those statements bindSubstrateKind(managed_by = $nwith.bind(SubstrateKind::Kubernetes)/::Space), each statement stays syntactically whole, and the two comparisons parse the label and match the variant.There are no schema, wire format, metric or configuration changes, and no behavior changes.
SubstrateKindencodes as the column's text, so a boundSubstrateKind::Spacematches and writes exactly what'space'did.Each new placeholder is numbered after the statement's existing parameters and its
.bindcomes last, except in the public sandbox select (below).crates/libs/api/src/)controllers/substrate/direct.rsrandom_ready_nodesn.managed_by = 'space'→$2=Spaceeligible_start_node_count$1=Spacecached_worker_create_candidates, cached-ready$5=Spacecached_worker_create_candidates, last-known fallback$5=Spacepull_on_create_worker_candidates$2=Spacepooled_worker_candidates$4=Spacesandbox_node$2=Spacedirect_provenance_source_node$2=Spacedirect_node_by_id$2=Spacedirect_restore_candidates_for_mode, source-node query$2=Spacedirect_restore_candidates_for_mode, remote query$3=Spacebuilder_client_for_node$2=Spaceroutes/admin/node_maintenance.rscreate_nodereactivationn.managed_by = 'space'andSET managed_by = 'space'→ one$10=Spacecreate_nodeinsertVALUES (…, 'space', …)→$9=Spaceupdate_noden.managed_by = 'space'→$9=Spacehard_delete_direct_node$2=Spacecontrollers/substrate/kubernetes/store/node_inventory.rsdemote_unobserved_k8s_nodes= 'k8s'→ one$4=Kubernetesupsert_k8s_node_roleVALUES (…, 'k8s', …), the ON CONFLICT guard and the fallback SELECT → one$6=Kubernetesmanaged_by = 'k8s'→$2=Kubernetesroutes/public/sandboxes.rsPUBLIC_SANDBOX_SELECT(by id and page)s.managed_by = 'k8s'/'space'→$6=Kubernetes,$7=Spaceroutes/admin/templates.rsreconcile_target_node_idsn.managed_by != 'space'→$1=Spacepublic_template_operations.rsnodes_with_template_inventoryn.managed_by != 'space'→$2=SpacePUBLIC_SANDBOX_SELECTis shared by the by-id and page statements, which append different suffixes numbered from$6. The two substrate placeholders take$6and$7in the shared select, so the suffixes renumber from$8(by id$8–$9, page$8–$12), and both callers bind the two values right after the five shared keys. A placeholder numbered after each suffix's own parameters would have had to differ between the two statements.bootstrap_k8s_origin_node).direct.rschanges only SQL text, binds and the import, since other work is changing this file's behavior in parallel.Rust comparisons
controllers/sandbox/spaced.rscontrol_runtime_client: the dormant-wake fallback guardsandbox.managed_by == "k8s"→matches!(sandbox.managed_by.parse::<SubstrateKind>(), Ok(SubstrateKind::Kubernetes))."k8s"takes the fallback;"space"and any unparseable value don't, as before.Sandbox.managed_bystays aString(a public model type).routes/session_auth.rsConsoleSubstrate::from_managed_by(the console/mesubstrate):matchon the string → match on the parsed variant;"k8s"→K8s, everything else →Space, as before. The serialized values are unchanged.Left as is
controllers/substrate/direct.rsload_active_direct_route_identitymatches the Direct route identity'ssubstrate_kindtext ("space"/"k8s"→ the sbx-mgr proto enum, anything else an error). It is a Rust match, not SQL, and this PR keeps itsdirect.rsedits to SQL statements; converting it toSubstrateKind::parsewould keep its behavior and can follow.DirectSubstrate::managed_by()returns"space": that is the adapter's label, not a comparison.controllers/substrate/kubernetes/adapter/substrate_adapter.rs(n.managed_by = 'k8s', and the adapter's"k8s"label): api, cluster-controller: reach Kubernetes-only state through one accessor #386 edits that file.routable_execution_or_non_k8sfragment (crates/libs/common/svc/src/sql/sandbox_routability.rs), whichwake_completion.rssplices: shared debt, as in cluster-controller: bind the substrate value instead of writing it in SQL #373 and internal-api, trigger-queue, store: bind the substrate value instead of writing it in SQL #383.managed_byvalues into the rows they set up.docs/rust/space-conventions.mdneeds an edit then. Its "Substrate values are bound parameters" paragraph still says the api crate'sdirect.rs"has about a dozen" raw literals (lines 194–195 onmain), which this PR makes untrue. It is left out here only because internal-api, trigger-queue, store: bind the substrate value instead of writing it in SQL #383 rewrites that paragraph, and editing it in both PRs would conflict. After both land, internal-api, trigger-queue, store: bind the substrate value instead of writing it in SQL #383's sentence "The production code of the cluster controller, internal-api and the trigger queue follows this, except where it embedsroutable_execution_or_non_k8s" should add the api crate, and the parenthetical "(crates/libs/api/src/controllers/substrate/direct.rshas about a dozen)" should be dropped (or point at the Kubernetes adapter's remaining literal and the route-identity match above), sincedirect.rs's SQL no longer has any.Evidence
A Postgres-backed test per converted statement
Each test seeds rows the statement would act on and shows a row of the statement's substrate included (or written) and an otherwise identical row of the other substrate excluded. For the Direct-scoped statements the
'space'row is the included one. Every twin keeps whatever else the statement joins on (for Direct nodes, a livenode_directrow), so onlymanaged_bydecides. The tests run on the shared nextest Postgres withSPACE_CI_REQUIRE_TEST_DEPS=1.New tests (
crates/libs/api/src/…):controllers/substrate/direct/tests/substrate_scope.rsworker_selection_reads_only_direct_nodes: a ready Direct worker and an identical worker relabelled'k8s', both with template inventory and pool slots. D1–D6 each return only the Direct worker; D4 is reached by making every heartbeat stale.placement_lookups_read_only_direct_nodes: a sandbox placed on each twin. D7 and D9 return the Direct node and refuse the Kubernetes one asUnavailable; D8 returnsNonefor it.restore_candidates_read_only_direct_nodes: D10's source-node candidate exists only for the snapshot the Direct twin holds; D11's remote candidates for a third node's durable snapshot are only the Direct twin.builder_lookup_reads_only_direct_nodes: D12 refuses the Kubernetes builder as "not registered"; the Direct builder gets past the lookup (it is dialed, or fails on transport configuration). Both outcomes are awaited under one 5 s bound.controllers/substrate/kubernetes/store/k8s_store_postgres_tests/substrate_scope_tests.rsdemotion_selects_only_kubernetes_nodes(K1, candidate scan): two stale, missing nodes withnode_k8srows; the older one is relabelled'space'. With one demotion per tick, only the Kubernetes node is demoted, so the Direct node did not take the slot.demotion_rechecks_the_substrate_of_the_row_it_locks(K1, UPDATE guard): a writer holds the candidate's row lock with an uncommitted relabel; the demotion's scan sees the committed'k8s'row and its UPDATE waits on that lock, then re-evaluates the committed row. The writer commits only oncepg_stat_activityshows the demotion statement waiting on a lock with the writer'spg_backend_pid()in itspg_blocking_pids(polled every 10 ms, bounded at 5 s). The demotion's join is bounded at 5 s as well. A relabel to'space'is skipped; a write that keeps'k8s'is demoted.inventory_upsert_writes_only_kubernetes_rows(K2, K3): a fresh id is written as'k8s', an existing'k8s'row is updated, and an identical'space'row at the target id is refused withSubstrateError::Failedand left unchanged, with nonode_k8srow written.routes/admin/node_maintenance/tests/substrate_scope.rsnode_registry_writes_only_direct_nodes(N1–N4), through the handlers on a DirectAppState: re-registering an endpoint reactivates the older retired Direct node there, not a newer retired Kubernetes node at the same endpoint; a new endpoint registers a'space'node;update_noderenames only the Direct twin (404 for the other);hard_delete_direct_nodedeletes only the Direct twin.routes/public/sandboxes/tests/substrate_scope.rscurrent_failure_reads_the_execution_record_of_the_sandbox_substrate(S1): four failed sandboxes whose observed failure payload matches both execution shapes:'k8s'and'space'rows with an active Kubernetes execution, and'space'and'k8s'rows with a Direct route identity. Only the'k8s'/execution and'space'/route rows report a current failure, through both the by-id read and the page.routes/admin/templates/tests/substrate_scope.rsreconcile_targets_require_a_live_registration_only_for_direct_nodes(T1): a'k8s'node without a registry row is a target; an identical'space'node is not.controllers/substrate/kubernetes/tests/substrate_scope.rscontrol_runtime_client_wakes_only_kubernetes_rows_after_an_unreachable_runtime(R1): a suspended row on the Kubernetes substrate, read back from Postgres withfetch_sandbox. With'k8s'the call reaches the dormant-wake fallback, whose activation charge the zero-capacity admission refuses (429) before any pod is created. With'space'the runtime error comes back instead. No earlier test exercised this guard.routes/session_auth/tests.rsconsole_substrate_is_kubernetes_only_for_the_kubernetes_label(R2):"k8s"→K8s;"space","kubernetes"and""→Space.Extended:
public_template_operations/tests.rstemplate_delete_inventory_query_uses_direct_node_deleted_state(P1) gains a'k8s'twin of its retired Direct node. The twin is included and the Direct node stays excluded.Predicate removal
For each converted predicate I replaced it with
(… OR TRUE), ran the api package's whole suite (cargo nextest run --locked -p pplx-space-api --all-features --retries 0 --no-fail-fast,SPACE_CI_REQUIRE_TEST_DEPS=1), and reverted. "Before" runs on a tree with this PR's production code andmain's tests; "after" runs on this PR's tests. Value sites are mutated by writing the other substrate's literal. R1 is mutated withtrue || matches!(…)(a) andfalse && matches!(…)(b), and R2 by mapping every label toK8s(a) or none (b). Unmutated, the two trees pass 1744/1744 and 1755/1755. The R1 test was added after this sweep, so I reran R1a/R1b on the final tests. After review round 1 (below), K1b, D12, R1a and R1b were rerun on this head.OR TRUEworker_selection_reads_only_direct_nodesOR TRUEplacement_lookups_read_only_direct_nodesOR TRUErestore_candidates_read_only_direct_nodesOR TRUEbuilder_lookup_reads_only_direct_nodesOR TRUEnode_registry_writes_only_direct_nodesSET managed_by = 'k8s'VALUES (…, 'k8s', …)OR TRUEOR TRUEdemotion_selects_only_kubernetes_nodesOR TRUEdemotion_rechecks_the_substrate_of_the_row_it_locksVALUES (…, 'space', …)missing_k8s_inventory_demotes_only_stale_ready_nodes,pod_placement_does_not_downgrade_ready_node_state), plus 3 of the new onesOR TRUEinventory_upsert_writes_only_kubernetes_rowsOR TRUEOR TRUEcurrent_failure_reads_the_execution_record_of_the_sandbox_substraten.managed_by != $1 OR TRUEreconcile_targets_require_a_live_registration_only_for_direct_nodesn.managed_by != $2 OR TRUEtemplate_delete_inventory_query_uses_direct_node_deleted_statecontrol_runtime_client_wakes_only_kubernetes_rows_after_an_unreachable_runtimeconsole_substrate_is_kubernetes_only_for_the_kubernetes_labelOn
main's tests, 2 of the 31 single mutations (K2a, P1) failed a test. On this PR's tests all 31 do, and each of the other 29 fails only the new test named for it.Plan cache: literal vs bound
What I measured, and why. Of the converted statements, these run in Kubernetes deployments: S1 (every public sandbox read and list), K1 (every node-sync tick), K2/K3 (node inventory upserts), T1 and P1 (template admin and delete operations). S1 is the only hot path there. K1 is periodic and scans
nodes, a small table whosemanaged_byis skewed toward'k8s'. Both got the full plan-cache sequence. The rest got a generic-plan comparison.Direct statements (D1–D12, N1–N4): not measured, and why binding is safe there. These don't run in Kubernetes deployments (the registry routes return 501 on that substrate), but several are hot on Direct: D2 on every admission check (
cluster_admission/mod.rs,eligible_start_node_count), and the create path: D3 on every create from a cached template, D4 as its fallback when no worker is cached-ready, and D6 on pooled creates. Their substrate predicate is always onnodes.managed_by, never onsandboxes. On a Direct deployment's database that column is effectively single-valued ('space': Direct registers its nodes through its own registry, and no Kubernetes node sync runs there). A generic plan estimatesmanaged_by = $nat 1/ndistinct, which then equals the literal's frequency (1.0), so bound and literal give the same row estimates and the same plan choice.nodesalso holds one row per registered node, so these plans are driven by the joins onid/node_idand their other filters, not by this predicate. That is an argument from the statistics, not a measurement; I did not seed a Direct-shaped database.Setup. A scratch database migrated from the test template on the repository's test image (PostgreSQL 16.15), with synthetic, deliberately skewed data: 60 orgs (one holds 40% of sandboxes, two hold 15% each), 400 nodes (10%
'space'), 200,000 sandboxes (5%'space', 30% retained deleted tombstones, 3% failed), an active Kubernetes execution or a Direct route identity per placed sandbox of each substrate, observed states, and a desired-state facet row per placed sandbox (failed ones carry a matching failure episode), followed byANALYZE. Each sequence ran on one connection withplan_cache_mode = auto:PREPAREonce with the parameter types sqlx sends, then 30 ×EXPLAIN (ANALYZE, TIMING OFF) EXECUTE, thenpg_prepared_statements. "Literal" is the statement text onmain, "bound" this PR's text.main(itsorg_idfilter is skewed in this data set). Binding doesn't change that.$6/$7instead of'k8s'::text/'space'::textand the renumbered suffix parameters. Its substrate predicates sit in the select list'shas_current_failureexpression, not inWHERE, so they can't affect scans, joins or estimates.managed_by = $4at 1/ndistinct (200 of 400 nodes) instead of the literal's real frequency (360). The row estimates and costs drop accordingly, and the plan cache latches the generic plan after five executions either way.managed_by. Three exist, all full (not partial):idx_nodes_managed_by (managed_by),idx_nodes_role (managed_by, role)andidx_sandboxes_managed_by (managed_by). None of the measured plans uses them, literal or bound:nodeswith a sequential scan in both plans (measured). The bound estimate (200 of 400) is lower than the literal's (360), which leans slightly toward an index, but at about half of a 400-row table the planner still seq-scans for both texts.managed_by != $ninside anOR, which a btree can't serve.No converted statement is materially worse bound, so no literal is kept as an exception. These are synthetic numbers on one PostgreSQL 16.15 instance; a production database's statistics decide the actual margins.
Characterization tests pass unchanged
The api characterization tests from #273, #290, #326 and #330 pass in every run below:
controllers::substrate::kubernetes::adapter_characterization_tests(57) andcontrollers::sandbox::lifecycle::characterization_tests(16). This PR doesn't touch their files orscripted_test_doubles.rs.Decisions made unattended
substrate.managed_by()const. Taking$6/$7in the shared part keeps one select and moves only the suffix numbers; the binds follow the same order in both callers.substrate_kindmatch indirect.rsdirect.rsedits to SQL statements while other work changes that file. It is a Rust match with an explicit error arm, so its behavior is already exhaustive; parsing it is a follow-up.Stringinstead of typing fieldsSandbox.managed_byis a public model field andmanaged_by()returns&'static stronSubstrateAdapter; typing either changes interfaces beyond this PR.insert_ready_worker,persist_k8s_node_inventory, the DirectAppState, the failure-episode payload keys), so the twins differ only inmanaged_by. Sibling files keep already-large test files from growing and avoid conflicts.'k8s'rows. A zero-capacity admission makes the fallback observable before it creates a pod.maingained production changes (#312, #165) after the first Kind run, so I reran Kind on a fresh cluster at this head.Locator note
Converted statements' text changes, so their
/*module_path:line*/locators, sqlx statement-cache keys and query signatures change; statements below an added.bindline move to new locators. Module paths don't change.Overlaps with open PRs
docs/rust/space-conventions.mdparagraph about remaining literals; see "Docs" above for the sentence to update once both land.substrate_adapter.rs, which keeps its'k8s'literal here.Review round 1 → changes
demotion_rechecks_the_substrate_of_the_row_it_locksmatched any lock-waiting backend with the demotion's text, not one the writer blockspg_backend_pid()before the demotion is spawned. A local bounded loop then waits for a backend withwait_event_type = 'Lock', that pid inpg_blocking_pids(pid)and the demotion's query text, aslifecycle/characterization_tests.rsandsession_auth/tests.rsdo. Both 5 s bounds stay. The sharedwait_for_blocked_queryhelper is unchanged. K1b rerun on this head: fails this test at its assertion (the relabelled'space'node is demoted), 1766/1767.builder_lookup_reads_only_direct_nodes'sselect!had no bound of its ownselect!over the accept and the dial is wrapped in a 5 stokio::time::timeout, and the dial task is aborted afterwards. D12 rerun: fails this test, 1766/1767.nodes.managed_byis effectively single-valued, so 1/ndistinct equals the literal's frequency, andnodesis small. It now namesidx_nodes_managed_by,idx_nodes_role (managed_by, role)andidx_sandboxes_managed_byand says why none changes these plans.kubernetes/tests.rskubernetes/tests/substrate_scope.rs.tests.rsnow differs frommainonly bymod substrate_scope;. The test now reads the row back withfetch_sandboxinstead of building aSandboxin memory. R1a and R1b rerun: both fail this test, 1766/1767.space-conventions.mdneeds the proposed sentence after this PR and #383 merge, and why it's not heremain("direct.rshas about a dozen") need it, and it's left out only to avoid conflicting with #383's edit of that paragraph.No production code changed in this round, so Kind wasn't rerun; the Kind rows below are from
4e5bcf3.Review round 2 → changes
tokio::select!over&mut demotionand the pid-tied poll, aslifecycle/characterization_tests.rsdoes. If the demotion finishes without ever waiting on the writer, the test fails with the demotion's own result, not with "never waited".JoinHandledetaches it)tokio_util::task::AbortOnDropHandle. They are aborted on every exit: the poll failing, the join timing out, or the builder bound'sexpectpanicking. The builder test drops its handle once the outcome is known.tokio-utilwas already an api dependency, and the workspace enables itsrtfeature.Validation for this round (tests and PR body only; no production code changed, so no Kind): fmt pass;
cargo clippy -p pplx-space-api --all-targets --all-features --locked -- -D warningspass; pre-commit (Gitleaks) pass;demotion_rechecks_the_substrate_of_the_row_it_locksandbuilder_lookup_reads_only_direct_nodespassed in each of three runs (SPACE_CI_REQUIRE_TEST_DEPS=1); the K1b and D12 mutations, once each against the whole suite, each fail only their test (1766/1767; K1b at its assertion, the relabelled'space'node demoted).Validation
Head
e400a90. Round 2 changed only the two tests above; the full-suite rows below were run at8891dcf. Base232cea1. It still merges cleanly with currentmain, so it wasn't rebased again. Before review round 1 the commit was rebased0f1d3be→9254f09→232cea1, andgit range-diffshowed it unchanged each time. Kind ran on4e5bcf3(this commit on9254f09, before round 1). Its production code matches this head; the head differs only by #491's one-line internal-api dev-dependency change and round 1's test-only edits.cargo fmt --all -- --checkcargo clippy --workspace --all-targets --all-features --locked -- -D warningspre-commit run gitleaks --from-ref origin/main --to-ref HEADspace-public-reviewon the staged change and this textcargo nextest run --locked -p pplx-space-api --all-features --retries 0,SPACE_CI_REQUIRE_TEST_DEPS=1, three runs8891dcf: 1767/1767 passed in each of runs 1, 2 and 3. The one skipped test is the#[ignore]d local occupancy benchmark. Each run includes the 73 characterization tests, the 12 new tests and the extended P1 test. Earlier heads:877ebdd1767/1767 three times;2e5e56d(on0f1d3be) 1756/1756 three times.8891dcf)scripts/check-rust-error-policy.sh;scripts/source_guards/*.pythin_service_mains.pyexits 0 with advisory notes oncrates/apps/*/main.rsfiles this PR doesn't touch)scripts/check_no_inline_rust_test_modules.pycrates/libs/observability/src/{otlp_endpoint,selection}.rs, untouched here: PRE-EXISTINGcargo archmain. Growth: in five files already over advisorylisted_no_growthcaps onmain, from the added.bindlines:direct.rs3913 → 3925,node_maintenance.rs1790 → 1794,routes/public/sandboxes.rs1252 → 1258,spaced.rs1146 → 1151,routes/admin/templates.rs2264 → 2266.docker builder prune -afreclaimed 18.59 GB (15.2 GB on the device) anddocker image prune -f0 B, leaving 28 GB. No later step was needed. During the final typed suite free space dipped to 5.6 GB with no disk-related failure.4e5bcf3)smoke-kind-e2e.sh(BOOTSTRAP_TIMEOUT=1800,DATA_PLANE_URL, as CI sets) createdspace-localand started Tilt from this checkout. The api, internal-api and cluster-controller services logversion: 4e5bcf33fc19. The deployed/space/bin/space-api-svcand/space/bin/space-cluster-controllermatch the artifact image's binaries by sha256 (e6819374…,d0276ce9…), before and after the suites. The api binary containss.managed_by = $6/$7andn.managed_by != $1, and nos.managed_by = 'k8s',s.managed_by = 'space',n.managed_by = 'space'or!= 'space'. Its onlyn.managed_by = 'k8s'issubstrate_adapter.rs's (out of scope). The cluster controller, which runs node sync and the template operations, containscandidate.managed_by = $4,VALUES ($1, $2::INET, $3, $6, $4)andn.managed_by != $2, and not their literal forms.smoke-kind-e2e.sh(start)smoke-kind-typed.sh all(unsharded, CI's env: admin token file, cold-receipts dir, host dm targets loaded)smoke-kind-sandbox-fork.pysmoke-kind-e2e.sh(end)dfa3239, this commit on0f1d3be, before the R1 test was added)version: dfa32396256a, deployed binaries match the artifact). e2e 7/7, typed ECW 78/78 and native 24/24, fork pass, e2e 7/7.teardown.shon the task-owned cluster; scratch Postgres container and scratch checkouts removed🤖 Generated with Claude Code