Skip to content

api: bind the substrate value instead of writing it in SQL - #506

Merged
ac-pplx merged 1 commit into
mainfrom
ac/substrate-kind-sql-api
Oct 15, 2026
Merged

ac-pplx merged 1 commit into
mainfrom
ac/substrate-kind-sql-api

Conversation

@ac-pplx

@ac-pplx ac-pplx commented Oct 15, 2026 •

Copy link
Copy Markdown
Contributor

Based on main (232cea1). This PR is one commit. Phase 2 of containing the Direct/Kubernetes substrate split, api crate: the same method as #373 (cluster controller) and #383 (internal-api, trigger queue, sandbox store).

What merging does

Control-plane Postgres sandboxes.managed_by and nodes.managed_by hold the substrate that owns a row, 'k8s' or 'space'. The api crate scoped on, and wrote, that value with raw literals in 22 production SQL statements, and compared it as a string in two places. After this PR those statements bind SubstrateKind (managed_by = $n with .bind(SubstrateKind::Kubernetes) / ::Space), each statement stays syntactically whole, and the two comparisons parse the label and match the variant.

There are no schema, wire format, metric or configuration changes, and no behavior changes. SubstrateKind encodes as the column's text, so a bound SubstrateKind::Space matches and writes exactly what 'space' did.

Each new placeholder is numbered after the statement's existing parameters and its .bind comes last, except in the public sandbox select (below).

# File (crates/libs/api/src/) Statement Before → after
D1 controllers/substrate/direct.rs random_ready_nodes n.managed_by = 'space' → $2 = Space
D2 same eligible_start_node_count → $1 = Space
D3 same cached_worker_create_candidates, cached-ready → $5 = Space
D4 same cached_worker_create_candidates, last-known fallback → $5 = Space
D5 same pull_on_create_worker_candidates → $2 = Space
D6 same pooled_worker_candidates → $4 = Space
D7 same sandbox_node → $2 = Space
D8 same direct_provenance_source_node → $2 = Space
D9 same direct_node_by_id → $2 = Space
D10 same direct_restore_candidates_for_mode, source-node query → $2 = Space
D11 same direct_restore_candidates_for_mode, remote query → $3 = Space
D12 same builder_client_for_node → $2 = Space
N1 routes/admin/node_maintenance.rs create_node reactivation candidate n.managed_by = 'space' and SET managed_by = 'space' → one $10 = Space
N2 same create_node insert VALUES (…, 'space', …) → $9 = Space
N3 same update_node n.managed_by = 'space' → $9 = Space
N4 same hard_delete_direct_node → $2 = Space
K1 controllers/substrate/kubernetes/store/node_inventory.rs demote_unobserved_k8s_nodes candidate scan and UPDATE guard = 'k8s' → one $4 = Kubernetes
K2 same upsert_k8s_node_role VALUES (…, 'k8s', …), the ON CONFLICT guard and the fallback SELECT → one $6 = Kubernetes
K3 same ownership re-check managed_by = 'k8s' → $2 = Kubernetes
S1 routes/public/sandboxes.rs PUBLIC_SANDBOX_SELECT (by id and page) failure-episode arms s.managed_by = 'k8s' / 'space' → $6 = Kubernetes, $7 = Space
T1 routes/admin/templates.rs reconcile_target_node_ids n.managed_by != 'space' → $1 = Space
P1 public_template_operations.rs nodes_with_template_inventory n.managed_by != 'space' → $2 = Space
  • S1, renumbering. PUBLIC_SANDBOX_SELECT is shared by the by-id and page statements, which append different suffixes numbered from $6. The two substrate placeholders take $6 and $7 in the shared select, so the suffixes renumber from $8 (by id $8–$9, page $8–$12), and both callers bind the two values right after the five shared keys. A placeholder numbered after each suffix's own parameters would have had to differ between the two statements.
  • N1, K1, K2. Where a statement uses the value more than once, one parameter serves every use, so the row it writes and the rows it guards on are the same value by construction (as internal-api, trigger-queue, store: bind the substrate value instead of writing it in SQL #383 did for bootstrap_k8s_origin_node).
  • direct.rs changes only SQL text, binds and the import, since other work is changing this file's behavior in parallel.

Rust comparisons

  • R1 controllers/sandbox/spaced.rs control_runtime_client: the dormant-wake fallback guard sandbox.managed_by == "k8s" → matches!(sandbox.managed_by.parse::<SubstrateKind>(), Ok(SubstrateKind::Kubernetes)). "k8s" takes the fallback; "space" and any unparseable value don't, as before. Sandbox.managed_by stays a String (a public model type).
  • R2 routes/session_auth.rs ConsoleSubstrate::from_managed_by (the console /me substrate): match on the string → match on the parsed variant; "k8s" → K8s, everything else → Space, as before. The serialized values are unchanged.

Left as is

Evidence

A Postgres-backed test per converted statement

Each test seeds rows the statement would act on and shows a row of the statement's substrate included (or written) and an otherwise identical row of the other substrate excluded. For the Direct-scoped statements the 'space' row is the included one. Every twin keeps whatever else the statement joins on (for Direct nodes, a live node_direct row), so only managed_by decides. The tests run on the shared nextest Postgres with SPACE_CI_REQUIRE_TEST_DEPS=1.

New tests (crates/libs/api/src/…):

  • controllers/substrate/direct/tests/substrate_scope.rs
    • worker_selection_reads_only_direct_nodes: a ready Direct worker and an identical worker relabelled 'k8s', both with template inventory and pool slots. D1–D6 each return only the Direct worker; D4 is reached by making every heartbeat stale.
    • placement_lookups_read_only_direct_nodes: a sandbox placed on each twin. D7 and D9 return the Direct node and refuse the Kubernetes one as Unavailable; D8 returns None for it.
    • restore_candidates_read_only_direct_nodes: D10's source-node candidate exists only for the snapshot the Direct twin holds; D11's remote candidates for a third node's durable snapshot are only the Direct twin.
    • builder_lookup_reads_only_direct_nodes: D12 refuses the Kubernetes builder as "not registered"; the Direct builder gets past the lookup (it is dialed, or fails on transport configuration). Both outcomes are awaited under one 5 s bound.
  • controllers/substrate/kubernetes/store/k8s_store_postgres_tests/substrate_scope_tests.rs
    • demotion_selects_only_kubernetes_nodes (K1, candidate scan): two stale, missing nodes with node_k8s rows; the older one is relabelled 'space'. With one demotion per tick, only the Kubernetes node is demoted, so the Direct node did not take the slot.
    • demotion_rechecks_the_substrate_of_the_row_it_locks (K1, UPDATE guard): a writer holds the candidate's row lock with an uncommitted relabel; the demotion's scan sees the committed 'k8s' row and its UPDATE waits on that lock, then re-evaluates the committed row. The writer commits only once pg_stat_activity shows the demotion statement waiting on a lock with the writer's pg_backend_pid() in its pg_blocking_pids (polled every 10 ms, bounded at 5 s). The demotion's join is bounded at 5 s as well. A relabel to 'space' is skipped; a write that keeps 'k8s' is demoted.
    • inventory_upsert_writes_only_kubernetes_rows (K2, K3): a fresh id is written as 'k8s', an existing 'k8s' row is updated, and an identical 'space' row at the target id is refused with SubstrateError::Failed and left unchanged, with no node_k8s row written.
  • routes/admin/node_maintenance/tests/substrate_scope.rs node_registry_writes_only_direct_nodes (N1–N4), through the handlers on a Direct AppState: re-registering an endpoint reactivates the older retired Direct node there, not a newer retired Kubernetes node at the same endpoint; a new endpoint registers a 'space' node; update_node renames only the Direct twin (404 for the other); hard_delete_direct_node deletes only the Direct twin.
  • routes/public/sandboxes/tests/substrate_scope.rs current_failure_reads_the_execution_record_of_the_sandbox_substrate (S1): four failed sandboxes whose observed failure payload matches both execution shapes: 'k8s' and 'space' rows with an active Kubernetes execution, and 'space' and 'k8s' rows with a Direct route identity. Only the 'k8s'/execution and 'space'/route rows report a current failure, through both the by-id read and the page.
  • routes/admin/templates/tests/substrate_scope.rs reconcile_targets_require_a_live_registration_only_for_direct_nodes (T1): a 'k8s' node without a registry row is a target; an identical 'space' node is not.
  • controllers/substrate/kubernetes/tests/substrate_scope.rs control_runtime_client_wakes_only_kubernetes_rows_after_an_unreachable_runtime (R1): a suspended row on the Kubernetes substrate, read back from Postgres with fetch_sandbox. With 'k8s' the call reaches the dormant-wake fallback, whose activation charge the zero-capacity admission refuses (429) before any pod is created. With 'space' the runtime error comes back instead. No earlier test exercised this guard.
  • routes/session_auth/tests.rs console_substrate_is_kubernetes_only_for_the_kubernetes_label (R2): "k8s" → K8s; "space", "kubernetes" and "" → Space.

Extended: public_template_operations/tests.rs template_delete_inventory_query_uses_direct_node_deleted_state (P1) gains a 'k8s' twin of its retired Direct node. The twin is included and the Direct node stays excluded.

Predicate removal

For each converted predicate I replaced it with (… OR TRUE), ran the api package's whole suite (cargo nextest run --locked -p pplx-space-api --all-features --retries 0 --no-fail-fast, SPACE_CI_REQUIRE_TEST_DEPS=1), and reverted. "Before" runs on a tree with this PR's production code and main's tests; "after" runs on this PR's tests. Value sites are mutated by writing the other substrate's literal. R1 is mutated with true || matches!(…) (a) and false && matches!(…) (b), and R2 by mapping every label to K8s (a) or none (b). Unmutated, the two trees pass 1744/1744 and 1755/1755. The R1 test was added after this sweep, so I reran R1a/R1b on the final tests. After review round 1 (below), K1b, D12, R1a and R1b were rerun on this head.

# Mutation Test that fails (after) Before → after
D1–D6 each worker-selection predicate OR TRUE worker_selection_reads_only_direct_nodes passed → fails (each of the 6)
D7–D9 each placement-lookup predicate OR TRUE placement_lookups_read_only_direct_nodes passed → fails (each of the 3)
D10, D11 source / remote restore predicate OR TRUE restore_candidates_read_only_direct_nodes passed → fails (each)
D12 builder lookup OR TRUE builder_lookup_reads_only_direct_nodes passed → fails
N1a reactivation candidate OR TRUE node_registry_writes_only_direct_nodes passed → fails
N1b reactivation SET managed_by = 'k8s' same passed → fails
N2 insert VALUES (…, 'k8s', …) same passed → fails
N3, N4 update / hard-delete predicate OR TRUE same passed → fails (each)
K1a demotion candidate scan OR TRUE demotion_selects_only_kubernetes_nodes passed → fails
K1b demotion UPDATE guard OR TRUE demotion_rechecks_the_substrate_of_the_row_it_locks passed → fails
K2a upsert VALUES (…, 'space', …) 6 existing node-inventory tests (e.g. missing_k8s_inventory_demotes_only_stale_ready_nodes, pod_placement_does_not_downgrade_ready_node_state), plus 3 of the new ones fails → fails
K2b, K2c upsert ON CONFLICT guard / fallback SELECT OR TRUE inventory_upsert_writes_only_kubernetes_rows passed → fails (each)
K3 ownership re-check OR TRUE same passed → fails
S1a, S1b Kubernetes / Direct failure arm OR TRUE current_failure_reads_the_execution_record_of_the_sandbox_substrate passed → fails (each)
T1 n.managed_by != $1 OR TRUE reconcile_targets_require_a_live_registration_only_for_direct_nodes passed → fails
P1 n.managed_by != $2 OR TRUE template_delete_inventory_query_uses_direct_node_deleted_state fails → fails
R1a, R1b wake fallback for every / no label control_runtime_client_wakes_only_kubernetes_rows_after_an_unreachable_runtime passed → fails (each)
R2a, R2b console substrate for every / no label console_substrate_is_kubernetes_only_for_the_kubernetes_label passed → fails (each)

On main's tests, 2 of the 31 single mutations (K2a, P1) failed a test. On this PR's tests all 31 do, and each of the other 29 fails only the new test named for it.

Plan cache: literal vs bound

What I measured, and why. Of the converted statements, these run in Kubernetes deployments: S1 (every public sandbox read and list), K1 (every node-sync tick), K2/K3 (node inventory upserts), T1 and P1 (template admin and delete operations). S1 is the only hot path there. K1 is periodic and scans nodes, a small table whose managed_by is skewed toward 'k8s'. Both got the full plan-cache sequence. The rest got a generic-plan comparison.

Direct statements (D1–D12, N1–N4): not measured, and why binding is safe there. These don't run in Kubernetes deployments (the registry routes return 501 on that substrate), but several are hot on Direct: D2 on every admission check (cluster_admission/mod.rs, eligible_start_node_count), and the create path: D3 on every create from a cached template, D4 as its fallback when no worker is cached-ready, and D6 on pooled creates. Their substrate predicate is always on nodes.managed_by, never on sandboxes. On a Direct deployment's database that column is effectively single-valued ('space': Direct registers its nodes through its own registry, and no Kubernetes node sync runs there). A generic plan estimates managed_by = $n at 1/ndistinct, which then equals the literal's frequency (1.0), so bound and literal give the same row estimates and the same plan choice. nodes also holds one row per registered node, so these plans are driven by the joins on id/node_id and their other filters, not by this predicate. That is an argument from the statistics, not a measurement; I did not seed a Direct-shaped database.

Setup. A scratch database migrated from the test template on the repository's test image (PostgreSQL 16.15), with synthetic, deliberately skewed data: 60 orgs (one holds 40% of sandboxes, two hold 15% each), 400 nodes (10% 'space'), 200,000 sandboxes (5% 'space', 30% retained deleted tombstones, 3% failed), an active Kubernetes execution or a Direct route identity per placed sandbox of each substrate, observed states, and a desired-state facet row per placed sandbox (failed ones carry a matching failure episode), followed by ANALYZE. Each sequence ran on one connection with plan_cache_mode = auto: PREPARE once with the parameter types sqlx sends, then 30 × EXPLAIN (ANALYZE, TIMING OFF) EXECUTE, then pg_prepared_statements. "Literal" is the statement text on main, "bound" this PR's text.

Sequence (30 executions, one connection) Literal: generic / custom Literal: planning; execution median, ms Bound: generic / custom Bound: planning; execution median, ms
S1 by id: 30 sandboxes across orgs, substrates and lifecycles 25 / 5 0.011; 0.138 25 / 5 0.006; 0.100
S1 page: 30 first pages, large and small orgs shuffled 0 / 30 0.673; 0.524 0 / 30 0.833; 0.592
S1 page: 5 large-org pages, then 25 small-org pages with cursors 0 / 30 0.641; 0.555 0 / 30 0.609; 0.564
S1 page: 30 small-org first pages 0 / 30 0.722; 0.608 0 / 30 0.819; 0.698
K1 demotion: 30 ticks, each omitting 0–6 nodes from the membership snapshot (each in a rolled-back transaction) 25 / 5 0.187; 1.964 25 / 5 0.140; 1.931
  • Plan choice is the same in every sequence. I ran the S1 sequences three times. The generic/custom counts were identical each time, and the medians moved within the same 0.5–0.85 ms band for both texts: run-to-run noise, not a difference between them. The page statement builds a custom plan on every execution with the literal text too, so it already replans on main (its org_id filter is skewed in this data set). Binding doesn't change that.
  • S1's generic plans are identical, costs and row estimates included, for both the by-id and page statements. The only differences are $6/$7 instead of 'k8s'::text/'space'::text and the renumbered suffix parameters. Its substrate predicates sit in the select list's has_current_failure expression, not in WHERE, so they can't affect scans, joins or estimates.
  • K1's generic plan keeps its shape (same scans, hash join, sort, limit and nested loop). With the bound value, Postgres estimates managed_by = $4 at 1/ndistinct (200 of 400 nodes) instead of the literal's real frequency (360). The row estimates and costs drop accordingly, and the plan cache latches the generic plan after five executions either way.
  • K2, K3, T1, P1: generic plans are identical apart from the parameter in place of the literal.
  • Indexes on managed_by. Three exist, all full (not partial): idx_nodes_managed_by (managed_by), idx_nodes_role (managed_by, role) and idx_sandboxes_managed_by (managed_by). None of the measured plans uses them, literal or bound:
    • S1's substrate predicates are in the select list, so no index can serve them.
    • K1 reads nodes with a sequential scan in both plans (measured). The bound estimate (200 of 400) is lower than the literal's (360), which leans slightly toward an index, but at about half of a 400-row table the planner still seq-scans for both texts.
    • K2/K3 are primary-key lookups, and T1/P1 use managed_by != $n inside an OR, which a btree can't serve.
    • Because they are full indexes, a generic plan never loses an index whose predicate needed the literal to match, which is the risk with partial indexes.

No converted statement is materially worse bound, so no literal is kept as an exception. These are synthetic numbers on one PostgreSQL 16.15 instance; a production database's statistics decide the actual margins.

Characterization tests pass unchanged

The api characterization tests from #273, #290, #326 and #330 pass in every run below: controllers::substrate::kubernetes::adapter_characterization_tests (57) and controllers::sandbox::lifecycle::characterization_tests (16). This PR doesn't touch their files or scripted_test_doubles.rs.

Decisions made unattended

Decision Rationale
Bind the fixed variant everywhere, not substrate.managed_by() Each converted statement targets one substrate's rows whichever substrate runs: Direct's node reads and registry, the Kubernetes node inventory, and the per-row failure arms. None scopes to "the substrate I run on".
One parameter per statement for repeated uses (N1, K1, K2) The written value and the guards that protect it are then the same value by construction. Same choice as #383's bootstrap.
S1: substrate placeholders in the shared select, suffixes renumbered The two suffixes have different parameter counts, so numbering the new placeholders after the existing ones would differ per statement inside a shared const. Taking $6/$7 in the shared part keeps one select and moves only the suffix numbers; the binds follow the same order in both callers.
Leave the route-identity substrate_kind match in direct.rs The task keeps direct.rs edits to SQL statements while other work changes that file. It is a Rust match with an explicit error arm, so its behavior is already exhaustive; parsing it is a follow-up.
R1 and R2 parse a String instead of typing fields Sandbox.managed_by is a public model field and managed_by() returns &'static str on SubstrateAdapter; typing either changes interfaces beyond this PR.
New test modules beside each owner's existing Postgres tests Each reuses its module's fixtures (insert_ready_worker, persist_k8s_node_inventory, the Direct AppState, the failure-episode payload keys), so the twins differ only in managed_by. Sibling files keep already-large test files from growing and avoid conflicts.
R1 tested on the Kubernetes substrate The guard reads the row's label, and only Kubernetes deployments reach it with 'k8s' rows. A zero-capacity admission makes the fallback observable before it creates a pod.
Kind rerun after rebasing main gained production changes (#312, #165) after the first Kind run, so I reran Kind on a fresh cluster at this head.
Disk: cleanup step 1 between Kind runs See Validation.

Locator note

Converted statements' text changes, so their /*module_path:line*/ locators, sqlx statement-cache keys and query signatures change; statements below an added .bind line move to new locators. Module paths don't change.

Overlaps with open PRs

Review round 1 → changes

Finding Change
MINOR: the lock-interleaving wait in demotion_rechecks_the_substrate_of_the_row_it_locks matched any lock-waiting backend with the demotion's text, not one the writer blocks The writer reads its pg_backend_pid() before the demotion is spawned. A local bounded loop then waits for a backend with wait_event_type = 'Lock', that pid in pg_blocking_pids(pid) and the demotion's query text, as lifecycle/characterization_tests.rs and session_auth/tests.rs do. Both 5 s bounds stay. The shared wait_for_blocked_query helper is unchanged. K1b rerun on this head: fails this test at its assertion (the relabelled 'space' node is demoted), 1766/1767.
NIT: builder_lookup_reads_only_direct_nodes's select! had no bound of its own The select! over the accept and the dial is wrapped in a 5 s tokio::time::timeout, and the dial task is aborted afterwards. D12 rerun: fails this test, 1766/1767.
NIT: Plan cache gave only "don't run in Kubernetes" for the Direct statements; "no partial indexes" didn't name the indexes "Plan cache" now explains why binding is safe for the Direct statements (D1–D12, N1–N4) and their hot paths (D2 per admission check, D3/D4/D6 on creates): on a Direct database nodes.managed_by is effectively single-valued, so 1/ndistinct equals the literal's frequency, and nodes is small. It now names idx_nodes_managed_by, idx_nodes_role (managed_by, role) and idx_sandboxes_managed_by and says why none changes these plans.
NIT: the R1 test added 73 lines to kubernetes/tests.rs Moved to the sibling module kubernetes/tests/substrate_scope.rs. tests.rs now differs from main only by mod substrate_scope;. The test now reads the row back with fetch_sandbox instead of building a Sandbox in memory. R1a and R1b rerun: both fail this test, 1766/1767.
NIT: say explicitly that space-conventions.md needs the proposed sentence after this PR and #383 merge, and why it's not here The "Docs" bullet now says so: lines 194–195 on main ("direct.rs has about a dozen") need it, and it's left out only to avoid conflicting with #383's edit of that paragraph.

No production code changed in this round, so Kind wasn't rerun; the Kind rows below are from 4e5bcf3.

Review round 2 → changes

Finding Change
NIT: the K1 index bullet said the lower bound estimate made an index "even less attractive" Reworded: the bound estimate (200 of 400) is lower, which leans slightly toward an index, but at about half of a 400-row table the planner still seq-scans for both texts (measured).
NIT: the demotion test's poll ignored the demotion task The 5 s timeout now wraps a tokio::select! over &mut demotion and the pid-tied poll, as lifecycle/characterization_tests.rs does. If the demotion finishes without ever waiting on the writer, the test fails with the demotion's own result, not with "never waited".
NIT: timeout paths left spawned tasks running (dropping a JoinHandle detaches it) Both the demotion task and the builder dial are held in tokio_util::task::AbortOnDropHandle. They are aborted on every exit: the poll failing, the join timing out, or the builder bound's expect panicking. The builder test drops its handle once the outcome is known. tokio-util was already an api dependency, and the workspace enables its rt feature.

Validation for this round (tests and PR body only; no production code changed, so no Kind): fmt pass; cargo clippy -p pplx-space-api --all-targets --all-features --locked -- -D warnings pass; pre-commit (Gitleaks) pass; demotion_rechecks_the_substrate_of_the_row_it_locks and builder_lookup_reads_only_direct_nodes passed in each of three runs (SPACE_CI_REQUIRE_TEST_DEPS=1); the K1b and D12 mutations, once each against the whole suite, each fail only their test (1766/1767; K1b at its assertion, the relabelled 'space' node demoted).

Validation

Head e400a90. Round 2 changed only the two tests above; the full-suite rows below were run at 8891dcf. Base 232cea1. It still merges cleanly with current main, so it wasn't rebased again. Before review round 1 the commit was rebased 0f1d3be → 9254f09 → 232cea1, and git range-diff showed it unchanged each time. Kind ran on 4e5bcf3 (this commit on 9254f09, before round 1). Its production code matches this head; the head differs only by #491's one-line internal-api dev-dependency change and round 1's test-only edits.

Check Result
cargo fmt --all -- --check pass
cargo clippy --workspace --all-targets --all-features --locked -- -D warnings pass
pre-commit (Gitleaks) at commit, and pre-commit run gitleaks --from-ref origin/main --to-ref HEAD pass
space-public-review on the staged change and this text no findings in the reviewed scope (synthetic names, loopback and RFC 1918 addresses, a synthetic bucket name)
cargo nextest run --locked -p pplx-space-api --all-features --retries 0, SPACE_CI_REQUIRE_TEST_DEPS=1, three runs 8891dcf: 1767/1767 passed in each of runs 1, 2 and 3. The one skipped test is the #[ignore]d local occupancy benchmark. Each run includes the 73 characterization tests, the 12 new tests and the extended P1 test. Earlier heads: 877ebdd 1767/1767 three times; 2e5e56d (on 0f1d3be) 1756/1756 three times.
Mutation reruns after round 1 (8891dcf) K1b, D12, R1a and R1b each fail only their test (1766/1767)
scripts/check-rust-error-policy.sh; scripts/source_guards/*.py pass (thin_service_mains.py exits 0 with advisory notes on crates/apps/*/main.rs files this PR doesn't touch)
scripts/check_no_inline_rust_test_modules.py flags only crates/libs/observability/src/{otlp_endpoint,selection}.rs, untouched here: PRE-EXISTING
cargo arch exit 0. 66 advisory and 0 required findings, the same set as main. Growth: in five files already over advisory listed_no_growth caps on main, from the added .bind lines: direct.rs 3913 → 3925, node_maintenance.rs 1790 → 1794, routes/public/sandboxes.rs 1252 → 1258, spaced.rs 1146 → 1151, routes/admin/templates.rs 2264 → 2266.
Predicate removal; plan cache see Evidence
Kind: disk Before the first bring-up: Docker data device 22 GB free, no cleanup. Before the rerun: 16 GB free, no cleanup. That bring-up left 6.9 GB, below the 15 GB floor, so I tore the cluster down and ran cleanup step 1: docker builder prune -af reclaimed 18.59 GB (15.2 GB on the device) and docker image prune -f 0 B, leaving 28 GB. No later step was needed. During the final typed suite free space dipped to 5.6 GB with no disk-related failure.
Kind: fresh cluster, build provenance (final run, 4e5bcf3) No cluster existed. smoke-kind-e2e.sh (BOOTSTRAP_TIMEOUT=1800, DATA_PLANE_URL, as CI sets) created space-local and started Tilt from this checkout. The api, internal-api and cluster-controller services log version: 4e5bcf33fc19. The deployed /space/bin/space-api-svc and /space/bin/space-cluster-controller match the artifact image's binaries by sha256 (e6819374…, d0276ce9…), before and after the suites. The api binary contains s.managed_by = $6/$7 and n.managed_by != $1, and no s.managed_by = 'k8s', s.managed_by = 'space', n.managed_by = 'space' or != 'space'. Its only n.managed_by = 'k8s' is substrate_adapter.rs's (out of scope). The cluster controller, which runs node sync and the template operations, contains candidate.managed_by = $4, VALUES ($1, $2::INET, $3, $6, $4) and n.managed_by != $2, and not their literal forms.
smoke-kind-e2e.sh (start) pass (runtime-manager checks and 7/7)
smoke-kind-typed.sh all (unsharded, CI's env: admin token file, cold-receipts dir, host dm targets loaded) ECW 78/78, native 24/24, 0 warned, failed, inconclusive or quarantined
smoke-kind-sandbox-fork.py pass (one source, four forked children)
smoke-kind-e2e.sh (end) pass (runtime-manager checks and 7/7)
Kind, earlier run (dfa3239, this commit on 0f1d3be, before the R1 test was added) fresh cluster, same provenance checks (version: dfa32396256a, deployed binaries match the artifact). e2e 7/7, typed ECW 78/78 and native 24/24, fork pass, e2e 7/7.
Cleanup teardown.sh on the task-owned cluster; scratch Postgres container and scratch checkouts removed

🤖 Generated with Claude Code

@ac-pplx
ac-pplx force-pushed the ac/substrate-kind-sql-api branch from 877ebdd to 8891dcf Compare October 15, 2026 00:43
Control-plane Postgres `sandboxes.managed_by` and `nodes.managed_by` hold
the substrate that owns a row, `'k8s'` or `'space'`. The api crate scoped
and wrote that value as a raw literal in 22 production SQL statements and
compared it as a string in two places. Those statements now bind
`SubstrateKind` (`managed_by = $n` with `.bind(SubstrateKind::Kubernetes)`
or `::Space`), each still one whole statement, and the comparisons parse
the label and match the variant. `SubstrateKind` encodes as the column's
text, so every statement matches and writes exactly the rows it did. No
schema, wire format, metric or configuration changes.

- Direct substrate (`direct.rs`, 12 statements): node selection for
  create, pooled adoption, pull-on-create, placement lookups, restore
  candidates and builder lookup bind `Space`. Only the SQL text and its
  binds change in this file.
- Direct node registry (`routes/admin/node_maintenance.rs`, 4): node
  reactivation, insert, update and hard delete bind `Space`; the
  reactivation statement uses one parameter for its filter and its
  `SET managed_by`.
- Kubernetes node inventory (`kubernetes/store/node_inventory.rs`, 3):
  demotion of unobserved nodes and the node upsert use one parameter for
  every use in the statement; the ownership re-check binds its own.
- Public sandbox read and list (`routes/public/sandboxes.rs`): the
  failure-episode projection binds `Kubernetes` and `Space` as `$6` and
  `$7` in the shared select, so the by-id and page suffixes renumber
  their parameters from `$8`, with binds in the same order.
- Template reconcile targets (`routes/admin/templates.rs`) and template
  delete inventory (`public_template_operations.rs`) bind `Space`.
- `spaced.rs`'s dormant-wake fallback and the console's `/me` substrate
  (`session_auth.rs`) parse the label as `SubstrateKind` and match
  `Kubernetes`; any other value, including an unparseable one, behaves as
  before.

Tests: each converted statement now has a Postgres-backed test in which
a row of the statement's substrate is selected or written and an
otherwise identical row of the other substrate is not. New modules:
`direct/tests/substrate_scope.rs`,
`k8s_store_postgres_tests/substrate_scope_tests.rs` (including a
concurrent relabel the demotion UPDATE must re-check, committed once the
writer's pid blocks it),
`node_maintenance/tests/substrate_scope.rs`,
`templates/tests/substrate_scope.rs` and
`routes/public/sandboxes/tests/substrate_scope.rs`; the template delete
inventory test gains a Kubernetes twin of its retired Direct node,
`session_auth` gains a label table, and
`kubernetes/tests/substrate_scope.rs` shows that only a `'k8s'` dormant
row takes `control_runtime_client`'s wake fallback. Test fixtures keep
writing literal `managed_by` values.

Left as is: the Direct route identity's `substrate_kind` match in
`direct.rs` (a Rust match on a column read in Direct-owned code outside
the SQL statements), `DirectSubstrate::managed_by()`'s `"space"`, the
Kubernetes adapter's literals (`substrate_adapter.rs`) and the shared
`routable_execution_or_non_k8s` fragment.

Converted statements' `/*module_path:line*/` locators, sqlx
statement-cache keys and query signatures change with their text.
@ac-pplx
ac-pplx force-pushed the ac/substrate-kind-sql-api branch from 8891dcf to e400a90 Compare October 15, 2026 00:58
@ac-pplx
ac-pplx marked this pull request as ready for review October 15, 2026 13:50

@nickdujay-pplx nickdujay-pplx left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Approved at e400a90. Independently ran 160 targeted tests with real PostgreSQL; all passed, including substrate ownership, concurrent relabel/demotion, public read/list behavior, and Direct selection/registry cases. Checked the SQL bind order, shared-select suffix renumbering, enum text encoding, and existing index predicates. CI is green; no blocking correctness findings.

Deployment and rollback: no schema, persisted-format, configuration, or inter-service protocol changes. The enum binds the same text values as the old literals, and the Rust comparisons preserve the existing fallback behavior. Old and new processes can coexist, and reverting the binaries does not require a data conversion or migration rollback. No new service deployment ordering is required.

Both this patch and #505 apply cleanly on top of the other. They were tested separately on their exact reviewed heads; I did not rerun the Kind suites or the author's synthetic plan-cache benchmark.

@ac-pplx
ac-pplx added this pull request to the merge queue Oct 15, 2026
Merged via the queue into main with commit ff36345 Oct 15, 2026
20 checks passed
@ac-pplx
ac-pplx deleted the ac/substrate-kind-sql-api branch October 15, 2026 15:23
ac-pplx added a commit that referenced this pull request Oct 16, 2026
Control-plane Postgres `sandboxes.managed_by` and `nodes.managed_by` hold the
substrate that owns a row, 'k8s' or 'space'. After #373, #383 and #506 the
remaining production literals were the shared routability fragment, the
Kubernetes adapter's builder lookup and the Direct route-identity match. They
now bind `SubstrateKind` or match its parsed variant, and a new archcheck
instance keeps new literals out of production code. No schema, wire format,
metric or configuration changes, and no behavior change.

- common-svc: `routable_execution_or_non_k8s(alias, kubernetes_placeholder)`
  writes `{alias}.managed_by <> {placeholder}` instead of `<> 'k8s'`. Each
  caller binds `SubstrateKind::Kubernetes` at the placeholder it numbers
  after its own parameters: the transient sweeper's aged-transient list and
  restoring converge (`$5`), internal-api's restored-state convergence (`$4`)
  and the api's wake-completion CAS (`$2`).
- api Kubernetes adapter: `builder_client_for_node` binds
  `n.managed_by = $2`.
- api Direct: the route identity's `substrate_kind` text is parsed as
  `SubstrateKind` in `route_identity_substrate_kind`, which maps exactly
  'space' and 'k8s' as before and rejects any other text with the same
  error.
- ARCHITECTURE.toml: `pattern_ratchet.substrate_literals` matches SQL
  comparisons of `managed_by` with a 'k8s'/'space' literal (=, <>, !=,
  IS [NOT] DISTINCT FROM, [NOT] IN; any case and alias) and Rust comparisons
  or match arms on "k8s"/"space". It allows test-support modules and
  fixtures that archcheck's filename-based test rule counts as production.
  On the parent commit it matches exactly the four converted lines; on this
  commit it matches nothing, so it has no baseline table and every behaviour
  is required.
- Tests: Postgres-backed twins for each converted statement. A new
  Kubernetes builder-lookup test, a unit table for the route-identity
  labels, and an otherwise identical 'space' row added to the sweeper's
  converge, internal-api's restored convergence and the wake-completion
  redrive tests. Test callers of the sweeper's shared builders bind the
  extra parameter.
- Docs: `docs/rust/space-conventions.md`, the substrate AGENTS.md and the
  space-rust-development skill say every crate's production SQL binds the
  value; Rust parses it and matches the variant (or calls the `SubstrateKind`
  method that names the question) and never compares the label string; and
  the archcheck rule enforces it. Plugin version 0.2.33 -> 0.2.34.

Converted statements' text changes, so their sqlx statement-cache keys and
query signatures change; statements below an added `.bind` move to new
`sql!` locators.
ac-pplx added a commit that referenced this pull request Oct 17, 2026
Control-plane Postgres `sandboxes.managed_by` and `nodes.managed_by` hold the
substrate that owns a row, 'k8s' or 'space'. After #373, #383 and #506 the
remaining production literals were the shared routability fragment, the
Kubernetes adapter's builder lookup and the Direct route-identity match, plus
three statements that landed on main since: internal-api's publisher-node
lookup and Direct's create-target claim and release. They now bind
`SubstrateKind` or match its parsed variant, and a new archcheck instance
keeps new literals out of production code. No schema, wire format, metric or
configuration changes, and no behavior change.

- common-svc: `routable_execution_or_non_k8s(alias, kubernetes_placeholder)`
  writes `{alias}.managed_by <> {placeholder}` instead of `<> 'k8s'`. Each
  caller binds `SubstrateKind::Kubernetes` at the placeholder it numbers
  after its own parameters: the transient sweeper's aged-transient list and
  restoring converge (`$5`), internal-api's restored-state convergence (`$4`)
  and the api's wake-completion CAS (`$2`).
- api Kubernetes adapter: `builder_client_for_node` binds
  `n.managed_by = $2`.
- internal-api: `publisher_node_for_identity` binds `n.managed_by = $3`
  (`Kubernetes`).
- api Direct: `claim_create_target` and `release_create_target` bind
  `managed_by = $3` (`Space`).
- api Direct: the route identity's `substrate_kind` text is parsed as
  `SubstrateKind` in `route_identity_substrate_kind`, which maps exactly
  'space' and 'k8s' as before and rejects any other text with the same
  error.
- ARCHITECTURE.toml: `pattern_ratchet.substrate_literals` matches SQL
  comparisons of `managed_by` with a 'k8s'/'space' literal (=, <>, !=,
  IS [NOT] DISTINCT FROM, [NOT] IN; any case and alias) and Rust comparisons
  or match arms on "k8s"/"space". It allows test-support modules and
  fixtures that archcheck's filename-based test rule counts as production.
  On the parent commit it matches exactly the seven converted lines; on
  this commit it matches nothing, so it has no baseline table and every
  behaviour is required.
- Tests: Postgres-backed twins for each converted statement. New
  Kubernetes builder-lookup and Direct create-target tests, a unit table for
  the route-identity labels, a relabelled-builder case in the tenant
  publication table, and an otherwise identical 'space' row added to the
  sweeper's converge, internal-api's restored convergence and the
  wake-completion redrive tests. Test callers of the sweeper's shared builders bind the
  extra parameter.
- Docs: `docs/rust/space-conventions.md`, the substrate AGENTS.md and the
  space-rust-development skill say every crate's production SQL binds the
  value; Rust parses it and matches the variant (or calls the `SubstrateKind`
  method that names the question) and never compares the label string; and
  the archcheck rule enforces it. Plugin version 0.2.33 -> 0.2.34.

Converted statements' text changes, so their sqlx statement-cache keys and
query signatures change; statements below an added `.bind` move to new
`sql!` locators.
ac-pplx added a commit that referenced this pull request Oct 17, 2026
Control-plane Postgres `sandboxes.managed_by` and `nodes.managed_by` hold the
substrate that owns a row, 'k8s' or 'space'. After #373, #383 and #506 the
remaining production literals were the shared routability fragment, the
Kubernetes adapter's builder lookup and the Direct route-identity match, plus
Direct's create-target claim and release, which landed on main since. They
now bind `SubstrateKind` or match its parsed variant, and a new archcheck
instance keeps new literals out of production code. No schema, wire format,
metric or configuration changes, and no behavior change.

- common-svc: `routable_execution_or_non_k8s(alias, kubernetes_placeholder)`
  writes `{alias}.managed_by <> {placeholder}` instead of `<> 'k8s'`. Each
  caller binds `SubstrateKind::Kubernetes` at the placeholder it numbers
  after its own parameters: the transient sweeper's aged-transient list and
  restoring converge (`$5`), internal-api's restored-state convergence (`$4`)
  and the api's wake-completion CAS (`$2`).
- api Kubernetes adapter: `builder_client_for_node` binds
  `n.managed_by = $2`.
- api Direct: `claim_create_target` and `release_create_target` bind
  `managed_by = $3` (`Space`).
- api Direct: the route identity's `substrate_kind` text is parsed as
  `SubstrateKind` in `route_identity_substrate_kind`, which maps exactly
  'space' and 'k8s' as before and rejects any other text with the same
  error.
- ARCHITECTURE.toml: `pattern_ratchet.substrate_literals` matches SQL
  comparisons of `managed_by` with a 'k8s'/'space' literal (=, <>, !=,
  IS [NOT] DISTINCT FROM, [NOT] IN; any case and alias) and Rust comparisons
  or match arms on "k8s"/"space". It allows test-support modules and
  fixtures that archcheck's filename-based test rule counts as production.
  On the parent commit it matches exactly the six converted lines; on
  this commit it matches nothing, so it has no baseline table and every
  behaviour is required.
- Tests: Postgres-backed twins for each converted statement. New
  Kubernetes builder-lookup and Direct create-target tests, a unit table for
  the route-identity labels, and an otherwise identical 'space' row added to
  the sweeper's converge, internal-api's restored convergence and the
  wake-completion redrive tests. Test callers of the sweeper's shared
  builders bind the extra parameter.
- Docs: `docs/rust/space-conventions.md`, the substrate AGENTS.md and the
  space-rust-development skill say every crate's production SQL binds the
  value; Rust parses it and matches the variant (or calls the `SubstrateKind`
  method that names the question) and never compares the label string; and
  the archcheck rule enforces it. Plugin version 0.2.33 -> 0.2.34.

Converted statements' text changes, so their sqlx statement-cache keys and
query signatures change; statements below an added `.bind` move to new
`sql!` locators.
ac-pplx added a commit that referenced this pull request Oct 17, 2026
Control-plane Postgres `sandboxes.managed_by` and `nodes.managed_by` hold the
substrate that owns a row, 'k8s' or 'space'. After #373, #383 and #506 the
remaining production literals were the shared routability fragment, the
Kubernetes adapter's builder lookup and the Direct route-identity match, plus
Direct's create-target claim and release, which landed on main since. They
now bind `SubstrateKind` or match its parsed variant, and a new archcheck
instance keeps new literals out of production code. No schema, wire format,
metric or configuration changes, and no behavior change.

- common-svc: `routable_execution_or_non_k8s(alias, kubernetes_placeholder)`
  writes `{alias}.managed_by <> {placeholder}` instead of `<> 'k8s'`. Each
  caller binds `SubstrateKind::Kubernetes` at the placeholder it numbers
  after its own parameters: the transient sweeper's aged-transient list and
  restoring converge (`$5`), internal-api's restored-state convergence (`$4`)
  and the api's wake-completion CAS (`$2`).
- api Kubernetes adapter: `builder_client_for_node` binds
  `n.managed_by = $2`.
- api Direct: `claim_create_target` and `release_create_target` bind
  `managed_by = $3` (`Space`).
- api Direct: the route identity's `substrate_kind` text is parsed as
  `SubstrateKind` in `route_identity_substrate_kind`, which maps exactly
  'space' and 'k8s' as before and rejects any other text with the same
  error.
- ARCHITECTURE.toml: `pattern_ratchet.substrate_literals` matches SQL
  comparisons of `managed_by` with a 'k8s'/'space' literal (=, <>, !=,
  IS [NOT] DISTINCT FROM, [NOT] IN; any case and alias) and Rust comparisons
  or match arms on "k8s"/"space". It allows test-support modules and
  fixtures that archcheck's filename-based test rule counts as production.
  On the parent commit it matches exactly the six converted lines; on
  this commit it matches nothing, so it has no baseline table and every
  behaviour is required.
- Tests: Postgres-backed twins for each converted statement. New
  Kubernetes builder-lookup and Direct create-target tests, a unit table for
  the route-identity labels, and an otherwise identical 'space' row added to
  the sweeper's converge, internal-api's restored convergence and the
  wake-completion redrive tests. Test callers of the sweeper's shared
  builders bind the extra parameter.
- Docs: `docs/rust/space-conventions.md`, the substrate AGENTS.md and the
  space-rust-development skill say every crate's production SQL binds the
  value; Rust parses it and matches the variant (or calls the `SubstrateKind`
  method that names the question) and never compares the label string; and
  the archcheck rule enforces it. Plugin version 0.2.33 -> 0.2.34.

Converted statements' text changes, so their sqlx statement-cache keys and
query signatures change; statements below an added `.bind` move to new
`sql!` locators.
ac-pplx added a commit that referenced this pull request Oct 17, 2026
Control-plane Postgres `sandboxes.managed_by` and `nodes.managed_by` hold the
substrate that owns a row, 'k8s' or 'space'. After #373, #383 and #506 the
remaining production literals were the shared routability fragment, the
Kubernetes adapter's builder lookup and the Direct route-identity match, plus
Direct's create-target claim and release, which landed on main since. They
now bind `SubstrateKind` or match its parsed variant, and a new archcheck
instance keeps new literals out of production code. No schema, wire format,
metric or configuration changes, and no behavior change.

- common-svc: `routable_execution_or_non_k8s(alias, kubernetes_placeholder)`
  writes `{alias}.managed_by <> {placeholder}` instead of `<> 'k8s'`. Each
  caller binds `SubstrateKind::Kubernetes` at the placeholder it numbers
  after its own parameters: the transient sweeper's aged-transient list and
  restoring converge (`$5`), internal-api's restored-state convergence (`$4`)
  and the api's wake-completion CAS (`$2`).
- api Kubernetes adapter: `builder_client_for_node` binds
  `n.managed_by = $2`.
- api Direct: `claim_create_target` and `release_create_target` bind
  `managed_by = $3` (`Space`).
- api Direct: the route identity's `substrate_kind` text is parsed as
  `SubstrateKind` in `route_identity_substrate_kind`, which maps exactly
  'space' and 'k8s' as before and rejects any other text with the same
  error.
- ARCHITECTURE.toml: `pattern_ratchet.substrate_literals` matches SQL
  comparisons of `managed_by` with a 'k8s'/'space' literal (=, <>, !=,
  IS [NOT] DISTINCT FROM, [NOT] IN; any case and alias) and Rust comparisons
  or match arms on "k8s"/"space". It allows test-support modules and
  fixtures that archcheck's filename-based test rule counts as production.
  On the parent commit it matches exactly the six converted lines; on
  this commit it matches nothing, so it has no baseline table and every
  behaviour is required.
- Tests: Postgres-backed twins for each converted statement. New
  Kubernetes builder-lookup and Direct create-target tests, a unit table for
  the route-identity labels, and an otherwise identical 'space' row added to
  the sweeper's converge, internal-api's restored convergence and the
  wake-completion redrive tests. Test callers of the sweeper's shared
  builders bind the extra parameter.
- Docs: `docs/rust/space-conventions.md`, the substrate AGENTS.md and the
  space-rust-development skill say every crate's production SQL binds the
  value; Rust parses it and matches the variant (or calls the `SubstrateKind`
  method that names the question) and never compares the label string; and
  the archcheck rule enforces it. Plugin version 0.2.33 -> 0.2.34.

Converted statements' text changes, so their sqlx statement-cache keys and
query signatures change; statements below an added `.bind` move to new
`sql!` locators.
ac-pplx added a commit that referenced this pull request Oct 17, 2026
Control-plane Postgres `sandboxes.managed_by` and `nodes.managed_by` hold the
substrate that owns a row, 'k8s' or 'space'. After #373, #383 and #506 the
remaining production literals were the shared routability fragment, the
Kubernetes adapter's builder lookup and the Direct route-identity match, plus
Direct's create-target claim and release, which landed on main since. They
now bind `SubstrateKind` or match its parsed variant, and a new archcheck
instance keeps new literals out of production code. No schema, wire format,
metric or configuration changes, and no behavior change.

- common-svc: `routable_execution_or_non_k8s(alias, kubernetes_placeholder)`
  writes `{alias}.managed_by <> {placeholder}` instead of `<> 'k8s'`. Each
  caller binds `SubstrateKind::Kubernetes` at the placeholder it numbers
  after its own parameters: the transient sweeper's aged-transient list and
  restoring converge (`$5`), internal-api's restored-state convergence (`$4`)
  and the api's wake-completion CAS (`$2`).
- api Kubernetes adapter: `builder_client_for_node` binds
  `n.managed_by = $2`.
- api Direct: `claim_create_target` and `release_create_target` bind
  `managed_by = $3` (`Space`).
- api Direct: the route identity's `substrate_kind` text is parsed as
  `SubstrateKind` in `route_identity_substrate_kind`, which maps exactly
  'space' and 'k8s' as before and rejects any other text with the same
  error.
- ARCHITECTURE.toml: `pattern_ratchet.substrate_literals` catches the
  common literal forms in production code: SQL comparisons of `managed_by`
  with 'k8s'/'space' (=, <>, !=, IS [NOT] DISTINCT FROM, [NOT] IN, = ANY,
  same-line CASE ... WHEN; any case, alias, prefix, cast or wrapping call),
  and Rust comparisons, label-string matches and match arms on "k8s"/"space"
  beside a substrate name. Its comment lists what it misses and its false
  positives. It allows test-support modules and fixtures that archcheck's
  filename-based test rule counts as production. On the parent commit it
  matches the seven lines this commit converts; on this commit it matches
  nothing, so it has no baseline table. no_crossing, listed_no_growth and
  stale_entries are required; no_new_entries and no_raised_caps are
  advisory, so a deliberate exception is a reviewed baseline entry.
- Tests: Postgres-backed twins for each converted statement. New
  Kubernetes builder-lookup and Direct create-target tests, a unit table for
  the route-identity labels, and an otherwise identical 'space' row added to
  the sweeper's converge, internal-api's restored convergence and the
  wake-completion redrive tests. Test callers of the sweeper's shared
  builders bind the extra parameter.
- Docs: `docs/rust/space-conventions.md`, the substrate AGENTS.md and the
  space-rust-development skill say every crate's production SQL binds the
  value; in Rust a decision about a row's substrate calls a `SubstrateKind`
  method named for the question, a fact about the running substrate reads
  `capabilities()`, and code never compares the label string; and the
  archcheck rule catches the common literal forms. Plugin version
  0.2.33 -> 0.2.34.

Converted statements' text changes, so their sqlx statement-cache keys and
query signatures change; statements below an added `.bind` move to new
`sql!` locators.
ac-pplx added a commit that referenced this pull request Oct 17, 2026
Control-plane Postgres `sandboxes.managed_by` and `nodes.managed_by` hold the
substrate that owns a row, 'k8s' or 'space'. After #373, #383 and #506 the
remaining production literals were the shared routability fragment, the
Kubernetes adapter's builder lookup and the Direct route-identity match, plus
Direct's create-target claim and release, which landed on main since. They
now bind `SubstrateKind` or match its parsed variant, and a new archcheck
instance keeps new literals out of production code. No schema, wire format,
metric or configuration changes, and no behavior change.

- common-svc: `routable_execution_or_non_k8s(alias, kubernetes_placeholder)`
  writes `{alias}.managed_by <> {placeholder}` instead of `<> 'k8s'`. Each
  caller binds `SubstrateKind::Kubernetes` at the placeholder it numbers
  after its own parameters: the transient sweeper's aged-transient list and
  restoring converge (`$5`), internal-api's restored-state convergence (`$4`)
  and the api's wake-completion CAS (`$2`).
- api Kubernetes adapter: `builder_client_for_node` binds
  `n.managed_by = $2`.
- api Direct: `claim_create_target` and `release_create_target` bind
  `managed_by = $3` (`Space`).
- api Direct: the route identity's `substrate_kind` text is parsed as
  `SubstrateKind` in `route_identity_substrate_kind`, which maps exactly
  'space' and 'k8s' as before and rejects any other text with the same
  error.
- ARCHITECTURE.toml: `pattern_ratchet.substrate_literals` catches the
  common literal forms in production code: SQL comparisons of `managed_by`
  with 'k8s'/'space' (=, <>, !=, IS [NOT] DISTINCT FROM, [NOT] IN, = ANY,
  same-line CASE ... WHEN; any case, alias, prefix, quoting, cast or wrapping
  call), and Rust comparisons with the label (raw strings included), a
  `NODE_MANAGED_BY_*` const or a variant's `.as_str()`/`.to_string()`,
  matches on the label string, and match arms on "k8s"/"space" beside a
  substrate name. Its comment lists what it misses and its false positives. It allows test-support modules and fixtures that archcheck's
  filename-based test rule counts as production. On the parent commit it
  matches the seven lines this commit converts; on this commit it matches
  nothing, so it has no baseline table. no_crossing, listed_no_growth and
  stale_entries are required; no_new_entries and no_raised_caps are
  advisory, so a deliberate exception is a reviewed baseline entry.
- Tests: Postgres-backed twins for each converted statement. New
  Kubernetes builder-lookup and Direct create-target tests, a unit table for
  the route-identity labels, and an otherwise identical 'space' row added to
  the sweeper's converge, internal-api's restored convergence and the
  wake-completion redrive tests. Test callers of the sweeper's shared
  builders bind the extra parameter.
- Docs: `docs/rust/space-conventions.md`, the substrate AGENTS.md and the
  space-rust-development skill say every crate's production SQL binds the
  value; in Rust a decision about a row's substrate calls a `SubstrateKind`
  method named for the question, a fact about the running substrate reads
  `capabilities()`, and code never compares the label string; and the
  archcheck rule catches the common literal forms. Plugin version
  0.2.33 -> 0.2.34.

Converted statements' text changes, so their sqlx statement-cache keys and
query signatures change; statements below an added `.bind` move to new
`sql!` locators.
ac-pplx added a commit that referenced this pull request Oct 17, 2026
Control-plane Postgres `sandboxes.managed_by` and `nodes.managed_by` hold the
substrate that owns a row, 'k8s' or 'space'. After #373, #383 and #506 the
remaining production literals were the shared routability fragment, the
Kubernetes adapter's builder lookup and the Direct route-identity match, plus
Direct's create-target claim and release, which landed on main since. They
now bind `SubstrateKind` or match its parsed variant, and a new archcheck
instance keeps new literals out of production code. No schema, wire format,
metric or configuration changes, and no behavior change.

- common-svc: `routable_execution_or_non_k8s(alias, kubernetes_placeholder)`
  writes `{alias}.managed_by <> {placeholder}` instead of `<> 'k8s'`. Each
  caller binds `SubstrateKind::Kubernetes` at the placeholder it numbers
  after its own parameters: the transient sweeper's aged-transient list and
  restoring converge (`$5`), internal-api's restored-state convergence (`$4`)
  and the api's wake-completion CAS (`$2`).
- api Kubernetes adapter: `builder_client_for_node` binds
  `n.managed_by = $2`.
- api Direct: `claim_create_target` and `release_create_target` bind
  `managed_by = $3` (`Space`).
- api Direct: the route identity's `substrate_kind` text is parsed as
  `SubstrateKind` in `route_identity_substrate_kind`, which maps exactly
  'space' and 'k8s' as before and rejects any other text with the same
  error.
- ARCHITECTURE.toml: `pattern_ratchet.substrate_literals` catches the common
  literal forms in production code: SQL comparisons of `managed_by` with
  'k8s'/'space' (=, <>, !=, IS [NOT] DISTINCT FROM, [NOT] IN, = ANY,
  same-line CASE ... WHEN; any case, alias, prefix, quoting, cast or wrapping
  call), and Rust comparisons with the label (raw strings included), a
  `NODE_MANAGED_BY_*` const or a variant's `.as_str()`/`.to_string()`,
  matches on the label string, and match arms on "k8s"/"space" beside a
  substrate name. Its comment lists what it misses and its false positives.
  It allows test-support modules and fixtures that archcheck's filename-based
  test rule counts as production. On the parent commit it matches the seven
  lines this commit converts; on this commit it matches nothing, so it has no
  baseline table. no_crossing, listed_no_growth and stale_entries are
  required; no_new_entries and no_raised_caps are advisory, so a deliberate
  exception is a reviewed baseline entry.
- Tests: Postgres-backed twins for each converted statement. New
  Kubernetes builder-lookup and Direct create-target tests, a unit table for
  the route-identity labels, and an otherwise identical 'space' row added to
  the sweeper's converge, internal-api's restored convergence and the
  wake-completion redrive tests. Test callers of the sweeper's shared
  builders bind the extra parameter.
- Docs: `docs/rust/space-conventions.md`, the substrate AGENTS.md and the
  space-rust-development skill say every crate's production SQL binds the
  value; in Rust a decision about a row's substrate calls a `SubstrateKind`
  method named for the question, a fact about the running substrate reads
  `capabilities()`, and code never compares the label string; and the
  archcheck rule catches the common literal forms. Plugin version
  0.2.33 -> 0.2.34.

Converted statements' text changes, so their sqlx statement-cache keys and
query signatures change; statements below an added `.bind` move to new
`sql!` locators.
meefs pushed a commit to meefs/SPACE that referenced this pull request Oct 18, 2026
…erplexityai#611)

Control-plane Postgres `sandboxes.managed_by` and `nodes.managed_by` hold the
substrate that owns a row, 'k8s' or 'space'. After perplexityai#373, perplexityai#383 and perplexityai#506 the
remaining production literals were the shared routability fragment, the
Kubernetes adapter's builder lookup and the Direct route-identity match, plus
Direct's create-target claim and release, which landed on main since. They
now bind `SubstrateKind` or match its parsed variant, and a new archcheck
instance keeps new literals out of production code. No schema, wire format,
metric or configuration changes, and no behavior change.

- common-svc: `routable_execution_or_non_k8s(alias, kubernetes_placeholder)`
  writes `{alias}.managed_by <> {placeholder}` instead of `<> 'k8s'`. Each
  caller binds `SubstrateKind::Kubernetes` at the placeholder it numbers
  after its own parameters: the transient sweeper's aged-transient list and
  restoring converge (`$5`), internal-api's restored-state convergence (`$4`)
  and the api's wake-completion CAS (`$2`).
- api Kubernetes adapter: `builder_client_for_node` binds
  `n.managed_by = $2`.
- api Direct: `claim_create_target` and `release_create_target` bind
  `managed_by = $3` (`Space`).
- api Direct: the route identity's `substrate_kind` text is parsed as
  `SubstrateKind` in `route_identity_substrate_kind`, which maps exactly
  'space' and 'k8s' as before and rejects any other text with the same
  error.
- ARCHITECTURE.toml: `pattern_ratchet.substrate_literals` catches the common
  literal forms in production code: SQL comparisons of `managed_by` with
  'k8s'/'space' (=, <>, !=, IS [NOT] DISTINCT FROM, [NOT] IN, = ANY,
  same-line CASE ... WHEN; any case, alias, prefix, quoting, cast or wrapping
  call), and Rust comparisons with the label (raw strings included), a
  `NODE_MANAGED_BY_*` const or a variant's `.as_str()`/`.to_string()`,
  matches on the label string, and match arms on "k8s"/"space" beside a
  substrate name. Its comment lists what it misses and its false positives.
  It allows test-support modules and fixtures that archcheck's filename-based
  test rule counts as production. On the parent commit it matches the seven
  lines this commit converts; on this commit it matches nothing, so it has no
  baseline table. no_crossing, listed_no_growth and stale_entries are
  required; no_new_entries and no_raised_caps are advisory, so a deliberate
  exception is a reviewed baseline entry.
- Tests: Postgres-backed twins for each converted statement. New
  Kubernetes builder-lookup and Direct create-target tests, a unit table for
  the route-identity labels, and an otherwise identical 'space' row added to
  the sweeper's converge, internal-api's restored convergence and the
  wake-completion redrive tests. Test callers of the sweeper's shared
  builders bind the extra parameter.
- Docs: `docs/rust/space-conventions.md`, the substrate AGENTS.md and the
  space-rust-development skill say every crate's production SQL binds the
  value; in Rust a decision about a row's substrate calls a `SubstrateKind`
  method named for the question, a fact about the running substrate reads
  `capabilities()`, and code never compares the label string; and the
  archcheck rule catches the common literal forms. Plugin version
  0.2.33 -> 0.2.34.

Converted statements' text changes, so their sqlx statement-cache keys and
query signatures change; statements below an added `.bind` move to new
`sql!` locators.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants