[the-gigi-pplx]

Summary

Adds a nullable activating_client_id text column to sandboxes, recording which client created each sandbox's current runtime. Nothing reads or writes it yet. It's the record the per-client admission design in PR #1072 builds on, for both substrates.

Terms

• Client id: the caller's X-Client-ID when it's a valid, unreserved Kubernetes label value. Otherwise it's unknown, and it's native for native /v1 and admin calls. It's what the RFC in PR [docs] add RFCs for per-client admission independent of the substrate #1072 stores in records, pod labels and persisted operations ("Decision 2").
• Client key: what a client id resolves to against the operator's configured key set, at the moment a limit or metric uses it. It's native for native, the id when the key set names it, and unknown otherwise. Limits and metrics use keys, and records store ids.
• Runtime: the sandbox's running VM: on Kubernetes, the pod its active execution row tracks; on Direct, the VM a node started or restored.
• Activation: any action that gives a sandbox a new runtime: a create, a start of a stopped sandbox, a restore of a suspended one, or a rebuild of a runtime that was lost. A reconnect to a running sandbox isn't one.
• Inheriting activation: an activation that acts for the sandbox's recorded client instead of a client of its own. Examples are implicit access (an ingress request, opening a port, a command wake), a rebuild that's a side effect of stop or kill, and reconcilers (RFC "Decision 1").

The problem

The per-client limits in PR #1072 have to know which client a sandbox's runtime belongs to, after the request that created it is gone:

1. Not every activation comes with the client's X-Client-ID. These create a runtime with no request from the client that owns the sandbox:
   • Data-plane access. The internal API's session routes (data_plane_sessions/service_route.rs and session_route.rs) call PrepareIngress and QueueImplicitWake to wake a sandbox at rest when traffic or a reattaching session reaches it. These are internal RPCs, so they carry no client header. On Kubernetes they create the pod through ensure_execution_ready_for_access and drive_prepared_wake.
   • Drain evacuation. The cluster controller's node drain (node_drain, restore_evacuated_sandbox) restores each resident sandbox onto another node, with no API request at all.
     Each of these is an inheriting activation. It still has to charge a client and label the pod it creates. An explicit ECW call is different, even when it comes from another client than the one that created the sandbox: its own header names the client, and the new runtime is attributed to that caller (RFC "Decision 1").
2. The client id only exists on the request. Nothing in Postgres stores it. Copying the label from the previous pod doesn't work in general: a sandbox at rest (suspended or stopped) has no pod. Labeling these runtimes unknown would move the sandbox out of its client's cap until that client activates it again itself. Warm-pool pods are the exception: they're created before any client exists, and a create that claims one has the header in hand.
3. The record has to work on both substrates, as rate limiting already does. A Kubernetes-only table such as sandbox_k8s_executions would leave Direct with nowhere to record the client.

How this fixes it

• Store the client id on sandboxes. One column, activating_client_id. On Kubernetes, pod tracking sets it from the tracked pod's space.pplx.net/client-id label. It means "the client whose activation created the current runtime". Both substrates can write it, and an inheriting activation reads it from the row it already loads. The writers ship separately (RFC "Decision 3"): row inserts, Kubernetes pod tracking, and Direct receipts.
• Store the id, not the key. The key set changes over time. Storing the id and resolving it at each use applies a key-set change to existing sandboxes without a rewrite.
• Add a nullable column with no default, backfill or index. NULL means nothing is recorded, as for rows from before the column and sandboxes whose tracked pod was created without a label. Nothing filters on the column.
• Hold it to the label-value format with a CHECK. sandboxes_activating_client_id_check allows NULL or a non-empty Kubernetes label value: at most 63 characters, alphanumeric at both ends, with -, _ and . allowed inside. The reserved ids (unknown, native, overflow) satisfy it. The format is a fixed rule, so it belongs in the schema, and a bad value fails at the write instead of later, when an inheriting activation copies it onto a pod label.
• Create the CHECK NOT VALID. Adding it doesn't scan sandboxes. Every existing row is NULL, and Postgres checks new writes either way. A later migration can VALIDATE it, but nothing depends on that.

It ships on its own because a Postgres migration can't share a pull request with app code (scripts/check-postgres-migrations-isolated.sh). sandbox_k8s_executions is unchanged.

Validation

• check-migration-versions.sh, check-postgres-migrations-isolated.sh and check-postgres-migrations-squawk.sh pass. Squawk reports no issues.
• cargo nextest run -p pplx-space-db-migrate passes (54 tests, Postgres).
• On Postgres 17 (Linux, Docker), space-db-migrator control-plane up, down and up again from a fresh database. After each up, sandboxes.activating_client_id is nullable text with no default, sandboxes_activating_client_id_check exists with convalidated = false, and sandbox_k8s_executions has no new column. down removes the column and the constraint, and leaves the previous migration as the latest applied version.
• The constraint's definition, as Postgres stores it (pg_get_constraintdef), applied to a probe table:
  • accepts NULL, unknown, native, overflow, cleanup, batch-7, a, A.b_c-9, and a 63-character value;
  • rejects the empty string, a 64-character value, -lead, trail-, _x, x., has space, a/b, é and Not A Label!.

Not validated: a migration against a populated sandboxes table under concurrent load.

Dependencies

• The RFC in PR [docs] add RFCs for per-client admission independent of the substrate #1072 decides this design, so this pull request merges after the RFC is approved.
• PR [api] record the activating client and label sandbox pods with it #1068, the attribution and pod-label change, is stacked on this branch and merges after it.

Rollout

• Merge ahead of the attribution change that writes the column. Older images don't name the column, so they keep working against the migrated schema.
• Run the migration outside peak load (RFC "Rollout", step 1). The ALTER TABLE needs a brief ACCESS EXCLUSIVE lock on a busy table. While it waits behind a long transaction, other sandboxes queries queue behind it. The one-second lock_timeout bounds each wait. At the default Job deadline, the migrator retries up to five attempts, about 20 seconds in the worst case. If every attempt loses, the migration Job fails and the deploy waits on it until a rerun succeeds.

Severity and risk

Severity: does not apply. This is new schema with no readers.

Risk: low. Adding a nullable column with no default, and a NOT VALID CHECK, changes only the catalog, so it doesn't rewrite or scan sandboxes. Postgres evaluates the CHECK on every insert and update of a row. For a NULL value that's a NULL test, and otherwise a length check and a regular expression on at most 63 characters. Each lock wait is bounded by the one-second lock_timeout, so queries queued behind it wait at most a second per attempt, or fail with 55P03 where the app roles set a shorter lock_timeout. A run that loses every attempt fails the migration Job and holds the deploy. Roll back by rolling back any image that reads or writes the column, then running the down migration, which drops it. Down runs once, without the lock retry up has, so rerun it if it times out. If the migration is reapplied, every sandbox starts NULL. A running Kubernetes sandbox whose pod carries a label gets the value back when pod tracking next sees the pod. Any other sandbox stays NULL until a caller's own activation writes it, because an inheriting activation keeps a NULL record.

AI assistance: Claude Code drafted the migrations and this description.

[the-gigi-pplx]

No longer on hold: the RFC in PR #1072 settles the table, and this pull request now adds activating_client_id to sandboxes instead of a column on sandbox_k8s_executions.