Repository navigation
ui: manage tenant keys, terminals, snapshots and rollback across orgs, and show node and template state - #1387
Merged
ac-pplx merged 16 commits intoNov 3, 2026
Conversation
|
Thanks for opening this pull request! A SPACE maintainer will take a look. CONTRIBUTING.md explains how we review outside contributions. |
ac-pplx
marked this pull request as ready for review
November 3, 2026 09:24
ac-pplx
force-pushed
the
ac/admin-ui-operator-actions
branch
from
November 3, 2026 18:58
6ac1cff to
818f815
Compare
mabdelraouf-pplx
approved these changes
Nov 3, 2026
nikhilbirmiwal-pplx
approved these changes
Nov 3, 2026
…, and show node and template state Create API keys for any organization, choosing the user a key acts as, and show when keys were last used and which were revoked. Open terminals, list snapshots and roll back in any organization's sandbox through the admin routes, and show refusals as the API words them: branding, user deletes that still own sandboxes, the rollback gate, and template creates without a release. Show node cordon, group and runtimes, template builds, quarantine and snapshot storage. Send the system build Idempotency-Key and the tag precondition. Releases without these routes or fields get the previous behaviour: each detection rule is unit-tested. Antd deprecation warnings on the touched pages are gone.
…back to its end The sandbox page re-listed snapshots every 20 s through the admin snapshot list, own organization included, and each admin read writes an admin audit event, so each open tab kept adding audit rows. The operator's own organization now lists through the public routes again, as before the admin list, which also keeps the rollback gate answering as a hidden operation. Another organization's sandbox lists through the admin list when the card or row opens and on its Refresh button, never on a timer, and says when the page held more snapshots than it read. A rollback's 202 only says it was queued; during a rolling deploy an older API pod can accept `:force-rollback`, drop its body, and fail the operation. Rollback now follows the operation and reports that it finished, failed with its reason, or is still running. A gate refusal is remembered for the organization for the browser session: Rollback is disabled with the reason, with a way to ask again. `isMissingRoute` no longer matches the public API's "route not found" answer, which no admin route sends. `pageRows` keeps the cursor and refuses page shapes no route answers. The own-org check is one tested helper. The list row's snapshots panel ignores a stale answer, and the terminal card says when it opens in another organization.
…system build after an unknown outcome The retag dialog read the expected holder from the page's releases, which a background refresh replaces while the dialog is open, so a move could overwrite a concurrent one. The dialog now keeps the holders it opened with and sends those; if the typed tag moved by submit time, it shows from where to where and asks again. The tag move with its precondition and the 412 message are one helper, used by the dialog and the system-template rebuild. A system build's Idempotency-Key was fixed per run and template, so once the API recorded a build's outcome as unknown every "Build again" answered the same non-retryable 409, and the console's promise that a retry returns the first build was false on that path. That 409 now shows the job it names, if any, points to Template Releases and the template's builds, and offers "Start a new build", which sends a new key. The unknown-outcome text says what each release answers. Expanded build histories read again on each of the page's refreshes, keep their rows meanwhile, and decide afresh whether the route exists.
…when quarantine was not read A key whose user was deleted showed "Acts as: Unassigned", but it acts as the organization's oldest user; the column now says so. Create waits while the organization's users load, and after a failed read says who the key will act as. The created dialog names the organization and the key's user. An API release that refuses a key for another organization with a 403 is shown as creating keys only for your own organization, and Create stops offering it there. The last-used tooltip says that data-plane calls are not recorded and why a key used before the API recorded use shows "Never". A failed quarantine read showed "not quarantined"; it now shows unknown. A user delete that fails with a 500 shows the API's message, and the old-release hint only when that message is generic. Two node and one template-build type comments now match the server docs.
…ead builds only for expanded rows When a tag moved while the retag dialog was open, its warning replaced every tag's precondition with the holders read at submit time. A second tag that had also moved could then be moved over at once by switching the input to it, overwriting that concurrent move. The warning now advances only the warned tag's precondition, to the holder it showed, so OK again moves that tag only while it is still there, and another tag that moved warns first. The warning shows only while its tag is the one typed. A second press of OK while the move was in flight sent the same precondition again, which the first press had just made stale, so the dialog reported a 412 "tag moved" for a move that had succeeded. OK now sends one move at a time and shows it loading. The table keeps a collapsed row's expanded content mounted, so a template's build history kept reading its builds on every page refresh after the row was collapsed. It now reads when the row opens, again when it reopens, and on the page's refreshes only while the row is expanded.
… and keep the attempts a new build replaced A build request's answer was applied to whatever key generation the run had when it arrived. If another tab started a new build while an older request was in flight, the older request's closed-key 409 closed the new generation, and its success could be recorded as the run's build. Each request now keeps the generation it sent; only an answer of the current generation becomes the run's build or its blocking notice, and an older one is recorded on that older attempt. "Start a new build" dropped the earlier attempt, including the job the 409 named, although that job may still be running and publish a release. The run now keeps each replaced attempt with its generation, job and what is known of its outcome, and the panel lists them with where to look for their releases and builds. Only the notice that blocked building is cleared.
…ach rollback only while its view shows it, and let snapshot views read again A rollback gate refusal was remembered per organization, but each panel read it once, so a refusal or a "Try again" in one panel did not reach the others until they remounted. The refusals are now a small store per organization that every panel subscribes to, and a rollback checks it right before it is sent. Storage that refused the write left the refusal unreadable; the store now holds it in memory and copies it to storage. A rollback's notification was keyed by sandbox, and its follower outlived the view that sent it, so an older rollback could replace a newer one's notification and, when it settled, read the audited admin list for a view that was gone. Each notification is now keyed by its operation, and unmounting the view, or showing another sandbox, stops following: the notification says so and nothing is read again. Another organization's snapshot views that found no admin list stayed that way until they remounted, so a view opened during a rollout kept saying the release could not list them. The sandbox page's card and the list row's panel now have a Refresh that asks again; the row's panel has one in every state. A test drives the card's triggers and shows that the page's reloads, which include its timer, never read another organization's admin list.
…red for, and drop a quarantine state the card never shows A 403 to a key for another organization disabled Create for every other organization until reload. During a rollout one old API pod answers it while the others can create the key. The refusal now applies only to the organization it was answered for, and "Try again" or picking an organization clears it. The quarantine status had an "absent" state the card returns before reaching, and its test covered only that state's branch. The state is removed, and the test checks instead that a failed read is unknown also when the earlier read showed the sandbox quarantined.
…wers of its key, and take a replayed job as the build already followed A build attempt "Start a new build" replaced is retired only while its outcome is unknown. A later answer of that attempt's key changed it: a refusal marked it not started and hid where to look for its release, although the API checks authorization and the request before it replays the key, so a refusal says nothing about the first request. A retired attempt now keeps its job and unknown outcome, and each later answer is listed on it as a note. Two tabs that send the same key can both be answered with the same job. That job is the build the run already follows; the extra-build notice is kept for a different job.
The table keeps a collapsed row's expanded content mounted, so a collapsed snapshot panel went on following the rollbacks it started. The row now renders the panel only while expanded; unmounting it stops its reads and followers as closing the view already did.
…ox page, and list catalog snapshots for display only The console sent a catalog snapshot ID (snap_<uuid>) as the rollback's snapshot_name. The API passes the name to the sandbox's node unchanged, and the node rolls back only to a snapshot directory it keeps, so every rollback failed, on the public route and on :force-rollback alike. The sandbox page's Snapshots card now lists the node's snapshots from the debug view's snapshots.inventory, which the page already reads on its cadence (no new request and no audit event), and offers Rollback there, sending the row's snapshot_id. It leaves out the names sbx-mgr keeps for itself (d-stop-, d-operation-, GC staging), disables Rollback on rows the node does not report ready, and on every row while the sandbox is suspended, saying why. A full snapshot's confirmation says it needs restore support on the node. The own organization rolls back through :rollback; another through :force-rollback once the admin snapshot list has answered, as before. The gate refusal is remembered and shared as before. A settled rollback reloads the page as any action does. The catalog lists (public for the own organization, admin for another) are display-only on the sandbox page and in the list row's panel, which points to the sandbox page for rollback.
OK in the retag dialog compared the typed tag against the page's release list, which can be up to one refresh old, so a tag moved in that window was sent with the precondition the dialog opened with and refused with a 412. OK now reads the releases first: if the tag moved since the dialog showed it, the dialog shows from where to where and asks again; otherwise the move is sent. A failed read sends nothing, and a dialog closed during the read sends nothing.
…ts view stops following it pollOperation passed no signal to its reads and checked the deadline and the caller's abort only between them, so a status request that never answered kept a rollback follower pending past unmount, abort and its two-minute limit. Each read now gets an AbortSignal, which the public and admin operation reads pass to fetch, and is raced against the remaining deadline and the caller's signal; the request is cancelled once the read is given up.
…ps its holder Any string is a valid tag, but tagHolders built a plain object, where holders['__proto__'] = id invokes the inherited setter and records nothing; the retag dialog then took an existing __proto__ tag as absent and sent "does not exist" as its precondition. Holders, the retag baseline and its updates are now a Map.
…ag only when its warning shows Changing the tag input while OK's re-read of the releases was pending let that answer advance the precondition of the tag OK was pressed for, behind a warning the dialog did not show: OK on that tag later moved it from where nobody had seen it. The tag input and OK are now disabled while a submit is in flight, and retagStepNow answers `stale`, sending nothing and advancing nothing, unless the dialog still shows that tag when the read answers.
The admin API now answers a key for an organization other than the caller's, sent without user_id, with 400 "user_id is required for another organization's key"; it no longer picks the organization's oldest user. The create dialog follows: for another organization the Acts as picker is required, with no default and no oldest-user text, and Create stays disabled until a user is chosen. When the organization's users could not be read, the dialog asks for a retry instead of naming a default. A key for the caller's own organization still sends no user_id and acts as the caller. Releases before cross-organization keys refuse another organization's key with a 403, or first with a 422 for the user_id it now always carries; both read as the existing "creates keys only for your own organization" notice, for that organization only. An own-organization 422 keeps the user-choice notice, and the new release's 400 shows its message.
ac-pplx
force-pushed
the
ac/admin-ui-operator-actions
branch
from
November 3, 2026 22:23
818f815 to
d2b0edf
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Operators can now act on any organization from the console, through the admin
routes that #1302 and #1303 add and the
actor_emailfield #1384 adds, and seenode and template state that #1307 already exposes. The console still works against an API without those PRs:
each new call detects the older answer and falls back to today's behaviour.
the "Acts as" user from that organization, with a preview of who the key
will act as. For your own organization the choice is optional and the key
acts as you, sending no
user_id. For another organization it isrequired: api: let operators manage keys, terminals and snapshots across orgs #1302 now answers a key without
user_idthere with 400("user_id is required for another organization's key") instead of picking
the organization's oldest user, so the picker has no default and Create
stays disabled until a user is chosen. Create waits while the
organization's users load; after a failed read it says the key acts as
you in your own organization, and asks for a retry in another. The
created dialog names the organization and the key's user. A key whose user was deleted
reads "No recorded user; acts as the organization's oldest user", which is
what every release resolves it to. The page shows when each key was last
used (SPACE and E2B-compatible API authentications, at most every five
minutes; data-plane ECW envd calls are not recorded, and a key not used
since the API began recording shows "Never") and, behind "Show revoked
keys", revoked keys marked with when they were revoked. The organization
page's key card shows the same columns.
its terminal through the admin terminal route, and the card says so; your
own organization's sandboxes keep the public route.
sandbox's node keeps, the debug view's
snapshots.inventory, and sends therow's
snapshot_idassnapshot_name. The API passes that name to thenode unchanged, and the node rolls back only to a snapshot directory it
keeps, so the catalog's
snap_<uuid>IDs, whichmainand earlier roundsof this PR sent, never rolled back. The sandbox page already reads the
debug view every 20 s, so this list adds no request and no audit event. It
leaves out the names sbx-mgr keeps for itself (
d-stop-*,d-operation-*, GC staging), disables Rollback on rows the node does notreport ready, and on every row while the sandbox is
suspended("Restoreit first"); a full (
f-*) snapshot's confirmation says the node mustsupport snapshot restore. Your own organization's sandboxes roll back
through the public
:rollback, as onmain; another organization'sthrough
:force-rollbackwith the public rollback body, once the adminsnapshot list has answered for the sandbox. The catalog snapshot lists stay,
for display only: your own organization's through the public routes,
re-listed every 20 s on the sandbox page; another organization's through
the admin list. api: let operators manage keys, terminals and snapshots across orgs #1302 writes an admin audit event for every admin list
read, so that list is read when the panel or card opens and on its Refresh
button, never on a timer; the card says when it was listed, and when more
than the 1000 newest snapshots exist. A card or panel that found no admin
list (an older release, or an old pod during a rollout) has a Refresh that
asks again. The sandbox list's Snapshots panel has no Rollback and points to
the sandbox page. After the 202, the console follows the rollback's
operation and reports that it finished, failed with its reason, or is still
running after two minutes, then reloads the page as after any action. Each
rollback has its own notification; closing the view or showing another
sandbox stops following it, and the notification says so. A refusal by the
rollback gate (off by default) is remembered for that organization for the
browser session, in every open view of that organization: Rollback is
disabled with the API's reason, with a "Try again" link that applies to all
of them. The sandbox page adds each catalog snapshot's storage (node-local,
object store, both, none, with the node).
the organization page and mark the field. A user delete refused with a 409
names the reason (owns sandboxes; deleted sandboxes kept until a time; owns
archived sandboxes); the Users table no longer reads the organization's
sandboxes to guess. A failed create names
template_has_no_releaseortemplate_release_not_foundwith the API's message, also when snapshotswere not requested. A user-delete 500 shows the API's message; the
old-release hint replaces it only when the message is the generic
"Internal server error".
installations on the list and the node page; absent fields show nothing.
Each template row expands to its recent builds (state, content hash, error,
status error, created time), read again on each of the page's refreshes
while the row is expanded, and not while it is collapsed.
Idempotency-Keyperrebuild run and template. Building again resends it: api: close six admin API gaps the admin console works around #1303 answers with the
first build when it recorded one (two tabs answered with the same job
follow that one build, with no extra-build notice), and with a
non-retryable 409 once the first request's outcome is unknown. On that 409 the rebuild panel shows the
job it names, points to Template Releases and the template's builds, and
offers "Start a new build", which sends a new key. The panel keeps listing
each attempt a new build replaced, with the job the 409 named, and keeps its
outcome unknown, since it may still publish a release. An answer that
arrives late for a replaced attempt is listed on it as a note and never
changes it, nor becomes the current build: a later refusal (a 403 after an
authorization change, say) says nothing about the first request, since the
API checks the request before it replays the key. Tag moves send
expected_current_release_idfrom the tagholders read when the dialog opened, not from a later background refresh.
OK first reads the template's releases again: if the typed tag moved since
the dialog showed it, the dialog shows from where to where and asks again,
advancing only that tag's precondition; otherwise it sends the move. A
failed read sends nothing. OK sends one move
at a time. A release that refuses the field gets one retry without it, and
the operator is told the precondition was not checked.
events, and "unknown" when the quarantine read failed; the environment
shows names only, whichever shape the API sends.
actor_email(api: answer 409 for org deletes with users and duplicate emails, and name the actor on admin operations #1384), the console usesit and skips that row's user lookup.
Space direction,Alert message,Input addonBefore,Statistic valueStyle, ModalmaskClosableanddestroyOnClose,modal.confirmautoFocusButton) and a TablerowKeyindex are replaced; behaviour is unchanged.
Also:




Against an API without #1302 and #1303:
Validation
At this head, in
ui/:npm ci && npm test && npx tsc --noEmit -p . && npm run build(278 tests pass, type check clean, build succeeds). The branch is rebased
onto
mainwith #1388's server-paged sandbox list;npx tsc --noEmit -p .and
npm testpass at every commit. Unit tests inui/src/libcover each detection rule (missing route, unknown-field 422, page object or
array, absent fields), the snapshot route choice (own organization never
reads the audited admin list, and another organization's card does not list
on the page's reloads, which include its timer), the rollback targets (only
the node's inventory names, never a catalog
snap_ID; sbx-mgr's internalnames left out; ready and unpublished full snapshots offered, others
disabled; every row disabled while
suspended; the body sent; the route foreach organization), following an operation to its end, the required key user for another
organization (Create disabled until one is chosen, no oldest-user text; an
own-organization key sends no
user_id), an older release's 422 or 403 toanother organization's key read as the "only your own organization" notice, stopping when its
view closes, one notification per operation, and the rollback outcome text,
the rollback gate refusal and its per-organization store shared by every
subscribed view, the retag dialog's captured tag holders, its re-read on OK
(a tag moved since the page's last refresh warns and sends nothing; a failed
read sends nothing), advancing only the warned tag (two tags that both
moved) and one move per OK, the tag-precondition retry and 412 message, the
system build's closed key, "Start a new build" keeping the attempt it
replaced, a replaced attempt that a late refusal, failure or job leaves
unchanged and only noted, a replayed job taken as the build already
followed, two interleaved key generations, the builds panel reading only
while expanded, the key "acts as" choice and its texts, a cross-organization
key refusal limited to its organization, the quarantine status after a
failed read, the own-org check, and the user-delete messages.
Key user and the sandbox list's row actions, at this head, on a local Kind
stack (Direct substrate) running
mainwith #1303, #1384 and #1302cherry-picked and this branch's
ui/served bynext dev, with onesynthetic tenant organization and the rollback gate on:
disabled; with a user chosen it sent
user_idand the created dialog namedthat user. The key created a sandbox through the public API, and the admin
API reported that sandbox's owner and the key's
created_byas the chosenuser. Before this change the console offered "The organization's oldest
user (default)", sent no
user_id, and the API answered 400 "user_id isrequired for another organization's key", shown as the error.
user_idsent;the key acts as the signed-in operator (owner and
created_bychecked thesame way).
?limit=50&cursor=): each row's Snaps switch queued:force-enable_snapshots, each row's panel listed the catalog (admin routefor the tenant, public for the own organization, display only) and links to
the sandbox page. There, Rollback on the node's
d-1m-*snapshot sent:force-rollback(tenant) and:rollback(own organization), and bothended "Rolled back to d-1m-…". The terminal opened through the admin
terminal route for the tenant and the public route for the own
organization, and ran a command in each.
Rollback and the retag re-read, at an earlier head, on a local Kind stack running
mainwith #1321, #1303, #1302 and #1384 cherry-picked and this branch'sui/asspace-ui, with the rollback gate on for the managing organizationand one synthetic tenant organization:
organization, sbx-mgr's scheduler kept a
d-1m-*snapshot for each. Fromthe sandbox page, Rollback on that row sent
{"snapshot_name":"d-1m-…","auto_restart":true}to the public:rollback(own organization) and to
:force-rollback(the tenant), and bothoperations ended
succeeded("Rolled back to d-1m-…"). Listing the targetsadded no request: the page read the debug view on its own cadence, and the
tenant's admin snapshot list once, when the card opened.
the reason.
the API and OK pressed at once, before the page's list refreshed: OK sent
only a releases read and showed the move; OK again read again and sent the
move with the new holder as the precondition.
the console: in both organizations, the first rollback attempt failed on
the node ("sandbox host mutation or reset is in progress; retry") after
stopping the VM. The operation's retry found the sandbox pod exited,
recreated it on another node, and that node had no such snapshot. On Direct
the same first-attempt failure occurred and the retry succeeded, since the
sandbox stays on its node. This is a node or API defect for a separate
issue.
The other screenshots above were taken before the review fixes of earlier
rounds. Those fixes were checked by the unit tests and the build only, not in a
browser or on Kind; that collapsing a sandbox row unmounts its snapshot
panel was checked once against the installed antd table in a throwaway
jsdom script, which is not committed; the rollback-refusal, rollback-queued
and rebuild-replayed screenshots no longer match the console and were
removed. The card's new Refresh, the replaced-attempt rows of the rebuild
panel and the API key "Try again" have no screenshot.
Before those review fixes, on a local Kind stack running
mainwith #1321,#1303 and #1302 cherry-picked (#1321 was on that stack; nothing here uses
it), the branch console was checked in a browser across two
synthetic tenant organizations: each action above, the rollback gate and
user-delete refusals, a real seed build listed and replayed by its key, and a
create against a digest with no release. The same stack was then redeployed
with
main's API alone and each fallback checked.npm run devon everyconsole route logged none of the listed warnings (one
rowKeywarning seenin that sweep was removed afterwards; that last change was checked by tests
and the build, not in the browser again).
Not validated live: the quarantine card (rendered from a synthetic debug
section; no sandbox was quarantined),
template_has_no_releaseand a 412 tagprecondition (unit tests only), rollback to a full (
f-*) snapshot and to arow the node does not report ready, the rollback gate refusal and the
older-release fallbacks at this head, the 405 fallback for template builds
on releases before #1307, and the admin audit rate of a sandbox page left
open (unit tests show the own organization's listing never reads the admin
list, and another organization's card does not list on the page's reloads).
Severity and risk
Severity: does not apply (operator console features and fallbacks).
Risk: for another organization's sandbox, the snapshot panels and terminal
call admin routes first. On an API without them, the console falls back on
the 404 the admin API's fallback answers (
{"code": 404, ...}); a proxy thatrewrote that answer would show an error instead of falling back. Rollback is
sent to
:force-rollbackonly when the admin snapshot list answered, becauseolder
:force-rollbackaccepts the request and then fails the operation.Rollback targets come from the node's last published inventory; a sandbox
that moved to another node since can list a name its new node lacks, which
the node refuses and the console reports as a failed rollback. The
gate refusal is recognized by its message ("rollback is not enabled"), since
it carries no reason code; a reworded refusal would show as an error, not be
remembered. Expanded template build histories re-read on each page refresh
(every 10 s while visible), which costs one builder status call per listed
build. Roll back by reverting the commits.
Rollout
The console can deploy before or after the API PRs. Older API releases get:
no "acts as" choice (422 named in the message); for another organization,
"this API version creates keys only for your own organization" after its
403, or after the 422 such a release answers first to the
user_idanotherorganization's key always carries,
instead of the acts-as promise, for that organization only and until "Try
again" or another organization is picked; live keys only; own-org snapshots and
terminals only; a generic message for a refused user delete; and tag moves
without the precondition (with a warning).
While the API rolls out, pods of both versions answer the console. A
snapshot listing answered by a new pod can send a rollback to an old one,
which accepts
:force-rollbackand drops its body, so the operation fails:the console shows that failure and its reason. During a mixed rollout the
first rollback attempt can fail this way; retry it once the rollout
finishes. A snapshot view answered by an old pod says the release cannot
list them; its Refresh asks again.
Stack
Merge after #1302, #1303 and #1384 (#1384 is stacked on #1303); the console
falls back cleanly on servers without them. #1302 and #1303/#1384 conflict in
IdentityAdminError::into_response, where #1302 builds the branding refusalbody (
{error, field}) and #1303/#1384 build the user-delete refusal body(
{error, reason, retained_until}); whichever merges second must keep both.The console reads both shapes. #1302 and #1303 also conflict in the admin
sandbox create, where #1302 splits the organization check out of
create_sandbox_for_organd #1303 adds itsrequested_byargument; theKind stack above kept both. This branch needs #1302's current contract
(another organization's key requires
user_id); against a #1302 that stilldefaults to the oldest user, it only stops offering that default.
Not covered
main; unchanged here.org_has_users,email_in_use) come from another PR; those flows already show any error'sAPI message.
AI assistance: an AI agent (Claude Code) wrote the code, tests, validation
and this description.