Skip to content

ui: show each node's sandbox and slot occupancy - #1252

Merged
ac-pplx merged 3 commits into
mainfrom
ac/admin-ui-node-occupancy
Nov 3, 2026
Merged

ac-pplx merged 3 commits into
mainfrom
ac/admin-ui-node-occupancy

Conversation

@ac-pplx

@ac-pplx ac-pplx commented Oct 31, 2026 •

Copy link
Copy Markdown
Contributor

Summary

The nodes page now shows how full each node is.

It listed nodes with include_occupancy=false, so it never showed:

  • how many sandboxes a node holds;
  • how many snapshots exist only on its disk;
  • whether its network slots are degraded or quarantined.

The admin API returns all of these on request. An operator choosing a node to drain, or looking for the full one, had to open each node's sandbox drawer.

Node reads stay metadata-only (include_occupancy=false), as on main: the one-second refresh, the initial load, Refresh and the reload after a node action. The per-node sandbox and snapshot counts come from a separate best-effort include_occupancy=true read, made after the page loads, on Refresh, after a node action and every 30 seconds while the page is visible. The page merges them into the rows by node ID and drops any response for an older request or another role filter. A new Sandboxes column shows:

  • the active sandbox count;
  • a "local snaps" tag when the node holds node-local snapshots;
  • an "unhealthy slots" tag when slots are degraded or quarantined.

A tooltip gives the full slot counts, from each row's own node response, and when the sandbox counts were taken. The counts are marked "stale" when the last occupancy read failed or is older than 60 seconds, and a failed read never fails the page or an action.

Stack

#1256 (the node page) is stacked on this PR. It edits ui/src/app/dashboard/nodes/page.tsx, which #1242 and #1250 also touch; whichever merges later may need a small rebase.

Validation

  • npm ci, npm test (50 pass) and npm run build in ui/.
  • A Playwright loop against a local Kind stack with synthetic data (two workers with 2 and 4 sandboxes, two idle builders), on main (production build) and on this branch (the Kind space-ui image). The nodes step checks that each node shows its active count.
  • No new console errors, page errors or failed requests compared with main.

Visual confirmation

Before After
before after

Severity and risk

Severity: medium. Capacity is the first thing to check before draining or placing work.

Risk: low. The page adds one occupancy query (per-node sandbox and snapshot counts) every 30 seconds per open nodes page, plus one on load, on Refresh and after each node action. Plain space-admin-cli list-nodes reads metadata only (include_occupancy=false). Drain ranking (list-nodes --sort drainability) and get-node --diagnostics are the other consumers of this counting query. Roll back by reverting the commit.

AI assistance: Claude Code wrote the code and this description, and ran the validation; a maintainer reviews it.

The nodes page listed nodes with include_occupancy=false, so it never
showed how many sandboxes a node holds, how many snapshots exist only on
its disk, or whether its network slots are degraded or quarantined, all
of which the admin API returns on request. An operator choosing a node
to drain, or looking for the full one, had to open each node's sandbox
drawer.

The list now asks for occupancy and shows a Sandboxes column: active
sandboxes, node-local snapshots when there are any, and unhealthy
network slots when there are any, with the full slot counts in a
tooltip.
@github-actions github-actions Bot added the triage:pending Waiting for a maintainer to triage label Oct 31, 2026
@github-actions

Copy link
Copy Markdown

Thanks for opening this pull request! A SPACE maintainer will take a look.

CONTRIBUTING.md explains how we review outside contributions.

The nodes page refreshes its node list every second while visible. With
occupancy enabled, every one of those refreshes asked the admin API for
include_occupancy=true, which counts sandbox and snapshot rows per node:
once a second for every open nodes page.

The one-second refresh is metadata-only again. Occupancy is read when
the page loads, on Refresh, after a node action and every 30 seconds
while the page is visible, and merged into the rows by node ID; the
Sandboxes column's tooltip says how long ago the counts were taken.
Review follow-ups for the nodes page's occupancy column:

- Every node read (the one-second refresh, the initial load, Refresh
  and the reload after a node action) is metadata-only. Occupancy, the
  per-node sandbox and snapshot counts, is a separate best-effort read
  afterwards and every 30 s; a failure marks the counts stale instead of
  failing the load or the action.
- Network slots come from each row's own node response, which carries
  them every second, rather than from the occupancy read.
- Occupancy responses for an older request or another role filter are
  dropped.
- The counts are marked stale when the last occupancy read failed or is
  older than two intervals; the tooltip gives their age ("counted 12s
  ago", not "ago ago") and prints missing slot fields as "—".
zibi-pplx
zibi-pplx previously approved these changes Nov 1, 2026
@zibi-pplx
zibi-pplx dismissed their stale review November 1, 2026 22:30

Withdrawing this approval for now. Any feedback above still stands.

@ac-pplx
ac-pplx added this pull request to stack #1281 November 2, 2026 04:29

@johnlouie-pplx johnlouie-pplx left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Approving with one nonblocking follow-up. At 27077d3, I checked the counts against the API contracts; the UI build passed, and mocked-API checks distinguished zero, missing and failed counts.

In ui/src/app/dashboard/nodes/page.tsx, the Sandboxes cell returns early when the sandbox count is unavailable, hiding the unhealthy-slot warning even though slot data comes from a separate read. Could we keep that warning beside "could not count" or "not computed", and say "sandbox counts are stale" so the tooltip doesn't imply that the slot sample has the same age?

@ac-pplx
ac-pplx added this pull request to the merge queue Nov 3, 2026
Merged via the queue into main with commit 2c804a5 Nov 3, 2026
28 checks passed
@ac-pplx
ac-pplx deleted the ac/admin-ui-node-occupancy branch November 3, 2026 04:54
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

triage:pending Waiting for a maintainer to triage

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants