The warm pool in status, doctor, one event, and the console #371

Part of #359.

Scope

After this PR an operator can see the warm pool: simlock status shows whether it is on, the reserve, and each target with how many are ready, booting, and why it is short. simlock doctor reports a target that can never be met as configured. One event marks a target going short. The console shows the block on a worker's page.

In short

$ simlock status
warm pool: enabled, reserve ios 1 android 0
  iPhone 17 / 26.0 / full   wanted 2  ready 1  booting 1
  iPhone 17 / 27.0 / full   wanted 1  ready 0  booting 0  short: runtime-missing
$ simlock doctor
warm-pool-target-unreachable  iPhone 17 / 27.0 / full: iOS 27.0 is not installed; run simlock component install ios 27.0

"warmPool": {
  "enabled": true,
  "reserveRunning": { "ios": 1, "android": 0 },
  "targets": [
    { "platform": "ios", "model": "iPhone 17", "osVersion": "26.0", "mode": "full", "count": 2, "ready": 1, "booting": 1 },
    { "platform": "ios", "model": "iPhone 17", "osVersion": "27.0", "mode": "full", "count": 1, "ready": 0, "booting": 0, "short": "runtime-missing" }
  ]
}

short: a target for which the pass could do nothing, with one reason. A target that is filling (a boot or creation in flight), held by the boot cap, or waiting behind a queued request is not short. When several reasons apply the first of this list is reported: disabled, no-driver, runtime-missing, unknown-model, unresolvable, boot-failed, device-limit, running-limit, reserve, ram-budget. With the pool off every configured target is short: disabled.

flowchart LR
  P[pass ends] --> Q{target below count and the pass<br/>recorded a refusal for it?}
  Q -->|no| N[not short]
  Q -->|yes| S[short with reason]
  S --> E{was it short<br/>after the last pass?}
  E -->|no| EV[emit warm-pool.target-missed]
  E -->|yes| Z[no event]

If this goes wrong: status says a target is short when it is filling, or the event fires every pass on a machine that is always full.

Technical spec

File and line references are to main at 4a52f6e, after #369 and #370.

Modules touched

flowchart LR
  classDef changed fill:#f3d9a4,stroke:#8a6a2a
  CONV[warm-pool/converger.ts<br/>figures, event]:::changed --> BUS((event bus)):::changed
  DISP[daemon/dispatcher.ts status.get]:::changed --> CONV
  DOC[core/doctor.ts]:::changed --> CONV
  SCHEMA[contract/schemas.ts, protocol.ts]:::changed
  CLI[cli/index.ts formatStatus, doctor]:::changed --> SCHEMA
  AGG[gateway/aggregate.ts]:::changed --> SCHEMA
  UI[ui worker-detail]:::changed --> SCHEMA

• src/core/warm-pool/converger.ts: figures(): WarmPoolFigures built after each pass: enabled, reserveRunning, and per resolved target the kind as resolved (platform, model, osVersion, mode), count, ready, booting (boots and provisions in flight for it), short?. policy.ts records, per target it could not fill, the first refusal reason in the order above, and records nothing for a target held by the boot cap or by a queued request. A target is short exactly when the policy recorded a reason for it. The one definition lives in policy.ts; status, doctor and the event read it. The converger keeps the last pass's short set and emits warm-pool.target-missed for each target that is short now and was not after the previous pass.
• src/contract/schemas.ts (statusGetOutput, beside capacity at :707): required warmPool block as in the JSON above; short is an enum of the ten reasons. src/contract/protocol.ts:47: range to {min: 17, max: 17}.
• src/daemon/dispatcher.ts:333 (#statusGet): reads the figures through a WarmPoolReader port on LeaseEngine (figures()), beside the capacity reader in src/core/lease-ports.ts.
• src/core/doctor.ts: finding kind warm-pool-target-unreachable with the target's kind and the reason, for a target whose last figures are short with runtime-missing or unknown-model, and one finding per platform whose targets' counts add up to more than that platform's maxRunning minus its reserve, plus one when all targets' counts add up to more than the global maxRunning minus the sum of the reserves. Remediation names the install command or the limit and the sum. src/contract/schemas.ts:595 (doctorFindingSchema) gains the kind.
• src/cli/index.ts:2307 (formatStatus): the block as printed above; simlock doctor prints the new finding like the others.
• src/contract/schemas.ts (workerViewSchema, :946-966): the view gains optional warmPool, the same block. src/gateway/worker-link.ts: the status.get refresh fills it. src/gateway/aggregate.ts: status.get on a gateway carries each worker's block under its worker entry as the view holds it: absent for a worker whose status was never read or that is incompatible, the last read block for a disconnected one. The gateway's own top-level warmPool is { enabled: false, reserveRunning: { ios: 0, android: 0 }, targets: [] }.
• src/http needs no route change: status is served as it is. ui/src/views/worker-detail.tsx: a "Warm pool" section listing the targets with the four numbers and the short reason, reading the existing status route.
• Docs: docs/CLI.md (status, doctor), docs/HTTP-API.md and docs/CLIENT.md (the status block), docs/CONSOLE.md (the section), docs/CONFIGURATION.md (how short is reported), both EVENTS.md files, docs/internal/ARCHITECTURE.md protocol paragraph.

Contract and event changes

• status.get output gains required warmPool; protocol 17.
• doctor findings gain kind warm-pool-target-unreachable.
• New event warm-pool.target-missed: payload { platform, model, osVersion, mode, count, ready, reason }, emitted by warm-pool after a pass commits, on the edge only. Row in both EVENTS.md files.

Rules in play

• events.md rules 1 (name: subject warm-pool, past-tense fact), 3 (post-commit), 8 (both catalogs).
• architecture.md rule 10: short is computed once, in the converger; status, doctor and the event read it.
• documentation.md rule 2: no ADR citation in the end-user docs.
• safety.md rule 10 is untouched: nothing new arrives over the wire.

Tests

• policy: a target with ready 1 booting 1 count 2 records no reason; one held by the boot cap or a queued request records none; one refused for the reserve and the running limit records running-limit.
• converger: warm-pool.target-missed fires once when a target first ends short, not on the next short pass, and again after it was met and missed again.
• dispatcher: status.get carries warmPool with the converger's figures; on a daemon whose pool is off it carries enabled: false, reserveRunning, and every configured target with ready 0, booting 0 and short: disabled.
• doctor: a target with runtime-missing yields warm-pool-target-unreachable naming the install command; two iOS targets of 2 on a limit of 3 yield one finding naming the sum and the limit; a target short for ram-budget does not.
• contract: a status.get output without warmPool fails to parse; protocol range is 17.
• CLI: simlock status prints the block and --json carries it; simlock doctor prints the finding.
• gateway: status.get on a gateway carries a connected worker's block from its last status refresh, none for an incompatible worker, and an empty one of its own.
• console: the worker page renders the targets table from the status route.

Done when

• simlock status --json shows warmPool with enabled, reserveRunning and every configured target with count, ready, booting, and short when short; with warmPool.enabled: false every target shows short: disabled.
• With a target naming a runtime that is not installed, simlock doctor prints warm-pool-target-unreachable with the install command.
• simlock events --follow shows one warm-pool.target-missed when a target first goes short and none on the following passes.
• Through a gateway, simlock status --json shows the warmPool block of each connected worker.
• The console's worker page shows the warm pool section.

Out of scope

• Fleet-level targets or a gateway pool.
• Changing targets through the console or an admin command.

Depends on

• A reserve of running slots, and a request waits for a device on its way #369
• Warm targets: devices booted ahead of demand, exempt from idle shutdown, pre-created under fresh identity #370

Approval

• Approved for delivery

Written by an agent.