Say which model each machine is actually serving - #124
Conversation
The fleet dashboard has rendered this field since 17 Sep and the API has always accepted it; the device publisher simply never put `model` in the payload. Finished code, unconnected wiring - so this is only the reporter. It DETECTS rather than reads a setting. `INTENT_DEVICE_MODEL` is the operator's statement frozen at daemon start: right until somebody swaps the weights, and then it is a name on a public dashboard that belongs to no running process, reported confidently forever because a setting cannot notice it went wrong. An OpenAI-compatible server answers about itself and stops answering the moment it stops serving. Nothing serving publishes NO `model` key - not null, not "unknown", not the last good answer. A 401 publishes nothing either: that box is serving something, but naming it is the whole job and we do not have the name. A reading that stops being refreshed expires rather than outliving its evidence. Every placeholder is a wrong model name waiting to be cropped into a screenshot; a blank field is not. The probe reuses `singleGet` via a new `probeServedModels()` export rather than adding a second prober - same auth header rules, same refusal to follow a redirect, same 401-splits-into-two verdict, same per-kind parsing. Credentials stay PATHS: the local entry is built through `loadRegistry`, so a token pasted into `INTENT_MODEL_KEY_FILE` is refused in the same words as one pasted into config/models.json. It runs on its own 5-minute timer, not the 30 s heartbeat, and the heartbeat reads a cached value and never awaits a socket. A dashboard that loses a whole machine because its LLM is down is worse than one missing a label. Default endpoint is 127.0.0.1:8080, deliberately not Ollama's 11434: measured 20 Sep 2026, Ollama's /v1/models named gpt-oss:20b while /api/ps was empty. A pulled model is not a served model. Verified against the live fleet through this code path: asus1:8888 + keyFile -> named "GLM-5.3-Flash-EXL3" asus1:8888 no key -> auth-blocked no model key m5:8080 no key -> auth-blocked no model key MacBook default -> no-server no model key 25 new tests, 559/559 in the full suite. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Codex Review SummaryThis comment shows the latest Codex review activity on this pull request.
ℹ️ About Codex in GitHubYour team has set up Codex to review pull requests in this repo. Reviews are triggered when you
Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings. |
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 05e62346e5
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| context: active ? 'active' : 'idle', | ||
| ...(idleSec === undefined ? {} : { idle_sec: idleSec }), | ||
| ...collectHostTelemetry({ machine: this.#machine, kind: this.#kind, model: this.#model }), | ||
| ...collectHostTelemetry({ machine: this.#machine, kind: this.#kind, model: this.#servedModel() }), |
There was a problem hiding this comment.
When a probe transitions from NAMED to idle, unauthorized, unreachable, or expired, this produces a state with no model property. publishState() sends that through patchDevice(), while the package README defines this PATCH as a partial update, so omitting the property leaves the previously published model in the server-side device slot. Since heartbeats keep that slot fresh, a dead server's stale label can remain indefinitely; use explicit field-removal or replacement semantics when the probe says no model.
Useful? React with 👍 / 👎.
| if (!this.#modelProbe) return this.#model; | ||
| const seen = this.#modelProbe.lastResult(); | ||
| if (seen?.model) return seen.model; | ||
| if (seen?.reachedServer) return undefined; |
There was a problem hiding this comment.
On Linux, when the endpoint responds with an empty list or 401/403, returning undefined here does not actually suppress the model: collectHostTelemetry() interprets an undefined explicit model as permission to scan /proc and can add the running llama-server model from its argv. Thus these environments still publish a model despite the documented rule that a reached-but-unnamed endpoint publishes none; propagate an explicit suppression state that is distinct from the no-server fallback.
Useful? React with 👍 / 👎.
| reason: `probe error: ${err?.message ?? err}`, | ||
| }; | ||
| } | ||
| this.#last = { reading, at: this.#now() }; |
There was a problem hiding this comment.
After a successful reading, one refused or timed-out request maps to NO_SERVER and is immediately stored over the named reading here. Consequently current() becomes undefined after the first transient failure, even though the configured staleAfterMs is three probe intervals and the constructor explicitly says a single timeout should not blank a correct label. Preserve the last named reading across unreachable results until its original timestamp exceeds the stale threshold, while still dropping it immediately for a conclusive idle response.
Useful? React with 👍 / 👎.
| export function parseEndpoint(raw) { | ||
| let text = String(raw ?? '').trim(); | ||
| if (!text) return null; | ||
| text = text.replace(/^https?:\/\//i, '').replace(/\/.*$/, ''); |
There was a problem hiding this comment.
When INTENT_MODEL_ENDPOINT is supplied as an https:// URL, this parser silently strips the scheme, and singleGet() later always constructs an http:// URL (also defaulting a scheme-only endpoint to port 80). A TLS-only endpoint therefore cannot be probed, and a bearer credential may be sent in cleartext if an HTTP service answers the selected port. Retain the scheme and use port 443 for HTTPS, or reject HTTPS input instead of silently downgrading it.
Useful? React with 👍 / 👎.
1 participant
Three of the four pieces already existed
IntentDashboard.tsx:555doesconst raw = slot['model'] ?? host['model']and line 726 draws the note. Asked for on 17 Sep: "what is missing is what model the machine is running (if any)".upsertIntentSlottakespayload: Record<string, unknown>, a free-form blob. No server change.model-capacity.js(Model capacity registry and probe: tailnet names, UP/BUSY/DOWN/UNREACHABLE, per-caller path #117) already asks an OpenAI-compatible endpoint what it serves and handles bearer auth viakeyFilePATHs.The missing piece was the reporter. The device publisher knew
INTENT_DEVICE_ID,INTENT_DEVICE_KINDandINTENT_DEVICE_PUBLISHand never putmodelin the payload. Classic finished-code-unfinished-wiring, so this PR is only the wire.Why detection beats configuration
INTENT_DEVICE_MODELalready existed and is honest about what it is: the operator's statement, frozen at daemon start. It is correct until somebody restarts a server with different weights, and then it is a name on a public dashboard belonging to no running process - reported confidently, forever, because a setting cannot notice it went wrong.An OpenAI-compatible server answers the question about itself and stops answering the moment it stops serving. So the ranking is now: a server that answered outranks a setting, and the setting survives only as a last resort for a box with no endpoint to ask (which is what its own docs already said it was for). On Linux the existing
llama-serverargv read still applies below both.INTENT_DEVICE_MODELis not removed andmodelProbe: nullreproduces the old behaviour exactly - there is a test for that.Nothing serving, and auth failing
Nothing serving publishes no
modelkey at all. Notnull, not"", not "unknown", not the last good answer. The dashboard already renders a card without a label and that blank is the honest picture. Every placeholder is a wrong model name waiting to be cropped into a screenshot; an empty field is not.A 401 also publishes nothing. That box is serving something - it bothered to reject us - but naming it is the entire job and we do not have the name.
m5:8080is the live case. The reachable-but-unnamed verdict is kept distinct from unreachable so an operator knows to configure a key rather than restart a server, and it suppresses the configured name too: we have live evidence about that port that we cannot verify a setting against.Point
INTENT_MODEL_KEY_FILEat a path to a file holding the token and the name appears. The local entry is built throughloadRegistry, so a token pasted where a path belongs is refused in the same words as one pasted intoconfig/models.json- a local entry is not an excuse for a weaker rule about secrets.A reading that stops being refreshed expires (3 missed probes) rather than outliving its evidence. A label that survives its own server is the exact defect being replaced.
Reuse, and cost
The probe is a new
probeServedModels()export inmodel-capacity.jsthat handssingleGetto a caller wanting one model-list GET instead of the switcher's p90 / free-memory / tailscale-path readings. Same auth header rules, same refusal to follow a redirect, same 401-splits-into-two verdict, same per-kind parsing. A second prober would drift, and the first thing it would drift on is which answers count as "it named a model".Cached, on its own timer.
INTENT_MODEL_PROBE_MSdefaults to 5 minutes, not the 30 s heartbeat. The heartbeat reads a cached value synchronously and never awaits a socket, and the probe starts after the first publish. A model swap is a human-scale event; a dashboard that loses a whole machine because its LLM is down is a worse bug than one missing a label.Default endpoint is
127.0.0.1:8080, deliberately not Ollama's 11434. Measured on the MacBook 20 Sep 2026: Ollama's/v1/modelsansweredgpt-oss:20bwhile/api/ps- what is actually resident - answered{"models":[]}. Its OpenAI-compatible list is the pulled catalogue, not the loaded set, so defaulting there would have an idle machine announce a model it is not running. A pulled model is not a served model.Verified against the live fleet
Through this code path, not curl:
asus1:8888+ keyFilenamed"GLM-5.3-Flash-EXL3"asus1:8888no keyauth-blockedmodelkeym5:8080no keyauth-blockedmodelkey127.0.0.1:8080no-servermodelkeyTests
25 new tests in
packages/user-intent-kit/test/served-model.test.js, node:test style, fetch always faked. Full suite 559/559, exit 0.Three proven fail-then-pass controls - each break applied to the real source, suite run, then reverted:
model: this.#model(the probe exists, nobody wires it)model: 'unknown'if (false)Restored: 25/25, exit 0.
There is also an in-suite control:
CONTROL: the same path that publishes nothing when idle publishes the id when servedruns both arms through one code path, because a "publishes nothing" assertion passes just as happily against a reporter wired to nothing at all.The heartbeat guarantee has its own tests:
refresh()never rejects, a throwing probe still publishesmachine,kind,load_pctandmem_total_gb, and a heartbeat goes out while a probe is still hanging.Deployment is not in this PR
Nothing was restarted;
asus1andm5were not touched. Deploying toasus1would need: pull the branch there, setINTENT_MODEL_ENDPOINT=127.0.0.1:8888andINTENT_MODEL_KEY_FILEto that box's own copy of the bearer token (a path, 0600), then restart itsuik-daemon. The label appears on the next heartbeat.🤖 Generated with Claude Code