Skip to content

Let a model investigate an alert through the operations catalog - #28

Merged
shipmill-romamo[bot] merged 6 commits into
mainfrom
feat/agent-investigate
Oct 8, 2026
Merged

shipmill-romamo[bot] merged 6 commits into
mainfrom
feat/agent-investigate

Conversation

@romamo

@romamo romamo commented Oct 8, 2026 •

Copy link
Copy Markdown
Collaborator

What

cloudfall agent investigate --alert TEXT --base-url URL --model NAME lets any OpenAI-compatible model work an alert through the team's operations catalog.

  • The model gets the same operation tools mcp serve --repository gives any client: a read operation runs and returns its output (operations propose: a read operation's output is not returned, only the path to its .diff #24), any other operation is proposed in check mode and recorded as a decision
  • No tool approves anything; approving stays decisions approve, a person's step (see decisions approve --yes runs headless, so an agent with a shell can approve its own proposal #25)
  • A call repeated with the same arguments is answered from the first result instead of running again
  • The run is recorded in investigations/ as an AgentInvestigation (new schema): alert, model, tokens, every call and its outcome, the finding; decision ids the model named but never recorded go under unrecorded
  • API key through treaty's secret flag (--api-key-from-env, --api-key-from-file, CLOUDFALL_API_KEY); it never reaches the record
  • New exit codes: 92 MODEL_UNAVAILABLE, 93 INVESTIGATION_INCOMPLETE (record in data)
  • Read operation tools now describe themselves as running, not as check mode only
  • The operation tool schema is shared between mcp serve and the agent

Proven live

Disposable Hetzner cx23, PostgreSQL down because 36G of unrotated shop worker logs filled the disk. With Nemotron 3 Super on Nebius Token Factory:

  • 5 turns, about 15k tokens: read the PostgreSQL journal, ran the disk report, proposed shop-logrotate then postgresql-restart, no unrecorded ids
  • Both approved by a person: verified, disk at 5%, PostgreSQL answering; host deleted

Review fixes

Each with a regression test that fails on the code before it:

  • 0b18ed5: an unreachable endpoint now exits 92 instead of 12 "retry, it had no side effects" (decisions may already be recorded); a 200 that isn't JSON exits 92 instead of crashing
  • 1701380: a read called twice no longer puts its decision id into finding.proposed
  • 034f195: a made-up tool name over 128 characters is cut to 128 in the record instead of failing its schema after the run and losing it; a blank --alert, or a model name or endpoint longer than the schema allows, is refused with exit 2 before the run
  • 16739db (security): a --base-url with a query or fragment is refused with exit 2, without echoing it; before, ?api-key=... was written to the record, and the scheme error echoed the whole URL
  • d4ca6a6: a second run in the same second is saved as <id>-2 instead of failing; a record that can't be written exits PRECONDITION, not 92

Follow-up: #29 (a 92 partway through a run writes no investigation record).

Checks

  • uv run pytest: 384 passed after the review fixes (374 before, 9 of them new: scripted model for the loop, plus an end-to-end CLI test against a local OpenAI-compatible server)
  • uv run ruff check ., uv run mypy, cloudfall config validate --project config/examples: clean

🤖 Generated with Claude Code

https://claude.ai/code/session_01APTdZZw8Jw9PQF4TD55wMv

romamo and others added 4 commits October 8, 2026 19:22
cloudfall agent investigate sends an alert to any OpenAI-compatible
endpoint and gives the model the catalog's operation tools, the same ones
mcp serve offers: reads run, changes are proposed in check mode, nothing
is approved. Repeated calls are answered from the first result. The run
is kept as an AgentInvestigation record with the model, tokens, steps and
finding, and decision ids the model named but never recorded are set apart.

Read operation tools now say they run, not that they run check mode only.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01APTdZZw8Jw9PQF4TD55wMv
ctx.http's own exit 12 told the caller the command had no side effects,
though earlier turns may have recorded decisions, and a 200 answer that
was not JSON crashed the handler.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A repeated step carries the decision id of the call it repeats, so a read
called twice and named under PROPOSED was listed as a proposal.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A tool name the model made up longer than 128 characters failed the
record's schema after the run and crashed the handler; the record now
keeps the first 128. A blank alert, or a model name or endpoint longer
than the schema takes, is refused before the run instead of after it.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
shipmill-romamo Bot and others added 2 commits October 8, 2026 19:31
A key passed as ?api-key= in --base-url was accepted and written to the
investigation record. The URL is refused before the run, as one with a
username or password is, and the refusal does not echo it.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Two runs in one second clashed on the investigation id, and the clash
was reported as MODEL_UNAVAILABLE, losing the second record. The store
now writes exclusively and gives a later record a numeric suffix
(investigation-YYYYmmddHHMMSS-2), and a failure to record exits
PRECONDITION instead of 92.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@shipmill-romamo
shipmill-romamo Bot merged commit cb367d9 into main Oct 8, 2026
1 check passed
@shipmill-romamo
shipmill-romamo Bot deleted the feat/agent-investigate branch October 8, 2026 16:36
@shipmill-romamo shipmill-romamo Bot added this to the 0.6.0 milestone Oct 8, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant