Skip to content

Record the investigation when the model endpoint fails partway - #32

Merged
shipmill-romamo[bot] merged 1 commit into
mainfrom
fix/29-record-on-model-failure
Oct 8, 2026
Merged

shipmill-romamo[bot] merged 1 commit into
mainfrom
fix/29-record-on-model-failure

Conversation

@shipmill-romamo

Copy link
Copy Markdown
Contributor

Summary

cloudfall agent investigate saved its AgentInvestigation only after the loop returned, so a model endpoint failing partway (exit 92) left decisions from earlier turns without the investigation that proposed them.

  • AgentInvestigation.spec.status gains model-unavailable (additive; the schema has only shipped in 0.6.0rc5)
  • investigate() catches any InvestigationError once the run started and raises ModelUnavailableError, which carries the investigation so far: steps, turns and tokens, answer: "", no finding
  • agent investigate saves that record, then still exits 92 MODEL_UNAVAILABLE with the record's id in error.context.investigation; a failure to save it exits PRECONDITION, as the success path does
  • The chat transport raises InvestigationError (same codes and messages as before) instead of Exit.MODEL_UNAVAILABLE, so every mid-run failure goes through one path: refused status, unreachable, not JSON, JSON that is not an object, no choices, a malformed tool call
  • A failure on the very first request records an investigation with turns: 1 and no steps (the request was sent). A refusal before any request (exit 2) still records nothing
  • The record stores no error text: an endpoint's error body can echo the request

The 93 INVESTIGATION_INCOMPLETE and success paths are unchanged.

Tests

  • New CLI test against the local OpenAI-compatible server: turn 1 runs a read operation (recording a decision), turn 2 fails in five ways (500 echoing the key, HTML, [], no choices, a tool call without a function). Each exits 92, names a record that is model-unavailable with one step, 2 turns, the turn-1 tokens, no finding, and sk-test appears neither in the record nor in the command's output
  • New unit test with the scripted transport for the refused and malformed-tool-call cases, saving the record through the schema
  • The unreachable-endpoint test now asserts the zero-step record
  • uv run ruff check ., uv run mypy, uv run pytest (391 passed), uv run cloudfall config validate --project config/examples

Fixes #29

🤖 Generated with Claude Code

agent investigate now saves its AgentInvestigation, ended model-unavailable
with the steps, turns and tokens so far, before exiting 92
MODEL_UNAVAILABLE, and names it in error.context.investigation. Decisions
proposed in earlier turns are no longer left without the investigation
that produced them. The chat transport raises InvestigationError instead
of exiting, so investigate() can keep the run up to the failure.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@shipmill-romamo
shipmill-romamo Bot merged commit 6d4c9e9 into main Oct 8, 2026
1 check passed
@shipmill-romamo
shipmill-romamo Bot deleted the fix/29-record-on-model-failure branch October 8, 2026 16:51
@shipmill-romamo shipmill-romamo Bot added this to the 0.6.0 milestone Oct 8, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

agent investigate: a model failure partway through a run leaves no investigation record

0 participants