Skip to content

Investigation record: answer and raw tool-call arguments are still unbounded #57

Closed

Description

@shipmill-romamo

What happens

Found in the review of #52 (#50). #52 caps each turn's reasoning and text in spec.trace at 8000 characters (D-4), but two fields of the investigation record still have no length limit:

  1. spec.answer (config/schemas/v1/agent-investigation.schema.json, set in sdk/src/cloudfall/investigation.py from the model's final text) holds the whole final answer, so the last turn's text is stored uncut in the record even though its trace copy is now capped
  2. trace[].toolCalls[].arguments keeps the model's raw argument string unbounded

Both can quote host output, the same reason D-4 capped reasoning.

Question

Should these get the same 8000-character cap with the cut marked? answer is what the finding is read from, so cutting it may need a different rule (keep the start, or refuse instead of cut).

Activity

  1. shipmill-romamo commented on Oct 9, 2026

    @shipmill-romamo
    ContributorAuthor

    Triage: implement, once the rule is settled. Waits on your decision in the next comment. Confirmed on main: neither spec.answer nor trace[].toolCalls[].arguments has a maxLength, and investigation.py stores both as the model returned them. The change bounds stored state and a published schema, so it goes through the design gate (D-4 covers only the trace's reasoning and text).

  2. shipmill-romamo commented on Oct 9, 2026

    @shipmill-romamo
    ContributorAuthor

    @romamo Decision needed: how should the investigation record bound spec.answer and the raw toolCalls[].arguments?

    1. Cap both at 8000 characters like D-4, but cut answer from the end (keep its start, where the finding is) and the arguments from the end too, with the cut marked (recommended)
    2. Cap both exactly like D-4 (keep the end)
    3. Cap only the arguments; leave answer whole, since it's the finding and the model writes it, not the host

    Reply here with a number or your own answer; shipmill takes this up on the tick after your reply.

  3. romamo commented on Oct 9, 2026

    @romamo
    Collaborator

    1 — cap both spec.answer and toolCalls[].arguments at 8000 characters, keeping the start and marking the cut

    (decision by @romamo, given in chat; posted by Claude on his behalf)

  4. shipmill-romamo commented on Oct 9, 2026

    @shipmill-romamo
    ContributorAuthor

    Triage: implement, with the answer above (option 1), recorded as D-6 in #59.

    Design:

    • spec.answer and each trace[].toolCalls[].arguments keep at most 8000 characters, keeping the start (unlike D-4's reasoning cap), and mark the cut the same way D-4 does
    • config/schemas/v1/agent-investigation.schema.json bounds both fields with a maxLength that allows the cut marker
    • Records already on disk with a longer answer would no longer validate; the PR says how the schema handles them

    A PR follows.

  5. removed
    needs-decisionA shipmill session asked a question here; waits for a reply
    on Oct 9, 2026
  6. shipmill-romamo commented on Oct 9, 2026

    @shipmill-romamo
    ContributorAuthor

    PR: #62. spec.answer and trace[].toolCalls[].arguments keep at most 8000 characters, keeping the start and ending with [last N characters cut]. The schema bounds both with maxLength: 8000. Only the stored copy is cut: the finding is still parsed from the whole answer and the tool runs on the whole arguments. So a record can hold a finding whose lines are past the cut, and capping never turns a long answer into unstructured.

    Still unbounded and out of scope here: spec.steps[].arguments, finding.rootCause and finding.why. That is filed separately.

  7. shipmill-romamo commented on Oct 9, 2026

    @shipmill-romamo
    ContributorAuthor

    Released in v0.6.0rc10.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions