Skip to content

fix(coding-agent): report a subagent's output-token-limit stop as failed, not completed - #173

Closed
jasonkneen wants to merge 1 commit into
stepfun-ai:mainfrom
jasonkneen:fix/harness-subagent-length-status
Closed

jasonkneen wants to merge 1 commit into
stepfun-ai:mainfrom
jasonkneen:fix/harness-subagent-length-status

Conversation

@jasonkneen

Copy link
Copy Markdown

Problem

packages/coding-agent/src/features/step-subagent.ts isFailed (~292) only treated exitCode !== 0 and stopReason "error"/"aborted" as failure. A child whose final assistant message stopped with stopReason: "length" (the provider's output token limit — the answer is truncated mid-generation, and agent-loop.ts refuses to execute any tool calls the truncated message may contain, see agent-loop.ts:240-246) has exitCode: 0 and a non-error/aborted stopReason, so isFailed returned false. statusForResult in packages/coding-agent/src/features/subagent/execute.ts (~56-59) then reported the record as "completed", and the parent (blocking subagent tool result, resultText/summaries in execute.ts ~400-410, and background-lane notifications via lane-events.ts laneOutputText/notifyLaneFinal) treated the truncated partial answer as a finished result — violating the invariant that hitting a limit must be a distinct terminal status, never "completed".

packages/coding-agent/src/features/workflow/agent-runner.ts also calls the shared isFailed, so a workflow step with a truncated child had the same bug.

Stop-reason propagation itself was already correct: parseJsonEvent (step-subagent.ts) copies message.stopReason from each message_end/tool_result_end event into the running StepSubagentRunResult, and rpc-adapter.ts's settleTurn only overwrites stopReason for the aborted case, so a "length" stop from the last assistant message already reached the parent's StepSubagentRunResult.stopReason. The only propagation gap was errorMessage: providers don't set one for a plain length stop, so the record had no message describing the truncation.

Fix

  • isFailed now also treats stopReason === "length" as a failure. This is a minimal, single-source-of-truth change: statusForResult (subagent/execute.ts) already derives its status from isFailed, so the success/failure classification, the success count (records.filter(status === "completed")), telemetry (reportFinished), the TUI status icon, and background-lane routing (background_done vs background_failed in lane-events.ts) all pick up the new outcome for free — no separate status enum was introduced, since the existing 4-value union (running/completed/failed/aborted) already has a distinct non-"completed" terminal value and is switched on in several small consumers.
  • parseJsonEvent now fills in a default errorMessage ("Subagent response hit the output token limit; result is truncated.") when the final assistant message's stopReason is "length" and the provider didn't supply its own errorMessage. A provider-supplied errorMessage is preserved.
  • resultText now special-cases a "length" stop: it returns the truncation message followed by the partial assistant output (via finalOutput), so the parent-visible text (single-task result, parallel-task summaries, and background-lane <agent-notification> bodies, which all route through resultText) clearly says the answer was truncated and still surfaces the partial content instead of hiding it behind a bare error string.
  • Exported statusForResult from subagent/execute.ts (was module-private) so it has direct test coverage.

Test

New file packages/coding-agent/test/subagent-length-status.test.ts:

  • isFailed treats a length stop as failure / a normal stop as success.
  • statusForResult reports a length stop as "failed" (not "completed") and a normal stop as "completed".
  • resultText for a length stop contains "truncat" and the partial output text; unaffected for a normal stop.
  • parseJsonEvent sets the default truncation errorMessage when the child provides none, and preserves a provider-supplied errorMessage.
  • An end-to-end test driving createStepSubagentExtension's subagent tool (parallel mode, one task) with a mock runner returning a length-stopped result: asserts the tool's returned text contains "0/1 succeeded", mentions truncation, includes the partial output, and details.results[0].status === "failed".

Before the fix, expect(isFailed({ exitCode: 0, stopReason: "length" })).toBe(true) failed (expected false to be true), expect(statusForResult(...)).not.toBe("completed") failed (expected 'completed' not to be 'completed'), and the resultText assertion failed (expected 'partial answer cut off mid-sen' to contain 'truncat'). All 9 new tests pass after the fix (npx vitest --run test/subagent-length-status.test.ts).

Risk / behaviour change

  • workflow/agent-runner.ts's WorkflowAgentRunResult.status is now "failed" (not "completed") for a length-truncated workflow step, since it shares isFailed. This is the correct behavior per the invariant, but it is a visible change: a workflow step that previously silently passed truncated output downstream as "completed" will now fail that step.
  • Chain-mode subagent calls (if (record.status !== "completed") break; in execute.ts) now stop the chain on a truncated step instead of feeding the truncated text into the next task's {previous} placeholder.
  • Telemetry's reportFinished maps the new "failed" outcome to status: "error" (existing mapping, unchanged code) rather than "completed".
  • No new status value was added to the "running" | "completed" | "failed" | "aborted" union, so all existing switches/filters over StepSubagentResultRecord["status"] / BackgroundAgentLane["status"] needed no changes.

https://claude.ai/code/session_01GUdnnHEaDThHUATSwXBpV9

…led, not completed

Invariant: hitting a limit is a distinct terminal status, never
"completed". A child whose final assistant message stopped with
stopReason "length" (output token limit; the answer is truncated and
agent-loop refuses tool calls from truncated messages) had exitCode 0
and a non-error/aborted stopReason, so isFailed() returned false and
statusForResult() reported it as "completed" to the parent, which then
treated the truncated partial answer as a finished result.

isFailed() now also treats stopReason "length" as failure, which
statusForResult, the success count, telemetry, the TUI icon, and
background-lane routing all derive from for free. parseJsonEvent now
fills in a default truncation errorMessage when the provider doesn't
supply one, and resultText appends the partial output after the
truncation message so the parent still sees what the child produced.
Copilot AI lite review requested due to automatic review settings September 23, 2026 11:42

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot review overview

🔵 Needs a closer look

Always include the canonical truncation message, even when a provider supplies an error message.

Review effort: Lite
Findings: None

What changed in this PR

Updates subagent handling so output-token-limit stops are reported as failures while preserving truncated output.

Changes:

  • Treats stopReason: "length" as a failure.
  • Adds truncation messaging and partial-output propagation.
  • Adds regression and end-to-end test coverage.
File Summary
packages/​coding-agent/​test/​subagent-length-status.test.ts Adds coverage for classification, parsing, output, and tool results.
packages/​coding-agent/​src/​features/​subagent/​execute.ts Exports status classification for direct testing.
packages/​coding-agent/​src/​features/​step-subagent.ts Handles length-stop failures and truncated output. Moderate issue (1 vote): provider-supplied error messages can omit the canonical truncation message.

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

@ZouR-Ma ZouR-Ma closed this Sep 23, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants