fix(coding-agent): report a subagent's output-token-limit stop as failed, not completed - #173
Closed
jasonkneen wants to merge 1 commit into
Closed
jasonkneen wants to merge 1 commit into
jasonkneen wants to merge 1 commit into
Conversation
…led, not completed Invariant: hitting a limit is a distinct terminal status, never "completed". A child whose final assistant message stopped with stopReason "length" (output token limit; the answer is truncated and agent-loop refuses tool calls from truncated messages) had exitCode 0 and a non-error/aborted stopReason, so isFailed() returned false and statusForResult() reported it as "completed" to the parent, which then treated the truncated partial answer as a finished result. isFailed() now also treats stopReason "length" as failure, which statusForResult, the success count, telemetry, the TUI icon, and background-lane routing all derive from for free. parseJsonEvent now fills in a default truncation errorMessage when the provider doesn't supply one, and resultText appends the partial output after the truncation message so the parent still sees what the child produced.
There was a problem hiding this comment.
Copilot review overview
🔵 Needs a closer look
Always include the canonical truncation message, even when a provider supplies an error message.
Review effort: Lite
Findings: None
What changed in this PR
Updates subagent handling so output-token-limit stops are reported as failures while preserving truncated output.
Changes:
- Treats
stopReason: "length"as a failure. - Adds truncation messaging and partial-output propagation.
- Adds regression and end-to-end test coverage.
| File | Summary |
|---|---|
packages/coding-agent/test/subagent-length-status.test.ts |
Adds coverage for classification, parsing, output, and tool results. |
packages/coding-agent/src/features/subagent/execute.ts |
Exports status classification for direct testing. |
packages/coding-agent/src/features/step-subagent.ts |
Handles length-stop failures and truncated output. Moderate issue (1 vote): provider-supplied error messages can omit the canonical truncation message. |
💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
packages/coding-agent/src/features/step-subagent.tsisFailed(~292) only treatedexitCode !== 0andstopReason"error"/"aborted"as failure. A child whose final assistant message stopped withstopReason: "length"(the provider's output token limit — the answer is truncated mid-generation, andagent-loop.tsrefuses to execute any tool calls the truncated message may contain, seeagent-loop.ts:240-246) hasexitCode: 0and a non-error/abortedstopReason, soisFailedreturnedfalse.statusForResultinpackages/coding-agent/src/features/subagent/execute.ts(~56-59) then reported the record as"completed", and the parent (blockingsubagenttool result,resultText/summaries inexecute.ts~400-410, and background-lane notifications vialane-events.tslaneOutputText/notifyLaneFinal) treated the truncated partial answer as a finished result — violating the invariant that hitting a limit must be a distinct terminal status, never"completed".packages/coding-agent/src/features/workflow/agent-runner.tsalso calls the sharedisFailed, so a workflow step with a truncated child had the same bug.Stop-reason propagation itself was already correct:
parseJsonEvent(step-subagent.ts) copiesmessage.stopReasonfrom eachmessage_end/tool_result_endevent into the runningStepSubagentRunResult, andrpc-adapter.ts'ssettleTurnonly overwritesstopReasonfor the aborted case, so a"length"stop from the last assistant message already reached the parent'sStepSubagentRunResult.stopReason. The only propagation gap waserrorMessage: providers don't set one for a plain length stop, so the record had no message describing the truncation.Fix
isFailednow also treatsstopReason === "length"as a failure. This is a minimal, single-source-of-truth change:statusForResult(subagent/execute.ts) already derives its status fromisFailed, so the success/failure classification, the success count (records.filter(status === "completed")), telemetry (reportFinished), the TUI status icon, and background-lane routing (background_donevsbackground_failedinlane-events.ts) all pick up the new outcome for free — no separate status enum was introduced, since the existing 4-value union (running/completed/failed/aborted) already has a distinct non-"completed" terminal value and is switched on in several small consumers.parseJsonEventnow fills in a defaulterrorMessage("Subagent response hit the output token limit; result is truncated.") when the final assistant message'sstopReasonis"length"and the provider didn't supply its ownerrorMessage. A provider-suppliederrorMessageis preserved.resultTextnow special-cases a"length"stop: it returns the truncation message followed by the partial assistant output (viafinalOutput), so the parent-visible text (single-task result, parallel-task summaries, and background-lane<agent-notification>bodies, which all route throughresultText) clearly says the answer was truncated and still surfaces the partial content instead of hiding it behind a bare error string.statusForResultfromsubagent/execute.ts(was module-private) so it has direct test coverage.Test
New file
packages/coding-agent/test/subagent-length-status.test.ts:isFailedtreats a length stop as failure / a normal stop as success.statusForResultreports a length stop as"failed"(not"completed") and a normal stop as"completed".resultTextfor a length stop contains "truncat" and the partial output text; unaffected for a normal stop.parseJsonEventsets the default truncationerrorMessagewhen the child provides none, and preserves a provider-suppliederrorMessage.createStepSubagentExtension'ssubagenttool (parallel mode, one task) with a mock runner returning a length-stopped result: asserts the tool's returned text contains"0/1 succeeded", mentions truncation, includes the partial output, anddetails.results[0].status === "failed".Before the fix,
expect(isFailed({ exitCode: 0, stopReason: "length" })).toBe(true)failed (expected false to be true),expect(statusForResult(...)).not.toBe("completed")failed (expected 'completed' not to be 'completed'), and theresultTextassertion failed (expected 'partial answer cut off mid-sen' to contain 'truncat'). All 9 new tests pass after the fix (npx vitest --run test/subagent-length-status.test.ts).Risk / behaviour change
workflow/agent-runner.ts'sWorkflowAgentRunResult.statusis now"failed"(not"completed") for a length-truncated workflow step, since it sharesisFailed. This is the correct behavior per the invariant, but it is a visible change: a workflow step that previously silently passed truncated output downstream as"completed"will now fail that step.if (record.status !== "completed") break;inexecute.ts) now stop the chain on a truncated step instead of feeding the truncated text into the next task's{previous}placeholder.reportFinishedmaps the new "failed" outcome tostatus: "error"(existing mapping, unchanged code) rather than"completed"."running" | "completed" | "failed" | "aborted"union, so all existing switches/filters overStepSubagentResultRecord["status"]/BackgroundAgentLane["status"]needed no changes.https://claude.ai/code/session_01GUdnnHEaDThHUATSwXBpV9