fix: preserve unfinished reasoning across renderers and bridges

[hallerite]

When a generation prompt already opens reasoning, a completion can end before emitting the closing marker. The parser currently treats that unfinished reasoning as the final answer, and a bridge can carry the open reasoning block into the next turn.

This PR preserves unfinished reasoning across all supported reasoning formats and closes it safely when building the next prompt.

Closes #137 — unfinished reasoning returned as final content.
Closes #139 — reasoning left open by bridge_to_next_turn.

Parsing

For example, with a prompt ending in <think> and a completion of Let me work this out, parsing returns:

ParsedResponse(
    content="",
    reasoning_content="Let me work this out",
    reasoning_complete=False,
)

For reasoning-first formats, reasoning starts only with an initial <think> after any assistant header, or when the prompt already opened the block. The first </think> switches to content. Later markers remain literal content: Before<think>example</think>After is entirely content. Harmony, Gemma, and Inkling follow their explicit channel or message-segment grammar.

Tool-call drafts inside unfinished reasoning also stay in reasoning_content.

parse_response accepts optional prompt_ids to identify the starting channel, including partial assistant prefixes and Gemma's post-tool reasoning. Omitting prompt_ids, passing None, and passing [] all mean the completion is self-contained: reasoning needs its own initial opener or channel header. Generation settings do not supply missing parsing context. Callers parsing a continuation whose prompt supplied the opener must pass that prompt. reasoning_complete is True for closed reasoning and None when the state cannot be determined. An empty final answer can still have closed reasoning; consumers decide whether an answer is useful.

The generate client supplies prompt context and forwards reasoning_complete. Custom renderers used with generate must accept the optional prompt_ids keyword.

Bridging

Each renderer appends its missing reasoning close before continuing the conversation. Added tokens are prompt scaffolding, marked non-sampled and non-content. The original prompt and sampled completion remain an exact prefix.

If a sampled stop prevents that repair without changing existing tokens, the bridge returns None for a full-render fallback. Parsing and bridging share boundary detection independently; each format owns its closing rules.

Validation

Regression coverage exercises every reasoning renderer with thinking enabled and disabled, reasoning opened in the prompt or completion, partial and closed prompt prefixes, complete or truncated output, and literal think markers after content begins. It also checks tool arguments and token spans, tool drafts, literal marker lookalikes, empty final answers, and exact bridge prefixes.

• Full offline suite: 10,755 passed, 70 skipped.
• Ruff lint and formatting checks passed.

---

Note

Medium Risk
Broad behavioral change to response parsing, multiturn bridging, and the parse_response protocol across many model families; incorrect adoption of prompt_ids could mis-classify reasoning vs content in custom integrations.

Overview
Adds shared reasoning-boundary detection (scan_reasoning, prompt_ends_in_reasoning in renderers/reasoning.py) and threads it through parse and bridge paths for every reasoning-capable renderer.

Parsing: parse_response now accepts optional prompt_ids so completions that continue a prompt-opened reasoning channel (e.g. prefilled ``, Gemma post-tool thought, Harmony analysis) are not treated as final content or executable tool calls when the close marker is missing. ParsedResponse.reasoning_complete reports whether the reasoning channel is closed (`False` when truncated inside reasoning). Self-contained parsing is unchanged when `prompt_ids` is omitted, `None`, or `[]`. Token-level parsers and `ThinkTextReasoningParser` only treat an initial think opener as reasoning; later markers stay literal content.

Bridging: bridge_to_next_turn checks the prior completion for an open reasoning region and appends the format’s closing marker as non-loss prompt context before the turn close; if a stop token was already sampled inside open reasoning, it returns None (no silent repair). The generate client passes prompt ids into parse_response and exposes reasoning_complete on the response. Examples and README document the contract.

^{Reviewed by Cursor Bugbot for commit be74213. Bugbot is set up for automated code reviews on this repo. Configure here.}

Note

Preserve unfinished reasoning in renderer parsers and multiturn bridges

• Adds an optional prompt_token argument to Renderer.parse_response in base.py so parsers can detect reasoning opened by the generation prompt
• Parsers for Qwen, Kimi, GLM, DeepSeek, MiniMax, Gemma, and GPT-OSS now return incomplete reasoning-only output when a reasoning boundary is open instead of misinterpreting it as content or tool calls
• Multiturn bridges across renderers now append reasoning-close tokens for open boundaries or return None when a stop token is already present
• Adds a reasoning_complete status field to ParsedResponse in base.py and forwards it through the generation client in client.py
• Behavioral Change: The Renderer.parse_response protocol method signature changes to accept prompt_token. Out-of-tree implementations must add the new parameter to match the interface

Changes since #152 opened

• Modified reasoning boundary detection in parse_gpt_oss function, Gemma4Renderer.parse_response method, and GptOssRenderer.bridge_to_next_turn method to track whether reasoning extends to the end of the token stream [be74213]
• Removed unfinished_reasoning function from renderers.parsing module and inlined its behavior at call sites [be74213]
• Added parameterized tests in tests.test_reasoning_boundaries module validating GPT-OSS reasoning boundary behavior [be74213]

^{Macroscope summarized 94689c2.}

[cursor]

Stale Bugbot comment from a previous run.

[macroscopeapp]

Approvability

Verdict: Not approved

Macroscope's review found this PR not approvable — This PR makes broad production-path changes to response parsing, tool-call classification, multiturn bridging, and the public renderer protocol across many model families. The new shared reasoning state and prompt-context behavior warrant human review for cross-renderer compatibility and API integration impact.

Notes:

• Macroscope's correctness review did not run, so approvability was decided on eligibility alone.

^{You can add or adjust custom eligibility rules. Learn more.}

[cursor]

Stale Bugbot comment from a previous run.

[cursor]

Cursor Bugbot has reviewed your changes and found 2 potential issues.

^{❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.}

^{Reviewed by Cursor Bugbot for commit 04a9321. Configure here.}

[garrett361]

I'm a little confused about the difference between renderer.parse_response(ids, prompt_ids=[]) vs renderer.parse_response(ids, prompt_ids=None). These code paths are treated differently. Can you clarify the intention here? Haven't understood when each is appropriate.

[hallerite]

good catch, this is a smell. the intention was to be able to tell apart “you weren’t given the prompt, so assume the renderer’s usual prompt” (the None) and “assume nothing opened reasoning beforehand.” (the empty list).

but actually, parsing shhould always depend on the supplied tokens, so reworking the API to make this clearer.