feat(conversation): reply steps as ai_* attachments, live reasoning deltas, client tools and paced live updates

[martinmitrevski]

Stacked on athena/live-reasoning-ephemeral-runtime (where #683 landed).

A persistent reply's steps (rounds of reasoning, tool calls) become Stream attachments, its live reasoning streams as deltas instead of a resent 4 KB tail, a caller can declare tools that run on a person's device, and live updates stay within Stream's limits.

What changes

Reasoning windows. The ephemeral reasoning field was the latest 4,000 bytes, resent on every update. It is now {id, offset, text, length} for the reasoning step id: the new thinking since the last delivered window, in Unicode scalars, which clients append.

• At most 2,000 bytes per window; a longer backlog goes out over several updates.
• Every 3 s a window repeats the last 1,500 bytes, so a watcher who opened the conversation midway or reconnected catches up.
• The last thoughts go out once after the reply settles.

Reply steps as attachments. Each round of thinking is an ai_reasoning attachment and each visible tool call an ai_tool_call attachment, in order, followed by the artifacts. They ride on the live updates and are stored with the settled reply.

• A completed round stores its first sentence (summary) and first 500 characters (preview); the rest of the thinking is only shown live.
• Tool calls use the provider's call ID. Server tools never show arguments or results.
• Everything fits Stream's limit of 30 attachments and 5 KB per message: artifacts are kept whole, and older steps' previews and summaries give way first.

Client tools.

• SessionTool gains executor (server or client) and display_title.
• RespondRequest gains client_id, the install a command came from, written on the person's message.
• A client tool's call is shown as awaiting_client, addressed to the command's initiator and install, with its arguments (at most 512 bytes). The caller answers it over the events socket once the device reports, and a summary (or the device's failure reason) is shown on the step.

Pacing and backoff.

• The answer goes out at most every 150 ms (Stream throttles message.updated to 10/s per channel) and thinking alone at most every 200 ms, on a 50 ms tick.
• After a refused live update the next one waits: Retry-After or the rate-limit window's reset on a 429 (at most a minute), otherwise 1 s doubling to 8 s. Stored writes keep their own retry, and nothing held back is lost.

Fix. A data race between sending a reply's steps and updating them. The send and the session broadcast shared the live steps' array; they now copy it.

Generated code. The OpenAPI spec changes, so internal/api/generated.go and the Go and JS SDKs are regenerated. The Go SDK also picks up the earlier image endpoint it had missed.

Verification

• internal/conversation (new tests: windows, keyframes, backlog, gaps, steps, client-tool targeting, attachment limits, pacing, a 429 backoff against the fake Stream server); internal/conversation ×3, internal/session and internal/api with -race. internal/agent and internal/harness pass without -race.
  • TestAgentSuite/TestASecondToolDoesNotStartACompetingReply fails under -race on this branch's base as well.
• Local Athena stack, DeepSeek V4 Pro:
  • 27 KB of reasoning over 105 s arrived in 588 windows with no gaps: 75.7 KB sent, against about 3.9 MB under the old scheme.
  • Live updates: the answer at 6.2/s, thinking-only at a 196 ms median, no failed updates. The ephemeral endpoint reported a 10,000/min app budget.
  • A device tool call went awaiting_client → completed through the iOS app.
• Not verified: web search between rounds (no Exa key locally), a runtime restart mid-reply, anything hosted.

Athena client and API: GetStream/athena-ai#14. Stream AI components: stream-chat-swift-ai 0.9.0.

🤖 Generated with Claude Code