Skip to content

feat(ai-components): agent reply steps (reasoning, tool calls, approvals) - #3924

Open
isekovanic wants to merge 1 commit into
feat/ai-agent-groundworkfrom
feat/ai-reply-steps
Open

isekovanic wants to merge 1 commit into
feat/ai-agent-groundworkfrom
feat/ai-reply-steps

Conversation

@isekovanic

Copy link
Copy Markdown
Contributor

Ticket: https://linear.app/stream/issue/RN-485

Stacked on #3921.

🎯 Goal

Show an agent reply's steps: its rounds of reasoning, its tool calls, and the questions a tool asks before it runs. The views are ported from React's AI components (#3376). Effects follow the native SDKs instead.

🛠 Implementation details

Views (all props-only, none of them reads <Channel>)

  • AIReasoning shows a header that counts up while the agent thinks ("Thinking... 7s") and then reads "Thought for 12s" with the step's summary.
    • The panel opens while the agent thinks and follows the reasoning as it streams in, then folds once the step is done. Tapping the header opens or folds it.
    • The first step takes over the turn's thinking clock, so the time carries on from AIThinking.
  • AIThinking is the same header for a turn that hasn't shown any reasoning yet.
  • AIToolCall shows the call's title, a detail line, a status icon and its duration. The detail line covers waiting for approval, a sign-in or the device, and declined, didn't complete or cancelled.
  • AIToolApproval and AIToolApprovalCard ask the person a tool's question, such as "Share your location?", with the agent's reason and Allow / Don't Allow.
    • The question is only asked of the targeted user on the targeted install.
    • The buttons wait while the answer is sent, and a failed send lets the person answer again.
  • AIMessageParts / AIMessagePart render a reply's steps in order, with a neutral line for a kind this version can't show.
  • AIToolApproverProvider / useAIToolApprover say who answers the questions on this device.

Shared bits

  • In-progress states use our NativeShimmerView through a small internal Shimmer, which AITypingIndicatorView now uses too. Running calls use the platform ActivityIndicator. This matches Swift and Android, not React's CSS.
  • Durations come out as "0.4s" and "1m 5s" on both platforms. Hermes on iOS ignores narrow units in Intl, so we fall back the same way React does.
  • There's a new, optional messageParts section in the AI theme.
  • New strings are added to all 13 locales. The thinking header reuses the existing Thinking... key. The ar and he translations are new and could use a native speaker's look.

ExpoChatGPTSample

  • A reply renders its steps above its text.
  • Live reasoning comes from calypso's LiveReasoning, fed by message.updated, and "thinking" comes from the session's state.
  • Approvals go through session.decide.
  • The drawer lists the router's sessions.
  • The steps take the row's full width. The stand-in MessageList sizes bubbles to their content, and AIConversation replaces it later anyway.

🎨 UI Changes

Reasoning steps (folded, open, streaming), tool calls (running, waiting, done, failed, declined) and the approval card.

🧪 Testing

  • Unit tests are ported from React's AIMessagePartViews suite, plus duration formatting, including the iOS fallback.
  • On an Android emulator against a local Vision-Agents router:
    • thinking, the reasoning steps folding and opening, and a web search with its spinner, check and duration;
    • the approval card: Share runs the call, Not now declines it, and no answer shows the call as failed after the router's 30s;
    • reasoning streaming into an open panel. Our router's models send their thinking in one piece, so this was driven with router-shaped events.
  • On the iOS simulator: a reply with a web search step and its duration.
  • To get an approval at all, the sample needs a phone-side tool, and the question has to carry the install id (/respond with client_id). calypso 0.2.3's responses.create doesn't send one. Both come with the client tool runner later in this stack.
  • Not tested: RTL and screen readers, on either platform.
  • yarn lint, yarn typecheck, yarn build and yarn test:unit pass.

☑️ Checklist

@Stream-SDK-Bot

Copy link
Copy Markdown
Contributor

SDK Size

title develop branch diff status
js_bundle_size 2037 KB 2038 KB +213 B 🟢
ai_components_size 163 KB 196 KB +34134 B 🔴

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants