Skip to content

feat(ai-components): message views, reply actions and icon overrides - #3928

Open
isekovanic wants to merge 1 commit into
feat/ai-agent-nativefrom
feat/ai-message-views
Open

isekovanic wants to merge 1 commit into
feat/ai-agent-nativefrom
feat/ai-message-views

Conversation

@isekovanic

Copy link
Copy Markdown
Contributor

Ticket: https://linear.app/stream/issue/RN-485

Stacked on #3926.

🎯 Goal

Show the person's messages and the agent's replies the way the native AI SDKs do, with the actions under a finished reply. The APIs come from React's AI components (#3376). The look follows StreamChatAI and the Android AI SDK.

πŸ›  Implementation details

Views (props only, under message/)

  • AIUserMessage: the person's attachments, then their text in a bubble on the trailing side. With onEdit it shows an edit button.
  • AIAssistantMessage: a reply's steps, then the answer (typed out while it streams), then what the agent attached.
  • AIMessageAttachment:
    • An image is a 180 Γ— 180 thumbnail that doesn't open on tap, as on both native SDKs.
    • A file is a card with its type's icon (PDF, image, audio, video, archive or file), name, type and size. One at a web URL opens in the browser.
  • AIReplyActions: copy (shows a check for a moment), share through the system sheet, regenerate with onRegenerate, and a thumbs up/down rating with rating / onRate. Copy is hidden when the app has no clipboard module.

Customising

  • The AI theme has a new optional message section: styles, and colours under StreamChatAI's names (userMessageBackground, attachmentIconBackground, messageAction, …). The defaults come from the design tokens.
  • The theme also takes icons, which override any of the agent replies' 24 icons. This covers the step views from earlier PRs as well as the new ones. defaultAIIcons and useAIIcons are exported. mergeThemes carries icons through as they are, since the JSON copy it makes would drop them.
  • AIUserMessage and AIAssistantMessage take an Attachment component, like the attachment view factories on Swift and Android.

Also

  • Fixed: copying crashed with current expo-clipboard, which no longer has setString. The AI clipboard service now uses setStringAsync when it's there, as the core Expo package already does. This affected copying code blocks too.
  • New strings are in all 13 locales. As before, ar and he could use a native speaker's look.

Left for later

  • The native SDKs offer Copy and Edit on the person's message through a long-press menu (Swift's context menu, Android's dropdown). Until then, onEdit shows a button, as in React. If we want Swift's lifted preview, the menu needs a portal host of its own, which AIConversation can own in a later PR.

ExpoChatGPTSample

  • Renders the person's messages with AIUserMessage, and replies with AIAssistantMessage plus AIReplyActions once the router clears generating.
  • Ratings are kept for the session only, as in Juno: the router doesn't take ratings yet. Like Juno, the sample doesn't wire edit or regenerate.

🎨 UI Changes

The person's bubble, replies with their steps and action row, and image and file attachments.

πŸ§ͺ Testing

  • Unit tests cover:
    • the four views;
    • file size and details formatting;
    • the file type icons;
    • icon overrides, and icons surviving mergeThemes;
    • the Attachment slot.
  • On an Android emulator and the iOS simulator, against a local Vision-Agents router:
    • a web-search reply with its steps;
    • the actions appearing once the reply finished;
    • rating;
    • copy (pasting confirmed the text);
    • the share sheet;
    • image and PDF attachments;
    • the PDF opening in Chrome;
    • an image tap doing nothing.
  • The attachments were posted straight to the channel. Asking with a photo fails at the router ("personal conversations require a command ID") because of how the sample sends photos with calypso 0.2.3, not because of these views.
  • Not tested: edit and regenerate on a device (the sample doesn't use them), RTL and screen readers.
  • yarn lint, yarn typecheck, yarn build and yarn test:unit pass.

β˜‘οΈ Checklist

@Stream-SDK-Bot

Copy link
Copy Markdown
Contributor

SDK Size

title develop branch diff status
js_bundle_size 2037 KB 2038 KB +213 B 🟒
ai_components_size 163 KB 226 KB +65237 B πŸ”΄

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants