Skip to content

feat(omni): Slack, WhatsApp, RCS, SMS and iMessage as Stream Chat messages - #738

Merged
Nash0x7E2 merged 3 commits into
acceleratefrom
cursor/omni-messaging-a9e9
Oct 3, 2026
Merged

Nash0x7E2 merged 3 commits into
acceleratefrom
cursor/omni-messaging-a9e9

Conversation

@tschellenbach

@tschellenbach tschellenbach commented Oct 3, 2026 •

Copy link
Copy Markdown
Member

Why

Sprint 4: an agent should answer Slack, WhatsApp, RCS, SMS and iMessage from one Stream channel per conversation. This adds vision-agents-plugins-omni (from vision_agents.plugins import omni).

Each provider reads its webhook into an OmniMessage, and to_stream turns that into a Stream MessageRequest with attachments and custom.omni. In the other direction, from_stream reads the agent's Stream reply back, and the provider's render builds the bodies for its send API.

Changes

  • Providers per channel. A channel can be carried by more than one provider. ProviderRegistry holds the providers by name and dispatches parse(name, payload) and render(message) on message.provider. It rejects a channel that provider doesn't carry.

    Provider Channels
    slack Slack
    whatsapp (Meta Cloud API) WhatsApp
    google_rbm RCS
    twilio SMS, plus WhatsApp and RCS when the sender has a whatsapp: or rcs: prefix
    linq iMessage, RCS, SMS, taken from each message's service

    To add a provider, subclass OmniProvider with a unique name and register it.

  • Message model. OmniMessage / OmniAttachment with kinds image, video, audio, file and location. Messages record their provider, which is also stored in custom.omni.

  • Stream mapping.

    • Message text stays the Stream text.
    • Files become image (image_url) or video / audio / file (asset_url) attachments, with mime_type, file_size and media_id on the attachment.
    • A shared place becomes a location attachment with latitude and longitude.
    • The readable ids (channel, provider, conversation, sender, account, thread, reply_to, sent_at) go in custom.omni.
    • from_stream(reply, route=inbound) sends a reply back the way the inbound message came, and skips ai_reasoning and ai_tool_call attachments.
  • Stream ids.

    • stream_channel_id is {provider}_{first 24 hex chars of a sha256 of account and conversation}. That's one channel per conversation per business number or account, and always at most 64 characters.
    • stream_user_id is {provider}_{digest of the sender}.
  • Slack. The provider remembers recent (channel, ts) pairs. A message delivered as both message and app_mention, or retried by Slack, is read once.

  • Workspace. The package is registered in the workspace, with a vision-agents[omni] extra. Its uv.lock entries are written by hand: uv lock doesn't resolve on accelerate because of an existing moondream/roboflow pillow conflict. UV_FROZEN=1 uv sync installs it.

Only pure translation is included. There are no HTTP senders, no credentials, and no webhook signature checks.

UV_FROZEN=1 uv run --no-sync dev.py check: ruff, the extras validation, and mypy for the core package and plugins all pass. Unit tests: 1689 passed, including the 45 omni tests. 44 tests error, and they error the same way without this change: the Redis testcontainer fixture needs Docker, which this VM doesn't have.

Open in Web Open in Cursor 

…es as Stream Chat messages

Co-authored-by: Thierry Schellenbach <thierryschellenbach@gmail.com>
cursoragent and others added 2 commits October 3, 2026 21:36
…ack message

- Channels are carried by providers behind a ProviderRegistry: Slack,
  WhatsApp Cloud API, Google RBM (RCS), Twilio (SMS, plus its whatsapp:
  and rcs: senders) and Linq (iMessage, RCS, SMS). Messages record their
  provider in custom.omni.
- Stream ids are {provider}_{sha256 digest}: channels per account and
  conversation, users per sender, always within Stream's limits.
- The Slack provider remembers recent messages, so one delivered as both
  message and app_mention, or retried, is read once.

Co-authored-by: Thierry Schellenbach <thierryschellenbach@gmail.com>
…saging-a9e9

Co-authored-by: Thierry Schellenbach <thierryschellenbach@gmail.com>
@tschellenbach
tschellenbach marked this pull request as ready for review October 3, 2026 23:29
@Nash0x7E2
Nash0x7E2 merged commit 5240b0e into accelerate Oct 3, 2026
17 checks passed
@Nash0x7E2
Nash0x7E2 deleted the cursor/omni-messaging-a9e9 branch October 3, 2026 23:46
kanat pushed a commit that referenced this pull request Oct 5, 2026
…sages (#738)

* feat(omni): standardize Slack, WhatsApp, RCS, SMS and iMessage messages as Stream Chat messages

Co-authored-by: Thierry Schellenbach <thierryschellenbach@gmail.com>

* feat(omni): providers per channel, hashed Stream ids, one read per Slack message

- Channels are carried by providers behind a ProviderRegistry: Slack,
  WhatsApp Cloud API, Google RBM (RCS), Twilio (SMS, plus its whatsapp:
  and rcs: senders) and Linq (iMessage, RCS, SMS). Messages record their
  provider in custom.omni.
- Stream ids are {provider}_{sha256 digest}: channels per account and
  conversation, users per sender, always within Stream's limits.
- The Slack provider remembers recent messages, so one delivered as both
  message and app_mention, or retried, is read once.

Co-authored-by: Thierry Schellenbach <thierryschellenbach@gmail.com>

---------

Co-authored-by: Cursor Agent <cursoragent@cursor.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants