From fad7604248052c13ced861cbd9489dd49689940c Mon Sep 17 00:00:00 2001 From: Andrew Bulat Date: Fri, 31 Jul 2026 02:33:42 +0100 Subject: [PATCH 1/4] Merge the AI Transport About and Why pages into one landing page Two pages at the top of the AI Transport nav were doing overlapping jobs: "About AI Transport" introduced the product, and "Why AI Transport" argued for it. A reader arriving from marketing or from another product's docs had to read both to get either answer, and each page repeated the other's framing. Collapsing them puts the high-level content in one place and lets the "Why AI Transport?" section sit above "Getting started", where the question it answers actually gets asked. The merged page keeps /docs/ai-transport, because that is the URL 26 inbound references already use as the product entry point, and moving it would have put a redirect on the highest-traffic path into the section. The page is written to answer necessity rather than comprehension. Two founder calls in late July concluded that readers understood the product and still could not see why they needed it before a V2, and that the scale-oriented framing actively invited "nice to have". So the page opens on the failure modes a reader already has, states that the threshold is response duration rather than traffic volume, shows both seams that change when retrofitting an application that already streams, and names the constraints and dependencies adoption brings. Several claims carried by the old pages are gone because the roadmap, the feature pages, and the SDK do not support them: LangChain and native model-provider integrations, human handover, two platform statistics that appear nowhere else in the docs, and a publish-order guarantee that message-ordering.mdx explicitly refutes for cross-region publishers. Co-Authored-By: Claude Opus 5 (1M context) --- src/data/nav/aitransport.ts | 27 ++- .../docs/ai-transport/concepts/index.mdx | 2 +- .../docs/ai-transport/going-to-production.mdx | 2 +- src/pages/docs/ai-transport/index.mdx | 181 +++++++++++++++--- src/pages/docs/ai-transport/roadmap.mdx | 2 +- .../why/http-streaming-and-ai.mdx | 3 +- src/pages/docs/ai-transport/why/index.mdx | 107 ----------- src/pages/docs/platform/products/index.mdx | 2 +- 8 files changed, 169 insertions(+), 157 deletions(-) delete mode 100644 src/pages/docs/ai-transport/why/index.mdx diff --git a/src/data/nav/aitransport.ts b/src/data/nav/aitransport.ts index e558d692b5..f742b2994c 100644 --- a/src/data/nav/aitransport.ts +++ b/src/data/nav/aitransport.ts @@ -9,13 +9,22 @@ export default { }, content: [ { - name: 'Getting started', + name: 'Why AI Transport?', pages: [ { - name: 'About AI Transport', + name: 'Overview', link: '/docs/ai-transport', index: true, }, + { + name: 'HTTP streaming and AI', + link: '/docs/ai-transport/why/http-streaming-and-ai', + }, + ], + }, + { + name: 'Getting started', + pages: [ { name: 'By SDK', pages: [ @@ -47,20 +56,6 @@ export default { }, ], }, - { - name: 'Why AI Transport?', - pages: [ - { - name: 'Overview', - link: '/docs/ai-transport/why', - index: true, - }, - { - name: 'HTTP streaming and AI', - link: '/docs/ai-transport/why/http-streaming-and-ai', - }, - ], - }, { name: 'Concepts', pages: [ diff --git a/src/pages/docs/ai-transport/concepts/index.mdx b/src/pages/docs/ai-transport/concepts/index.mdx index e6005499d2..6302073746 100644 --- a/src/pages/docs/ai-transport/concepts/index.mdx +++ b/src/pages/docs/ai-transport/concepts/index.mdx @@ -38,6 +38,6 @@ You do not need this section before you start building. Pick a [getting started ## Read next + +Most AI frameworks tie a turn to [one HTTP request](/docs/ai-transport/why/http-streaming-and-ai), so the conversation and the connection share a lifetime. That connection only ever joins one client to one agent. + +For a one-shot interaction that holds. The failures start where the two lifetimes come apart: + +| In production | Direct HTTP streaming | AI Transport session | +| --- | --- | --- | +| The connection drops mid-response | The response is gone. The model keeps generating tokens with nowhere to deliver them, and the user sees an error and starts over. | The agent keeps publishing into the session, and the client reconnects and [picks up where it stopped](/docs/ai-transport/features/reconnection-and-recovery). | +| The user presses stop | Closing the connection is the only signal available, and the agent cannot tell it apart from a network drop. | The client publishes a [cancel signal](/docs/ai-transport/features/cancellation), and the agent's current [run](/docs/ai-transport/concepts/runs) ends while the session stays open. | +| The agent process restarts mid-answer | The outbound stream ends with the process, and the client keeps a response that stopped halfway. | Your workflow engine retries the step, which publishes into [the same session](/docs/ai-transport/features/durable-execution), and the client keeps reading. | +| The user wants to know the agent is still working | A long pause looks the same as a stalled process. | The agent reports thinking, streaming, or idle through [presence](/docs/ai-transport/features/agent-presence) on the session. | +| The user opens the conversation on a phone | The stream belongs to the client that opened it, so a second device has no way in. | Every device [attaches to the same session](/docs/ai-transport/features/multi-device) and sees the conversation as it streams. | +| The user changes direction halfway through an answer | There is no upstream channel to send on. | The client [steers the active run](/docs/ai-transport/features/interruption-and-steering), cancels it and re-prompts, or [sends alongside it](/docs/ai-transport/features/double-texting). | +| The agent needs approval before it acts | The pending request lives on one connection, so a drop before anyone answers loses it. | The request [waits in the session](/docs/ai-transport/features/human-in-the-loop), and any user approves it minutes or hours later. | + +A single user on a train, moving from Wi-Fi to cellular and then to a laptop, runs into most of these on their own. The threshold is whether a response takes long enough to be interrupted, or a conversation is worth coming back to. + +You meet these failures one at a time and solve them one at a time. What accumulates is a buffer for stream resume, a database for conversation state, a queue or a second WebSocket for upstream signals, a status endpoint with a client polling it, and a [reconciliation path](/docs/ai-transport/why/http-streaming-and-ai) to merge stored state with the live stream every time a client joins. + +Each piece is tractable on its own. What does not get easier is keeping them correct together under reconnects, retries, and out-of-order delivery, and none of it is specific to the AI product you are shipping. + +## How a durable session changes the architecture + +A [session](/docs/ai-transport/concepts/sessions) is the whole state of a conversation, addressed by name and living independently of anything connected to it. No client or agent is privileged. A client that drops, a serverless agent that exits, a phone joining an hour later, and an orchestrator delegating to sub-agents all read and write the session the same way. + +![Diagram showing a laptop, a phone, and a second tab attached to one AI Transport session, with two agents on the other side. The session holds the durable conversation, token streaming, concurrent runs, cancellation, tool calling, multi-device sync, reconnection, history, branching, and agent presence](../../../images/content/diagrams/ait-overview.png) + +[Token streams](/docs/ai-transport/features/token-streaming) accumulate in the session as they arrive, so a reconnecting client receives the response as one assembled message, not a replay of every token. Within its retention window the session is also the conversation's [history](/docs/ai-transport/features/history), so a client opening the conversation later reads it from the same place. [Ably Push](/docs/ai-transport/features/push-notifications) reaches a user who closed the application before the answer was ready. + +A session outlives the agent process, which is what lets it pair with a workflow engine. [Durable execution](/docs/ai-transport/features/durable-execution) in Temporal or Vercel WDK retries the failed [step](/docs/ai-transport/concepts/runs#steps), and the retry publishes into the session the first attempt was already writing to. + +## Keep the stack you have + +Adoption changes where the response goes. How you produce it stays the same. In the agent, you pipe the stream you already build into a run on the session, using the Vercel entry point: + + +```javascript +// Agent-side, in place of `return result.toUIMessageStreamResponse()`: +import * as Ably from 'ably'; +import { Invocation } from '@ably/ai-transport'; +import { createAgentSession } from '@ably/ai-transport/vercel'; + +const ably = new Ably.Realtime({ key: process.env.ABLY_API_KEY }); + +export async function POST(req) { + const invocation = Invocation.fromJSON(await req.json()); + const session = createAgentSession({ client: ably, channelName: invocation.sessionName }); + await session.connect(); + + const run = session.createRun(invocation, { signal: req.signal }); + // Drain the triggering input first, or run.start() waits for it to arrive live. + while (run.view.hasOlder()) await run.view.loadOlder(); + + await run.start(); + const streamResult = await run.pipe(result.toUIMessageStream()); + await run.end({ reason: streamResult.reason }); + await session.end(); + + return Response.json({ runId: run.runId, invocationId: run.invocationId }); +} +``` + + +The route returns the run's identifiers instead of the stream, so a client reads the answer from the session rather than from this response. Wrap the body in `try`/`finally` so a thrown error still ends the run. Where an answer can outrun your function's budget, [durable execution](/docs/ai-transport/features/durable-execution) moves the run into a workflow engine and the route returns before it finishes. + +In the browser, a client attaches to that same session by name: + + +```javascript +// Client-side. +import * as Ably from 'ably'; +import { createClientSession } from '@ably/ai-transport/vercel'; + +const ably = new Ably.Realtime({ authUrl: '/auth' }); +const session = createClientSession({ client: ably, channelName: 'conversations:42' }); + +await session.connect(); +``` + -## Get started +Those two changes are the whole diff for an application that already streams. You do not touch prompts, tool definitions, or model calls, and on the UI side AI Transport implements Vercel AI SDK's [`ChatTransport` interface](/docs/ai-transport/frameworks/vercel-ai-sdk-ui), so swapping it into `useChat` leaves your components as they are. + +Two one-time setup steps come first. Browser clients connect with [token authentication](/docs/ai-transport/getting-started/authentication) rather than an API key. The namespace your conversations live on needs [one channel rule](/docs/ai-transport/getting-started/channel-rules) enabled, because AI Transport streams tokens by appending to a message. + +The SDK is JavaScript and TypeScript, with React hooks for the client. The [roadmap](/docs/ai-transport/roadmap) covers other languages. + +## What you take on + +Before you commit, these are the constraints and dependencies that come with it: + +| What you take on | What it means in practice | +| --- | --- | +| Retention bounds what the session remembers | The session holds the conversation for as long as [channel retention](/docs/storage-history/storage) covers it, which is a window you set per namespace. Past it you keep your own store, and [database hydration](/docs/ai-transport/features/database-hydration) joins the stored history to the live session with no gaps and no duplicates. An application that already writes conversations to a database carries on doing so. | +| Hydrated history is linear | Messages you have persisted yourself cannot be edited or regenerated yet, so branching applies to the part of the conversation the session still holds. | +| Session content is visible to subscribers | Every message reaches every subscriber whose token capability allows it, tool inputs and outputs included. Scope [capabilities](/docs/ai-transport/getting-started/authentication) per namespace to control who sees what. | +| A network hop and a platform dependency | Conversations travel over Ably rather than down your own response body, on a platform [designed for 99.999% global service availability](/docs/platform/architecture). [Append rollup](/docs/ai-transport/features/token-streaming#rollup) batches tokens rather than publishing one message each. | +| A pre-1.0 SDK | Minor releases can still change the API. The [wire protocol](/docs/ai-transport/internals/wire-protocol) is documented, and channel activity streams out through [webhooks and Kafka](/docs/platform/integrations/webhooks) for your own analytics and audit. | + +[Going to production](/docs/ai-transport/going-to-production) covers limits, retention, monitoring, auth hardening, and pricing in full. + +## How a session is built + +Every session runs on an [Ably channel](/docs/channels): a durable, ordered, append-only log that any client or agent attaches to by name. Messages outlive the connection, device, or process that published them, and they carry a total order. A client that drops reattaches and resumes without gaps or duplicates. + +Any client or agent publishes, which puts cancel and steering on the same path as tokens. The SDK turns that log into an AI conversation. Subscribers decode the same events the agent published, and a run and step lifecycle gives them structure: + +- A [codec](/docs/ai-transport/internals/codec-architecture) maps your framework's event types onto channel messages. Tokens stream by [appending to a single message](/docs/messages/updates-deletes#append), so a client arriving late reads one assembled response instead of every token event separately. +- The transport has no opinion about what publishes into it, so a [custom codec](/docs/ai-transport/internals/codec-architecture#write-a-custom-codec) carries a framework that has no adapter yet. +- A [conversation tree](/docs/ai-transport/concepts/conversation-tree) turns the log into branching history, and a [view](/docs/ai-transport/concepts/conversation-tree#views) selects one path through it for each client. That is how edit and regenerate work. +- [Client and agent sessions](/docs/ai-transport/concepts/sessions#connect) own attachment, the [run lifecycle](/docs/ai-transport/concepts/runs), and cancel routing. +- React hooks cover streaming, pagination, and branch navigation in the UI. + +## When a durable session is unnecessary + +A single-turn chatbot does not need a durable session. If a user asks one question, reads the answer, and never returns to that conversation, direct HTTP streaming is the simpler choice. + +## Where the guarantees come from + +Ordering, persistence, replication, and [regional failover](/docs/platform/architecture/fault-tolerance) are guarantees of the Ably platform. They hold for AI Transport on the same terms as for every other product on it. + +Ably is [SOC 2 Type II certified and HIPAA compliant](https://ably.com/security-and-compliance), and operates a bug bounty program. + +## Start building + +Pick the route that matches what you need next: {[ { - title: 'Getting started', - description: 'Build a working app with Vercel AI SDK or the Core SDK in a few minutes.', + title: 'Get started', + description: 'Build a working chat application with the Vercel AI SDK.', image: 'icon-tech-javascript', link: '/docs/ai-transport/getting-started/vercel-ai-sdk', }, - { - title: 'Frameworks', - description: 'See how AI Transport composes with the AI framework you already use.', - image: 'icon-tech-javascript', - link: '/docs/ai-transport/frameworks/vercel-ai-sdk-ui', - }, { title: 'Features', - description: 'Browse what AI Transport does once you have a session running.', - image: 'icon-tech-javascript', + description: 'What a session gives you once it is running, from streaming to branching.', + image: 'icon-product-ai-transport', link: '/docs/ai-transport/features/token-streaming', }, { - title: 'Why AI Transport', - description: 'Understand the production problems that direct HTTP streaming does not solve.', - image: 'icon-tech-javascript', - link: '/docs/ai-transport/why', + title: 'Concepts', + description: 'Sessions, runs, and the conversation tree behind the SDK.', + image: 'icon-gui-resources', + link: '/docs/ai-transport/concepts', + }, + { + title: 'Going to production', + description: 'Limits, retention, monitoring, auth hardening, and pricing.', + image: 'icon-product-platform', + link: '/docs/ai-transport/going-to-production', }, ]} - -## Read next - -- [Concepts](/docs/ai-transport/concepts): sessions, runs, and the conversation tree. -- [Going to production](/docs/ai-transport/going-to-production): the production checklist for shipping AI Transport. diff --git a/src/pages/docs/ai-transport/roadmap.mdx b/src/pages/docs/ai-transport/roadmap.mdx index 7ce58f6794..914f96169a 100644 --- a/src/pages/docs/ai-transport/roadmap.mdx +++ b/src/pages/docs/ai-transport/roadmap.mdx @@ -23,7 +23,7 @@ Available today. | Drop-in framework integration | A drop-in transport for the [Vercel AI SDK](/docs/ai-transport/getting-started/vercel-ai-sdk), plus the [Core SDK](/docs/ai-transport/getting-started/core-sdk) for everything else. | | Shared live state and presence | [Agent presence](/docs/ai-transport/features/agent-presence) and [shared session state](/docs/ai-transport/features/liveobjects) exposed directly through the SDK. | | [Durable execution](/docs/ai-transport/features/durable-execution) | Pair durable sessions with a workflow engine such as [Temporal](/docs/ai-transport/frameworks/temporal) or [Vercel WDK](/docs/ai-transport/frameworks/vercel-wdk), so a mid-flight process crash retries the failed step cleanly instead of stranding the turn. | -| Enterprise-ready platform | SOC 2 Type II and HIPAA, on [Ably's realtime platform](/docs/ai-transport/why#why-ably). | +| Enterprise-ready platform | SOC 2 Type II and HIPAA, on [Ably's realtime platform](/docs/ai-transport#why-ably). | ## Now diff --git a/src/pages/docs/ai-transport/why/http-streaming-and-ai.mdx b/src/pages/docs/ai-transport/why/http-streaming-and-ai.mdx index 04defde16d..7ce89822f2 100644 --- a/src/pages/docs/ai-transport/why/http-streaming-and-ai.mdx +++ b/src/pages/docs/ai-transport/why/http-streaming-and-ai.mdx @@ -105,12 +105,11 @@ All of this infrastructure is the transport layer being reinvented around the li A durable session replaces the ephemeral HTTP stream with a persistent, shared medium that any client or agent connects to. The properties the infrastructure above tries to assemble (persistence, ordering, multi-subscriber fan-out, bidirectional publishing, presence) are properties the session already has. -AI Transport implements durable [sessions](/docs/ai-transport/concepts/sessions) on Ably channels. Read [how a durable session solves each of these problems](/docs/ai-transport/why), or [get started with the Vercel AI SDK](/docs/ai-transport/getting-started/vercel-ai-sdk). +AI Transport implements durable [sessions](/docs/ai-transport/concepts/sessions) on Ably channels. Read [how a durable session solves each of these problems](/docs/ai-transport), or [get started with the Vercel AI SDK](/docs/ai-transport/getting-started/vercel-ai-sdk). ![Diagram showing how AI Transport concepts compose around the session, with connections attaching from outside and the conversation tree, runs, invocations, codecs, authentication, and infrastructure relating to it](../../../../images/content/diagrams/ait-concepts-overview.png) ## Read next - -The client and the agent are coupled by a single HTTP request and response for the lifetime of the interaction. With simple HTTP streaming: - -- Streams cannot resume: when the connection drops (network switch, page refresh, laptop lid closes), the response is gone. The agent keeps generating tokens; there is nowhere to deliver them. -- Sessions do not span devices: the stream exists only for the client that opened it. A second tab or a phone has no way in. -- No way back to the agent: HTTP streams are server-to-client. The only upstream signal a client has is to close the connection, which is indistinguishable from a disconnect. -- Agents cannot recover from their own restarts: a serverless agent that restarts mid-stream loses its outbound stream. The client sees a dead response. -- No stateful features: presence, shared mutable state, and multi-participant observation do not exist over a one-shot HTTP request. - -[HTTP streaming and AI](/docs/ai-transport/why/http-streaming-and-ai) works through each of these issues in detail. - -## Durable sessions change the model - -A durable session drops in between your agent framework and your users. It is persistent, shared, and stateful, and it handles reconnection, ordering, multi-device sync, presence, and failover so you don't need to build them. - -![Diagram showing clients and agents attached to a single Ably AI Transport session, with durable-conversation, streaming, multi-device, tool calling, reconnection, history, and presence as the features the session provides](../../../../images/content/diagrams/ait-overview.png) - -On the client, the minimal code to adopt it: - - -```javascript -import * as Ably from 'ably'; -import { createClientSession } from '@ably/ai-transport/vercel'; - -const ably = new Ably.Realtime({ authUrl: '/api/auth/token' }); -const session = createClientSession({ - client: ably, - channelName: 'chat-123', -}); - -await session.connect(); -``` - - -A durable session provides: - -- Durable streaming: [token streams](/docs/ai-transport/features/token-streaming) persist, accumulate, and resume. Reconnecting clients [receive assembled state](/docs/ai-transport/features/reconnection-and-recovery) rather than a replay of every token. -- Session continuity: the session follows the user rather than the connection, across devices, users, and tabs. Users switch devices; the [session continues with full state](/docs/ai-transport/features/multi-device). Agents [hand off to humans](/docs/ai-transport/features/human-in-the-loop) without losing context. -- Visibility and control: the session is bidirectional. [Cancel, interrupt, and steer](/docs/ai-transport/features/interruption-and-steering) mid-response. [Push to users](/docs/ai-transport/features/push-notifications) when they are offline. Know which [agents are online](/docs/ai-transport/features/agent-presence). The session also holds state of its own. - -How direct HTTP and a durable session compare: - -| Feature | Direct HTTP | Durable session | -| --- | --- | --- | -| Resume after disconnect | Build from scratch: buffer, order, sequence-number, and add a resume endpoint. | Automatic. Client reconnects and picks up where it left off. | -| Multi-device sync | Not possible without custom infrastructure. | Any device subscribes to the same session. | -| Cancel mid-stream | Close the connection (and lose the ability to resume). | Publish a cancel signal. Stream and session survive. | -| Steer or interrupt | Requires a separate back channel. | Signal the agent through the session. | -| Multi-agent visibility | Route all updates through a single HTTP orchestrator. | Each agent publishes directly to the session. | - -## How AI Transport implements this - -The sessions in AI Transport are built on [Ably channels](/docs/channels), allowing: - -- Any client or agent to connect by specifying a channel name. -- Messages outlive any single connection, device, or agent process. -- Events arrive at subscribers in publish order, even across disconnects. -- A client that drops reconnects and picks up where it left off. -- Any participant publishes. Cancel, steer, and interrupt all happen through the same session. -- Multiple participants subscribe; every participant sees every event. - -No participant is special. A client that drops and reconnects, a serverless agent that spins up for one run and terminates, a second client joining from another device, and an orchestrator delegating to sub-agents all interact with the same session in the same way. - -The SDK provides: - -- A [codec layer](/docs/ai-transport/internals/codec-architecture) that bridges your framework's event types and Ably's message primitives, with type-safe input and output messages and [message-append](/docs/messages/updates-deletes#append) accumulation. -- A [conversation tree](/docs/ai-transport/concepts/conversation-tree) that materialises session state into a branching structure with views for pagination and branch navigation. -- [Client and agent sessions](/docs/ai-transport/concepts/sessions#connect) that own connecting to the session, the [run lifecycle](/docs/ai-transport/concepts/runs), and cancel routing. -- React hooks for building UIs with streaming, pagination, and branch navigation. -- Adapters that drop into existing frameworks. AI Transport plugs directly into Vercel AI SDK, Temporal, and Vercel WDK. - -## When you don't need this - -Quick, single-turn chatbots do not need this. AI Transport is for experiences that are long-lived, agentic, and interactive, where sessions span conversations, devices, and time. If your users start and finish in one request, direct HTTP streaming is the simpler choice. -AI Transport is also not the right tool for conversations between people: for human-to-human chat such as group messaging or human-staffed support, use the [Chat SDK](/docs/chat). - -## Why Ably - -AI Transport runs on infrastructure that has been delivering realtime experiences at scale for over a decade: - -- Trillions of realtime transactions monthly. -- Billions of devices reached. -- Seven years of zero global downtime. -- Global edge network, multi-region, SOC 2 Type II, HIPAA-compliant. - -The hard problems of running stateful infrastructure (ordering, persistence, replication, presence, failover) are already solved. AI Transport inherits all of it. Ably also runs a bug bounty programme with independent security researchers, and [security and compliance](https://ably.com/security-and-compliance) lists the current certifications. - -## Read next +## What you gain over HTTP streaming -Most AI frameworks tie a turn to [one HTTP request](/docs/ai-transport/why/http-streaming-and-ai), so the conversation and the connection share a lifetime. That connection only ever joins one client to one agent. +Most AI frameworks send [one HTTP request per turn](/docs/ai-transport/why/http-streaming-and-ai) and stream the response back over it. The response lasts exactly as long as that request, and only the client that made the request can read it. -For a one-shot interaction that holds. The failures start where the two lifetimes come apart: - -| In production | Direct HTTP streaming | AI Transport session | +| Scenario | Direct HTTP streaming | AI Transport session | | --- | --- | --- | -| The connection drops mid-response | The response is gone. The model keeps generating tokens with nowhere to deliver them, and the user sees an error and starts over. | The agent keeps publishing into the session, and the client reconnects and [picks up where it stopped](/docs/ai-transport/features/reconnection-and-recovery). | -| The user presses stop | Closing the connection is the only signal available, and the agent cannot tell it apart from a network drop. | The client publishes a [cancel signal](/docs/ai-transport/features/cancellation), and the agent's current [run](/docs/ai-transport/concepts/runs) ends while the session stays open. | -| The agent process restarts mid-answer | The outbound stream ends with the process, and the client keeps a response that stopped halfway. | Your workflow engine retries the step, which publishes into [the same session](/docs/ai-transport/features/durable-execution), and the client keeps reading. | -| The user wants to know the agent is still working | A long pause looks the same as a stalled process. | The agent reports thinking, streaming, or idle through [presence](/docs/ai-transport/features/agent-presence) on the session. | -| The user opens the conversation on a phone | The stream belongs to the client that opened it, so a second device has no way in. | Every device [attaches to the same session](/docs/ai-transport/features/multi-device) and sees the conversation as it streams. | -| The user changes direction halfway through an answer | There is no upstream channel to send on. | The client [steers the active run](/docs/ai-transport/features/interruption-and-steering), cancels it and re-prompts, or [sends alongside it](/docs/ai-transport/features/double-texting). | -| The agent needs approval before it acts | The pending request lives on one connection, so a drop before anyone answers loses it. | The request [waits in the session](/docs/ai-transport/features/human-in-the-loop), and any user approves it minutes or hours later. | - -A single user on a train, moving from Wi-Fi to cellular and then to a laptop, runs into most of these on their own. The threshold is whether a response takes long enough to be interrupted, or a conversation is worth coming back to. +| The connection drops mid-response | The response is lost. The model keeps generating tokens that are discarded, and the user sees an error and starts again. | The agent keeps publishing into the session. The client reconnects and [resumes from the last token it received](/docs/ai-transport/features/reconnection-and-recovery). | +| The user opens the conversation on a phone | The stream belongs to the client that opened it, so a second device cannot read it. | Every device [attaches to the same session](/docs/ai-transport/features/multi-device) and sees the conversation as it streams. | +| The user reloads the page mid-answer | The request ends with the page, and the partial answer is lost. | The client reattaches and reads [history](/docs/ai-transport/features/history) and the in-flight response from the session, which holds the answer as one accumulating message. | +| The user presses stop | Closing the connection is the only signal available, and the agent cannot tell it apart from a network drop. | The client publishes a [cancel signal](/docs/ai-transport/features/cancellation). The agent's current [run](/docs/ai-transport/concepts/runs) ends and the session stays open. | +| The user sends again before the first answer finishes | A second request opens a second stream, and the client interleaves two responses or drops one. | Each turn is [its own run](/docs/ai-transport/features/concurrent-turns) on the same session, with its own stream and cancel handle. | +| The agent process restarts mid-answer | The outbound stream ends with the process, and the client is left displaying a truncated response. | Where the agent runs inside a workflow engine, the retried step publishes into [the same session](/docs/ai-transport/features/durable-execution). | -You meet these failures one at a time and solve them one at a time. What accumulates is a buffer for stream resume, a database for conversation state, a queue or a second WebSocket for upstream signals, a status endpoint with a client polling it, and a [reconciliation path](/docs/ai-transport/why/http-streaming-and-ai) to merge stored state with the live stream every time a client joins. +They show up at any scale. A single user on a poor connection loses a response mid-stream on their first long answer. -Each piece is tractable on its own. What does not get easier is keeping them correct together under reconnects, retries, and out-of-order delivery, and none of it is specific to the AI product you are shipping. +Solving these individually means building a buffer so a stream can resume, a database for conversation state, and a queue or a second WebSocket for the client to signal on. It also means a reconciliation step that merges stored state with the live stream whenever a client joins, which keeps costing after the rest is finished. None of that work is specific to the AI product you are shipping, and [HTTP streaming and AI](/docs/ai-transport/why/http-streaming-and-ai) goes through each limitation and the workaround code it requires. -## How a durable session changes the architecture +## What the session adds beyond delivery -A [session](/docs/ai-transport/concepts/sessions) is the whole state of a conversation, addressed by name and living independently of anything connected to it. No client or agent is privileged. A client that drops, a serverless agent that exits, a phone joining an hour later, and an orchestrator delegating to sub-agents all read and write the session the same way. +A [session](/docs/ai-transport/concepts/sessions) runs on an Ably channel, with a conversation model on top of it. The rest of Ably's channel features work on that channel, and the SDK folds its message log into a [conversation tree](/docs/ai-transport/concepts/conversation-tree), which is what makes branching and editing possible. -![Diagram showing a laptop, a phone, and a second tab attached to one AI Transport session, with two agents on the other side. The session holds the durable conversation, token streaming, concurrent runs, cancellation, tool calling, multi-device sync, reconnection, history, branching, and agent presence](../../../images/content/diagrams/ait-overview.png) +Each of the following is part of the session: -[Token streams](/docs/ai-transport/features/token-streaming) accumulate in the session as they arrive, so a reconnecting client receives the response as one assembled message, not a replay of every token. Within its retention window the session is also the conversation's [history](/docs/ai-transport/features/history), so a client opening the conversation later reads it from the same place. [Ably Push](/docs/ai-transport/features/push-notifications) reaches a user who closed the application before the answer was ready. +| What you get | How it works | +| --- | --- | +| [Branching, edit, and regenerate](/docs/ai-transport/features/branching) | Editing a message or regenerating a response forks the conversation instead of overwriting it. The tree keeps every branch, and a view selects one path through it per client. | +| [Agent and client presence](/docs/ai-transport/features/agent-presence) | An agent reports whether it is thinking, streaming, idle, or offline, and clients that enter presence appear alongside it. An agent can also watch presence and stop work when nobody is connected to read the answer. | +| [Shared live state](/docs/ai-transport/features/liveobjects) | The agent reacts to what the user is doing, such as the record they have selected, without polling or extra tool calls. Client and agent read and write the same state over the session. | +| [Interruption and steering](/docs/ai-transport/features/interruption-and-steering) | A client sends a follow-up into the run that is already streaming, or cancels it and re-prompts. A one-way stream offers no upstream path for either. | +| [Human-in-the-loop approval](/docs/ai-transport/features/human-in-the-loop) | A run suspends until a tool call is approved. The request waits in the session, so any user can answer it from any device, minutes or hours later. | +| [Chain of thought](/docs/ai-transport/features/chain-of-thought) | Reasoning streams as a separate stream within the same run, so a UI can render it beside the response. | -A session outlives the agent process, which is what lets it pair with a workflow engine. [Durable execution](/docs/ai-transport/features/durable-execution) in Temporal or Vercel WDK retries the failed [step](/docs/ai-transport/concepts/runs#steps), and the retry publishes into the session the first attempt was already writing to. +Built on direct HTTP streaming, each of these needs a second transport running alongside the stream. ## Keep the stack you have -Adoption changes where the response goes. How you produce it stays the same. In the agent, you pipe the stream you already build into a run on the session, using the Vercel entry point: +In the agent, pipe the stream you already build into a run on the session instead of returning it as the HTTP response body: ```javascript // Agent-side, in place of `return result.toUIMessageStreamResponse()`: import * as Ably from 'ably'; +import { streamText, convertToModelMessages } from 'ai'; +import { anthropic } from '@ai-sdk/anthropic'; import { Invocation } from '@ably/ai-transport'; import { createAgentSession } from '@ably/ai-transport/vercel'; @@ -69,22 +73,32 @@ export async function POST(req) { const invocation = Invocation.fromJSON(await req.json()); const session = createAgentSession({ client: ably, channelName: invocation.sessionName }); await session.connect(); - const run = session.createRun(invocation, { signal: req.signal }); - // Drain the triggering input first, or run.start() waits for it to arrive live. - while (run.view.hasOlder()) await run.view.loadOlder(); - await run.start(); - const streamResult = await run.pipe(result.toUIMessageStream()); - await run.end({ reason: streamResult.reason }); - await session.end(); + try { + // Page the view back through channel history to rebuild the prompt. This also + // supplies the triggering input that run.start() waits for. + while (run.view.hasOlder()) await run.view.loadOlder(); + const messages = run.view.getMessages().map(({ message }) => message); + + await run.start(); + const result = streamText({ + model: anthropic('claude-sonnet-4-20250514'), + messages: await convertToModelMessages(messages), + abortSignal: run.abortSignal, + }); + const streamResult = await run.pipe(result.toUIMessageStream()); + await run.end({ reason: streamResult.reason }); + } finally { + await session.end(); + } return Response.json({ runId: run.runId, invocationId: run.invocationId }); } ``` -The route returns the run's identifiers instead of the stream, so a client reads the answer from the session rather than from this response. Wrap the body in `try`/`finally` so a thrown error still ends the run. Where an answer can outrun your function's budget, [durable execution](/docs/ai-transport/features/durable-execution) moves the run into a workflow engine and the route returns before it finishes. +The route returns the run's identifiers. The client reads the answer from the session. Where an answer takes longer than your function's timeout, [durable execution](/docs/ai-transport/features/durable-execution) moves the run into a workflow engine and the route returns before it finishes. In the browser, a client attaches to that same session by name: @@ -101,52 +115,50 @@ await session.connect(); ``` -Those two changes are the whole diff for an application that already streams. You do not touch prompts, tool definitions, or model calls, and on the UI side AI Transport implements Vercel AI SDK's [`ChatTransport` interface](/docs/ai-transport/frameworks/vercel-ai-sdk-ui), so swapping it into `useChat` leaves your components as they are. +Prompts, tool definitions, model calls, and rendering stay as they are. AI Transport implements Vercel AI SDK's [`ChatTransport` interface](/docs/ai-transport/frameworks/vercel-ai-sdk-ui), so `useChat` takes it in place of its default transport. -Two one-time setup steps come first. Browser clients connect with [token authentication](/docs/ai-transport/getting-started/authentication) rather than an API key. The namespace your conversations live on needs [one channel rule](/docs/ai-transport/getting-started/channel-rules) enabled, because AI Transport streams tokens by appending to a message. +Browser clients connect with [token authentication](/docs/ai-transport/getting-started/authentication) rather than an API key, and the namespace your conversations live on needs [one channel rule](/docs/ai-transport/getting-started/channel-rules) enabled, because AI Transport streams tokens by appending to a message. Both are one-time. The SDK is JavaScript and TypeScript, with React hooks for the client. The [roadmap](/docs/ai-transport/roadmap) covers other languages. -## What you take on +## Limits and dependencies -Before you commit, these are the constraints and dependencies that come with it: +Running the conversation on Ably rather than inside your own process is a trade. These are the parts of it you take on: -| What you take on | What it means in practice | +| Limit or dependency | Detail | | --- | --- | -| Retention bounds what the session remembers | The session holds the conversation for as long as [channel retention](/docs/storage-history/storage) covers it, which is a window you set per namespace. Past it you keep your own store, and [database hydration](/docs/ai-transport/features/database-hydration) joins the stored history to the live session with no gaps and no duplicates. An application that already writes conversations to a database carries on doing so. | -| Hydrated history is linear | Messages you have persisted yourself cannot be edited or regenerated yet, so branching applies to the part of the conversation the session still holds. | -| Session content is visible to subscribers | Every message reaches every subscriber whose token capability allows it, tool inputs and outputs included. Scope [capabilities](/docs/ai-transport/getting-started/authentication) per namespace to control who sees what. | -| A network hop and a platform dependency | Conversations travel over Ably rather than down your own response body, on a platform [designed for 99.999% global service availability](/docs/platform/architecture). [Append rollup](/docs/ai-transport/features/token-streaming#rollup) batches tokens rather than publishing one message each. | -| Keeping the conversation yourself | Persist each completed run and the union of those runs [reconstructs the whole conversation](/docs/ai-transport/features/database-hydration) in your own store, with no gaps and no duplicates. The [wire protocol](/docs/ai-transport/internals/wire-protocol) is documented, and channel activity streams out through [webhooks and Kafka](/docs/platform/integrations/webhooks) for analytics and audit. | +| Retention limits what the session holds | The session holds the conversation for as long as [channel retention](/docs/storage-history/storage) covers it, a window you set per namespace. Beyond that your own database is the durable record: persist each completed run, and the union of those runs reconstructs the conversation. [Database hydration](/docs/ai-transport/features/database-hydration) joins the stored history to the live session with no gaps and no duplicates. | +| Hydrated history is linear | Messages you have persisted yourself cannot be edited or regenerated, so branching applies to the part of the conversation the session still holds. | +| Session content is visible to subscribers | Every message reaches every subscriber whose token capability allows it, tool inputs and outputs included. Scope [capabilities](/docs/ai-transport/getting-started/authentication) per namespace so a token only grants access to the sessions that client should read. | +| A network hop and a third-party dependency | Conversations go through Ably instead of your own infrastructure. [Append rollup](/docs/ai-transport/features/token-streaming#rollup) batches tokens into fewer published messages, so the added latency lands per batch rather than per token. | | A pre-1.0 SDK | Minor releases can still change the API. | -[Going to production](/docs/ai-transport/going-to-production) covers limits, retention, monitoring, auth hardening, and pricing in full. +[Going to production](/docs/ai-transport/going-to-production) covers limits, retention, monitoring, auth hardening, and pricing. ## How a session is built -Every session runs on an [Ably channel](/docs/channels): a durable, ordered, append-only log that any client or agent attaches to by name. Messages outlive the connection, device, or process that published them, and they carry a total order. A client that drops reattaches and resumes without gaps or duplicates. +Every session runs on an [Ably channel](/docs/channels): a durable, ordered, append-only log that any client or agent attaches to by name. Messages outlive the connection, device, or process that published them, and they have a total order. A client that drops reattaches and resumes without gaps or duplicates. -Any client or agent publishes, which puts cancel and steering on the same path as tokens. The SDK turns that log into an AI conversation. Subscribers decode the same events the agent published, and a run and step lifecycle gives them structure: +Any client or agent publishes, which puts cancel and steering on the same path as tokens. The SDK layers a conversation on top of that log: -- A [codec](/docs/ai-transport/internals/codec-architecture) maps your framework's event types onto channel messages. Tokens stream by [appending to a single message](/docs/messages/updates-deletes#append), so a client arriving late reads one assembled response instead of every token event separately. -- The transport has no opinion about what publishes into it, so a [custom codec](/docs/ai-transport/internals/codec-architecture#write-a-custom-codec) carries a framework that has no adapter yet. -- A [conversation tree](/docs/ai-transport/concepts/conversation-tree) turns the log into branching history, and a [view](/docs/ai-transport/concepts/conversation-tree#views) selects one path through it for each client. That is how edit and regenerate work. +- A [codec](/docs/ai-transport/internals/codec-architecture) maps your framework's event types onto channel messages. Tokens stream by [appending to a single message](/docs/messages/updates-deletes#append), so a client arriving late reads one assembled response. +- The transport does not constrain what publishes into it, so a [custom codec](/docs/ai-transport/internals/codec-architecture#write-a-custom-codec) adds support for a framework with no bundled adapter. - [Client and agent sessions](/docs/ai-transport/concepts/sessions#connect) own attachment, the [run lifecycle](/docs/ai-transport/concepts/runs), and cancel routing. -- React hooks cover streaming, pagination, and branch navigation in the UI. +- React hooks cover streaming, pagination, and [branch navigation](/docs/ai-transport/concepts/conversation-tree#views) in the UI. -## When a durable session is unnecessary +## When you do not need a durable session -A single-turn chatbot does not need a durable session. If a user asks one question, reads the answer, and never returns to that conversation, direct HTTP streaming is the simpler choice. +A single-turn chatbot does not need a durable session. If a user asks one question and never returns to the conversation, direct HTTP streaming is enough. -## Where the guarantees come from +## Platform guarantees -Ordering, persistence, replication, and [regional failover](/docs/platform/architecture/fault-tolerance) are guarantees of the Ably platform. They hold for AI Transport on the same terms as for every other product on it. +Ordering, persistence, replication, and [regional failover](/docs/platform/architecture/fault-tolerance) are guarantees of the Ably platform, which is [designed for 99.999% global service availability](/docs/platform/architecture). They apply to AI Transport in the same way as to every other Ably product. Ably is [SOC 2 Type II certified and HIPAA compliant](https://ably.com/security-and-compliance), and operates a bug bounty program. ## Start building -Pick the route that matches what you need next: +Choose where to go next: {[ @@ -158,7 +170,7 @@ Pick the route that matches what you need next: }, { title: 'Features', - description: 'What a session gives you once it is running, from streaming to branching.', + description: 'Streaming, branching, presence, cancellation, and the rest.', image: 'icon-product-ai-transport', link: '/docs/ai-transport/features/token-streaming', }, @@ -170,7 +182,7 @@ Pick the route that matches what you need next: }, { title: 'Going to production', - description: 'Limits, retention, monitoring, auth hardening, and pricing.', + description: 'The checklist to work through before you ship.', image: 'icon-product-platform', link: '/docs/ai-transport/going-to-production', }, From 9e1aa96bd48c18cabd7d86574da14c9b3c84e598 Mon Sep 17 00:00:00 2001 From: Andrew Bulat Date: Fri, 7 Aug 2026 14:04:49 +0100 Subject: [PATCH 4/4] Cut the rhetoric that reinstalled itself during the last pass A second read found that removing a device tended to put an equivalent one back two lines away, in a new slot. The reveal that opened the page came back as an antithesis couplet in the paragraph below it. The objection about traffic volume came back one synonym over. The scaffold that announced the setup steps moved from the front of its paragraph to the back as a three-word fragment. Two more scaffolds appeared in a section that previously went straight from heading to table. Cuts those, states the scale point as a single fact, breaks the first table out of a shape where four rows opened the same way and five closed the same way, and puts the cost before the mitigation in the row that discloses the extra round trip. Also restores the drain comment to the warning it was. Explaining the same thing in two balanced sentences reads as documentation; the original "do this, or X happens" reads like someone who hit it. Co-Authored-By: Claude Opus 5 (1M context) --- src/pages/docs/ai-transport/index.mdx | 40 +++++++++++++-------------- 1 file changed, 19 insertions(+), 21 deletions(-) diff --git a/src/pages/docs/ai-transport/index.mdx b/src/pages/docs/ai-transport/index.mdx index d011f81e65..0b757b4f5e 100644 --- a/src/pages/docs/ai-transport/index.mdx +++ b/src/pages/docs/ai-transport/index.mdx @@ -2,15 +2,13 @@ title: "AI Transport" meta_description: "AI Transport is a durable session layer for AI applications. The agent publishes into a session on Ably instead of the HTTP response body." meta_keywords: "AI Transport, durable sessions, AI infrastructure, resumable token streaming, HTTP streaming limitations, multi-device AI, conversation branching, Ably" -intro: "AI Transport is a durable session layer for AI applications. Your agent publishes its response into a session rather than an HTTP response body." +intro: "AI Transport is a durable session layer for AI applications. Your agent publishes its response into a durable session on Ably." redirect_from: - /docs/ai-transport/why - /docs/ai-transport/concepts/infrastructure --- -Instead of writing an AI response to the body of the HTTP request that asked for it, the agent publishes it into a durable session held on Ably. Clients attach to that session by name to read it. - -Writing the response to the request body is why a dropped connection loses the answer and a stop button cannot stop the generation behind it. A session outlives the connection that started it, and any client or agent can read from it and publish to it. +Instead of writing an AI response to the body of the HTTP request that asked for it, the agent publishes it into a durable session held on Ably. Clients attach to that session by name to read it. A session outlives the connection that started it, and any client or agent can read from it and publish to it. AI Transport is delivery infrastructure. You keep your model provider, whether that is OpenAI, Anthropic, Google, or a model you host, and your prompts, orchestration, and hosting stay where they are. AI Transport ships a drop-in transport for the [Vercel AI SDK](/docs/ai-transport/frameworks/vercel-ai-sdk-ui), and the [Core SDK](/docs/ai-transport/getting-started/core-sdk) composes with other frameworks, including [Vercel WDK](/docs/ai-transport/frameworks/vercel-wdk) and [Temporal](/docs/ai-transport/frameworks/temporal). @@ -26,20 +24,20 @@ Most AI frameworks send [one HTTP request per turn](/docs/ai-transport/why/http- | Scenario | Direct HTTP streaming | AI Transport session | | --- | --- | --- | -| The connection drops mid-response | The response is lost. The model keeps generating tokens that are discarded, and the user sees an error and starts again. | The agent keeps publishing into the session. The client reconnects and [resumes from the last token it received](/docs/ai-transport/features/reconnection-and-recovery). | -| The user opens the conversation on a phone | The stream belongs to the client that opened it, so a second device cannot read it. | Every device [attaches to the same session](/docs/ai-transport/features/multi-device) and sees the conversation as it streams. | -| The user reloads the page mid-answer | The request ends with the page, and the partial answer is lost. | The client reattaches and reads [history](/docs/ai-transport/features/history) and the in-flight response from the session, which holds the answer as one accumulating message. | +| The connection drops mid-response | The response is lost, the model keeps generating tokens that are discarded, and the user sees an error and starts again. | The agent keeps publishing into the session. The client reconnects and [resumes from the last token it received](/docs/ai-transport/features/reconnection-and-recovery). | +| A phone opens the same conversation | The stream belongs to the client that opened it, so a second device cannot read it. | Every device [attaches to the same session](/docs/ai-transport/features/multi-device) and sees the conversation as it streams. | +| The user reloads the page mid-answer | The reload discards the request and the partial answer with it. | The client reattaches and reads [history](/docs/ai-transport/features/history) and the in-flight response from the session, where the answer is a single message that grows as tokens arrive. | | The user presses stop | Closing the connection is the only signal available, and the agent cannot tell it apart from a network drop. | The client publishes a [cancel signal](/docs/ai-transport/features/cancellation). The agent's current [run](/docs/ai-transport/concepts/runs) ends and the session stays open. | -| The user sends again before the first answer finishes | A second request opens a second stream, and the client interleaves two responses or drops one. | Each turn is [its own run](/docs/ai-transport/features/concurrent-turns) on the same session, with its own stream and cancel handle. | +| A second message arrives before the first answer finishes | A second request opens a second stream, and the client interleaves two responses or drops one. | Each turn is [its own run](/docs/ai-transport/features/concurrent-turns) on the same session, with its own stream and cancel handle. | | The agent process restarts mid-answer | The outbound stream ends with the process, and the client is left displaying a truncated response. | Where the agent runs inside a workflow engine, the retried step publishes into [the same session](/docs/ai-transport/features/durable-execution). | -They show up at any scale. A single user on a poor connection loses a response mid-stream on their first long answer. +A dropped connection loses a response whether the application has one user or a million. -Solving these individually means building a buffer so a stream can resume, a database for conversation state, and a queue or a second WebSocket for the client to signal on. It also means a reconciliation step that merges stored state with the live stream whenever a client joins, which keeps costing after the rest is finished. None of that work is specific to the AI product you are shipping, and [HTTP streaming and AI](/docs/ai-transport/why/http-streaming-and-ai) goes through each limitation and the workaround code it requires. +Solving these individually means building a buffer so a stream can resume, a database for conversation state, and a queue or a second WebSocket for the client to signal on. It also means a reconciliation step that merges stored state with the live stream whenever a client joins. None of it is specific to the AI product, and [HTTP streaming and AI](/docs/ai-transport/why/http-streaming-and-ai) goes through each limitation and the workaround code it requires. ## What the session adds beyond delivery -A [session](/docs/ai-transport/concepts/sessions) runs on an Ably channel, with a conversation model on top of it. The rest of Ably's channel features work on that channel, and the SDK folds its message log into a [conversation tree](/docs/ai-transport/concepts/conversation-tree), which is what makes branching and editing possible. +A [session](/docs/ai-transport/concepts/sessions) runs on an Ably channel, with a conversation model on top of it. The rest of Ably's channel features work on that channel, and the SDK builds a [conversation tree](/docs/ai-transport/concepts/conversation-tree) from its message log, which is what makes branching and editing possible. Each of the following is part of the session: @@ -48,11 +46,11 @@ Each of the following is part of the session: | [Branching, edit, and regenerate](/docs/ai-transport/features/branching) | Editing a message or regenerating a response forks the conversation instead of overwriting it. The tree keeps every branch, and a view selects one path through it per client. | | [Agent and client presence](/docs/ai-transport/features/agent-presence) | An agent reports whether it is thinking, streaming, idle, or offline, and clients that enter presence appear alongside it. An agent can also watch presence and stop work when nobody is connected to read the answer. | | [Shared live state](/docs/ai-transport/features/liveobjects) | The agent reacts to what the user is doing, such as the record they have selected, without polling or extra tool calls. Client and agent read and write the same state over the session. | -| [Interruption and steering](/docs/ai-transport/features/interruption-and-steering) | A client sends a follow-up into the run that is already streaming, or cancels it and re-prompts. A one-way stream offers no upstream path for either. | +| [Interruption and steering](/docs/ai-transport/features/interruption-and-steering) | A client sends a follow-up into the run that is already streaming, or cancels it and re-prompts. | | [Human-in-the-loop approval](/docs/ai-transport/features/human-in-the-loop) | A run suspends until a tool call is approved. The request waits in the session, so any user can answer it from any device, minutes or hours later. | -| [Chain of thought](/docs/ai-transport/features/chain-of-thought) | Reasoning streams as a separate stream within the same run, so a UI can render it beside the response. | +| [Chain of thought](/docs/ai-transport/features/chain-of-thought) | Reasoning arrives as a separate stream within the same run, so a UI can render it beside the response. | -Built on direct HTTP streaming, each of these needs a second transport running alongside the stream. +Each of these needs a second transport alongside the stream if you build it on direct HTTP streaming. ## Keep the stack you have @@ -76,8 +74,8 @@ export async function POST(req) { const run = session.createRun(invocation, { signal: req.signal }); try { - // Page the view back through channel history to rebuild the prompt. This also - // supplies the triggering input that run.start() waits for. + // Page the view back through history to rebuild the prompt. Do this before + // run.start(), or it waits for the triggering input to arrive live. while (run.view.hasOlder()) await run.view.loadOlder(); const messages = run.view.getMessages().map(({ message }) => message); @@ -115,22 +113,22 @@ await session.connect(); ``` -Prompts, tool definitions, model calls, and rendering stay as they are. AI Transport implements Vercel AI SDK's [`ChatTransport` interface](/docs/ai-transport/frameworks/vercel-ai-sdk-ui), so `useChat` takes it in place of its default transport. +Prompts, tool definitions, model calls, and rendering are untouched. AI Transport implements Vercel AI SDK's [`ChatTransport` interface](/docs/ai-transport/frameworks/vercel-ai-sdk-ui), so `useChat` accepts it as its transport. -Browser clients connect with [token authentication](/docs/ai-transport/getting-started/authentication) rather than an API key, and the namespace your conversations live on needs [one channel rule](/docs/ai-transport/getting-started/channel-rules) enabled, because AI Transport streams tokens by appending to a message. Both are one-time. +Two one-time setup steps apply. Browser clients connect with [token authentication](/docs/ai-transport/getting-started/authentication) rather than an API key, and the namespace your conversations live on needs [one channel rule](/docs/ai-transport/getting-started/channel-rules) enabled, because AI Transport streams tokens by appending to a message. The SDK is JavaScript and TypeScript, with React hooks for the client. The [roadmap](/docs/ai-transport/roadmap) covers other languages. ## Limits and dependencies -Running the conversation on Ably rather than inside your own process is a trade. These are the parts of it you take on: +These apply to every application built on AI Transport: | Limit or dependency | Detail | | --- | --- | -| Retention limits what the session holds | The session holds the conversation for as long as [channel retention](/docs/storage-history/storage) covers it, a window you set per namespace. Beyond that your own database is the durable record: persist each completed run, and the union of those runs reconstructs the conversation. [Database hydration](/docs/ai-transport/features/database-hydration) joins the stored history to the live session with no gaps and no duplicates. | +| Retention limits what the session holds | The session holds the conversation for as long as [channel retention](/docs/storage-history/storage) covers it, a window you set per namespace. Beyond that window you keep your own database. Persist each completed run, and the union of those runs reconstructs the conversation. [Database hydration](/docs/ai-transport/features/database-hydration) joins the stored history to the live session with no gaps and no duplicates. | | Hydrated history is linear | Messages you have persisted yourself cannot be edited or regenerated, so branching applies to the part of the conversation the session still holds. | | Session content is visible to subscribers | Every message reaches every subscriber whose token capability allows it, tool inputs and outputs included. Scope [capabilities](/docs/ai-transport/getting-started/authentication) per namespace so a token only grants access to the sessions that client should read. | -| A network hop and a third-party dependency | Conversations go through Ably instead of your own infrastructure. [Append rollup](/docs/ai-transport/features/token-streaming#rollup) batches tokens into fewer published messages, so the added latency lands per batch rather than per token. | +| A network hop and a third-party dependency | Conversations go through Ably instead of your own infrastructure, so every publish makes a round trip. [Append rollup](/docs/ai-transport/features/token-streaming#rollup) batches tokens, so that round trip happens once per batch rather than once per token. | | A pre-1.0 SDK | Minor releases can still change the API. | [Going to production](/docs/ai-transport/going-to-production) covers limits, retention, monitoring, auth hardening, and pricing.