Data tracks

[pblazej]

Adds data tracks to the Swift SDK, implemented on top of the Rust data-track engine via UniFFI (livekit-uniffi-xcframework 0.1.8).

API

// Publish + push
let track = try await room.localParticipant.publishDataTrack(name: "telemetry")
try track.tryPush(frame: DataTrackFrame(payload: data))          // non-blocking
try await track.send(contentsOf: frames)                          // AsyncSequence, drop-on-full by default

// Subscribe + receive
let stream = try await remoteTrack.subscribe(bufferSize: 16)
for await frame in stream.values { ... }

Publish/unpublish events fire on RoomDelegate and ParticipantDelegate for both local and remote tracks, mirroring the media-track conventions. Everything is Objective-C compatible (DataTrackObjCTests covers the surface).

Design decisions

Rust owns the protocol, Swift owns the session. The UniFFI managers (LocalDataTrackManager/RemoteDataTrackManager) implement the publish/subscribe state machine, packetization, and E2EE. The SDK contributes what only it has: the signal connection, the WebRTC data channels, participant identity, and delegate fan-out. Concretely, a session-scoped DataTracks coordinator (one per Room, Room+DataTrack.swift) feeds SFU signal messages into the managers, forwards their outbound requests through SignalClient (serialized, so publish/unpublish ordering is preserved), and pumps packets between the managers and the DTP data channels.

Thin public wrappers over the bindings. The generated bindings are internal imported and every public type (LocalDataTrack, RemoteDataTrack, DataTrackFrame, DataTrackStream, DataTrackInfo, error enums) is a small SDK-owned wrapper (~740 lines total). Tradeoff considered: exposing the generated types directly would save the layer, but pins the public API to regenerated code we don't control, and UniFFI emits Swift-only structs — NSObject wrappers are what makes the API reach Objective-C. The wrappers also carry the SDK idioms (lowerCamel error enums, Track.Sid-style identifiers, delegate naming).

Retain-cycle handling at the FFI boundary. UniFFI callback interfaces hold their Swift delegate strongly (no weak references over FFI). A dedicated ManagerDelegate object references the Room and coordinator weakly, making it the designated weak link so manager → delegate → room → manager cycles can't leak (Room+DataTrack.swift documents the shape).

Reconnect semantics. Quick reconnect preserves publications via SyncState.publishDataTracks (the managers replay their publish responses); full reconnect republishes and re-asserts subscriptions. The DataTracks subsystem survives reconnects (channels are swapped in) and is torn down only on real disconnect, which also unpublishes remote tracks and notifies delegates. When a remote publisher full-reconnects, the Rust manager treats the republication as a SID reassignment: the subscriber's existing RemoteDataTrack survives with its SID rewritten in place and active subscriptions transparently re-requested — no unpublish/republish events fire. RemoteParticipant.dataTracks derives its keys from live track info for this reason (stored keys would go stale on reassignment).

Backpressure. tryPush is non-blocking and throws queueFull; send(contentsOf:) defaults to dropping frames when the queue is full (DTP is lossy by design — unbounded buffering would trade a dropped frame for unbounded latency). The publisher channel additionally drops frames beyond a 2 MiB buffered threshold, matching the lossy data-channel behavior. Subscribe-side buffer is caller-configurable (subscribe(bufferSize:)).

E2EE. When the room has E2EE configured, the same key provider drives data-track frame encryption via UniFFI EncryptionProvider/DecryptionProvider shims; DataTrackInfo.usesE2ee reflects it.

Size impact

Measured cost of the 0.0.6 → 0.1.8 UniFFI package bump plus this integration: ~+190 KB compiled Swift bindings (dead-strip floor) + ~190 KB Rust dylib (ios-arm64) ≈ +0.4 MB expected on the App Size job (~2% of baseline, within tolerance). Isolating data tracks behind a Swift package trait was evaluated and rejected for now: the core SDK already depends on LiveKitUniFFI (tokens, log forwarding), both UniFFI components live in one dylib, and the bindings are one module — a trait could only gate the ~740-line wrapper layer that dead-strip already removes. Revisit if the Rust side ships a feature-split artifact.

Not in this PR

• Schema/frame-encoding metadata (DataTrackOptions.schema/frameEncoding, added upstream in 0.1.8) is not yet surfaced in the public API.
• Frame timestamps are millisecond UInt64s carried verbatim (DataTrackFrame.now(payload:) stamps the current time); no derived latency helpers yet.

Testing

End-to-end DataTrackTests (publish/subscribe/roundtrip, delegate events on all four surfaces, buffer size, pre-join publications, publication retention, publisher full-reconnect SID reassignment, quick-reconnect sync state, publisher-disconnect unpublish) against livekit-server --dev, plus DataTrackObjCTests for the Objective-C surface.

[github-actions]

⚠️ This PR does not contain any files in the .changes directory.

[1egoman]

Generally makes sense to me from a high level / as a non swift expert, I think it would be good to get a swift expert to go through this as well

[1egoman]

Ah this is pretty clever, so you give the user access to control whether non queueable messages drop or throw.

Two thoughts:

1. Is this always something you would want to expose? Or is this transitional until we come to a decision on the max data track frame size / the performance bounds which data tracks are expected to support?
2. Should the default be throw or drop? I would argue throw maybe makes more sense, or at a minimum, drop should log when it drops so a user isn't surprised when things break unexpectedly if they send a large data track frame.

cc @lukasIO for when you are back, this might be worth doing in web as well 🤔

[pblazej]

My initial idea was more of a "pipe" between (buffered) AsyncSequence and data track, the backpressure (default) is a follow-up, no real opinion/example here.

[1egoman]

thought: Should this be async? What we did in the web (which worked quite well) was to immediately return the DataTrackStream equivalent object from subscribe() and lazily initialize the underlying sfu subscription on the first stream iteration .next() type call. The benefit to doing it this way is it centralized all error handling in one path (both stream read errors and stream initialization errors). Another side benefit was it abstracted away the actual "subscribe" call so it could be run initially or later after a full reconnect all transparently to a user.

[pblazej]

So now (without reading your code), these errors would propagate just from the stream? try await track.subscribeWithOptions?

[1egoman]

Yes, they would propagate from the stream. For example, here's how it works on the web currently (unrolling the for await to make it a little easier to comprehend):

const stream = track.subscribe();

while (true) {
  const { value, done } = await stream.next();
  if (done) {
    break;
  }
  console.log('Received frame:', value.payload);
}

The important thing is that this doesn't do await track.subscribe(), the only await happens at the stream.next() level.

[1egoman]

Also, to be clear - not trying to argue for / against doing this in swift, only that it turned out to work quite well in js and consistency might be nice.

[pblazej]

No hard feelings here, it's just not very natural to make subscribe non-throwing and move it to the stream @lukasIO any bias here?

[ladvoc]

I agree, I think this question is somewhat based on language conventions, and as a Swift dev it would feel weird to me if the error was deferred. Same with Rust which has the subscribe method return a result.

[pblazej]

Yeah I'm more on the Rust side here.

[1egoman]

This is something else I think could be good to have on the web 🤔

[pblazej]

Yes if you "produce"/yield the frames inside. Take these async helpers with a grain of salt tho as they've got no equivalent for other tracks.

[1egoman]

I am not sure I fully understand the reason this has to be done - I would assume that a user would need to keep a LocalDataTrack reference alive so they can call tryPush / etc on it? So if it drops, then unpublishing should always be the right thing to do?

I think I might need to see a small example to fully wrap my head around it, if that isn't too hard to put together.

[pblazej]

@1egoman @ladvoc a variant of buffering, similar to our existing DataChannel infrastructure 91e9203

Semantic similarities/diffs between JS/Rust indicated in tests.

I also see some potential extraction to FlowControlledDataChannel 👍

BTW this logic could be moved to Rust as well 🙃

[devin-ai-integration]

Devin Review found 1 new potential issue.

View 15 additional findings in Devin Review.

[devin-ai-integration]

🟡 Publishing a data track right after a connection drop can silently fail instead of waiting for the reconnect

The readiness gate that publishing waits on stays latched open after the connection is torn down and is only re-armed when a fresh channel arrives (_publisherChannelOpen.reset() at Sources/LiveKit/Core/Room+DataTrack.swift:217), so a publish started during a reconnect proceeds as if the link were live.

Impact: A publishDataTrack(name:) call made during the reconnect window fails with a timeout (and any frames pushed in that window are discarded) instead of waiting for the connection to come back.

Why the gate is stale during the full-reconnect window

DataTracks is session-scoped and deliberately survives a full reconnect (Sources/LiveKit/Core/Room+DataTrack.swift:323-328). _publisherChannelOpen is an AsyncCompleter whose result is sticky once resumed (Sources/LiveKit/Support/Async/AsyncCompleter.swift, wait() returns the cached _result immediately). It is reset only inside setPublisherChannel, which is called from the .join branch of configureTransports (Sources/LiveKit/Core/Room+Engine.swift:191-193).

During a full reconnect the order is: cleanUp(isFullReconnect: true) → cleanUpRTC closes the transports and sets _state.transport = nil, but _publisherChannelOpen is left resolved and DataTracks is kept. Until fullConnectSequence reaches configureTransports, a concurrent publish(name:):

1. calls room?.ensurePublisherConnected(), which returns immediately because guard case .subscriberPrimary = _state.transport fails when transport is nil (Sources/LiveKit/Core/Room+Engine.swift:74-77);
2. passes _publisherChannelOpen.wait() instantly on the stale cached result;
3. calls local.publishTrack(...), whose outbound signal request is dropped by ManagerDelegate (try? await room.signalClient.sendRequest(...) at Sources/LiveKit/Core/Room+DataTrack.swift:262; _sendRequest throws while connectionState == .disconnected).

The same latch also never re-arms if the publisher data channel merely transitions to .closed — dataChannelDidChangeState only handles the .open transition (Sources/LiveKit/Core/Room+DataTrack.swift:294-302).

Prompt for agents

In Sources/LiveKit/Core/Room+DataTrack.swift, `_publisherChannelOpen` is a sticky AsyncCompleter that gates `publish(name:)`. It is only re-armed in `setPublisherChannel`, which runs when a brand-new publisher data channel is created in the `.join` branch of `configureTransports`. Consequently, between `Room.cleanUp(isFullReconnect: true)` (which closes transports but keeps the DataTracks subsystem) and the new channel being created, the gate still reports "open", and `ensurePublisherConnected()` also returns early because `_state.transport` is nil. A `publishDataTrack` issued in that window proceeds against a dead transport and ultimately times out instead of waiting for the reconnect. The same latch is never cleared when the publisher data channel transitions to `.closed`/`.closing` on its own, since `dataChannelDidChangeState` only handles the `.open` case.

Consider re-arming (resetting) the completer when the publisher channel leaves the `.open` state and/or when the room tears down its transports for a full reconnect, so publishing blocks until the new channel is actually open.

---

Was this helpful? React with 👍 or 👎 to provide feedback.

[ladvoc]

note: This will also resolve when the track has been unpublished due to the manager having been shutdown. If the manager is shutdown before the room has been destroyed, this will resolve, match some for self and room, and fire the delegate events.

[pblazej]

I think it's way cleaner to just revert this ugly task-leak and keep the raii/retain approach (without edge cases).