Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 4 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,9 @@
# Changelog

## [4.41.5]

- Add `speakerLabelsRevisionIntervalMs` streaming param (sent as `speaker_labels_revision_interval_ms`) — the cadence, in ms of audio time, at which the server emits mid-stream `speakerRevision` events when `speakerLabels` is enabled. Unset or `0` sends only the end-of-stream revision; non-zero values are clamped server-side to 120 000–300 000 ms. The first mid-stream revision needs ~120 s of streamed speech, and one is sent only when some earlier turn's label changed

## [4.41.4]

- Fix `RealtimeTranscriber.connect()` promise hanging indefinitely when a socket error or unexpected close occurs before `SessionBegins`. The promise now rejects, and the socket reference is discarded so that connections can be safely retried.
Expand Down
6 changes: 5 additions & 1 deletion CLAUDE.md
Original file line number Diff line number Diff line change
Expand Up @@ -103,7 +103,11 @@ when `speakerLabels` is enabled). Sent once per offline-recluster resolve; each
message batches every earlier Turn whose speaker label changed (unchanged turns
are omitted). For each `revisions[i]`, match `turn_order` against the original
Turn and replace its per-word `speaker` (and the turn-level `speaker_label`)
with the revision's values — text and word timestamps are unchanged:
with the revision's values — text and word timestamps are unchanged.
By default only the end-of-stream revision is sent; set
`speakerLabelsRevisionIntervalMs` (ms of audio time, clamped server-side to
120 000–300 000) to also get them mid-stream — the first can only arrive after
~120 s of speech, and one is sent only when a label actually changed:

```typescript
transcriber.on("speakerRevision", (event) => {
Expand Down
2 changes: 1 addition & 1 deletion package.json
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
{
"name": "assemblyai",
"version": "4.41.4",
"version": "4.41.5",
"description": "The AssemblyAI JavaScript SDK provides an easy-to-use interface for interacting with the AssemblyAI API, which supports async and real-time transcription.",
"engines": {
"node": ">=18"
Expand Down
7 changes: 7 additions & 0 deletions src/services/streaming/service.ts
Original file line number Diff line number Diff line change
Expand Up @@ -406,6 +406,13 @@ export class StreamingTranscriber {
searchParams.set("max_speakers", this.params.maxSpeakers.toString());
}

if (this.params.speakerLabelsRevisionIntervalMs !== undefined) {
searchParams.set(
"speaker_labels_revision_interval_ms",
this.params.speakerLabelsRevisionIntervalMs.toString(),
);
}

if (this.params.voiceFocus) {
searchParams.set("voice_focus", this.params.voiceFocus);
}
Expand Down
13 changes: 13 additions & 0 deletions src/types/streaming/index.ts
Original file line number Diff line number Diff line change
Expand Up @@ -139,6 +139,19 @@ export type StreamingTranscriberParams = {
inactivityTimeout?: number;
speakerLabels?: boolean;
maxSpeakers?: number;
/**
* Cadence, in milliseconds of audio time, at which the server emits
* mid-stream `speakerRevision` events with corrected speaker labels for
* earlier turns (plus a final one at end of stream). Unset or `0` sends no
* mid-stream revisions — only the end-of-stream one. Non-zero values are
* clamped server-side to 120 000–300 000 ms; the server rejects values
* outside 0–86 400 000 (24 h). Only used when `speakerLabels` is enabled.
*
* The first mid-stream revision can only arrive after ~120 s of speech has
* been streamed, and a revision is sent only when some earlier turn's label
* actually changed — so expect "at most one per interval", not exactly one.
*/
speakerLabelsRevisionIntervalMs?: number;
voiceFocus?: VoiceFocusModel;
voiceFocusThreshold?: number;
continuousPartials?: boolean;
Expand Down
38 changes: 38 additions & 0 deletions tests/unit/streaming.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -127,6 +127,44 @@ describe("streaming", () => {
await connect(rt, server);
});

it("should include speaker_labels_revision_interval_ms in connection URL", async () => {
await cleanup();
WS.clean();

const wsUrl = `${websocketBaseUrl}?token=123&sample_rate=16000&speech_model=universal-streaming-english&speaker_labels=true&speaker_labels_revision_interval_ms=120000`;
server = new WS(wsUrl);
rt = new StreamingTranscriber({
websocketBaseUrl,
token: "123",
sampleRate: 16_000,
speechModel: "universal-streaming-english",
speakerLabels: true,
speakerLabelsRevisionIntervalMs: 120_000,
});
onOpen = jest.fn();
rt.on("open", onOpen);
await connect(rt, server);
});

it("should send speaker_labels_revision_interval_ms=0 when explicitly set to 0", async () => {
await cleanup();
WS.clean();

const wsUrl = `${websocketBaseUrl}?token=123&sample_rate=16000&speech_model=universal-streaming-english&speaker_labels=true&speaker_labels_revision_interval_ms=0`;
server = new WS(wsUrl);
rt = new StreamingTranscriber({
websocketBaseUrl,
token: "123",
sampleRate: 16_000,
speechModel: "universal-streaming-english",
speakerLabels: true,
speakerLabelsRevisionIntervalMs: 0,
});
onOpen = jest.fn();
rt.on("open", onOpen);
await connect(rt, server);
});

it("should normalize deprecated minEndOfTurnSilenceWhenConfident to min_turn_silence in connection URL", async () => {
await cleanup();
WS.clean();
Expand Down
Loading