Skip to content

fix(streaming): keep gating dual-channel mixing on live channels after close_channel - #267

Closed
RaphaelFakhri wants to merge 1 commit into
AssemblyAI:masterfrom
RaphaelFakhri:fix/channel-mixer-live-gating
Closed

RaphaelFakhri wants to merge 1 commit into
AssemblyAI:masterfrom
RaphaelFakhri:fix/channel-mixer-live-gating

Conversation

@RaphaelFakhri

Copy link
Copy Markdown

Summary

With three or more channels, ChannelStreamer.close_channel() (and the async equivalent) stops the mixer from aligning the channels that are still open. After any channel is closed, the mixer sends everything the longest buffer holds and pads the shorter live channels with silence. Audio that arrives later for a padded channel is then mixed at the wrong position, so it is shifted out of time against the other channels in the stream sent to the server and against the VAD timeline used for channel attribution.

The mixer already intends closed channels to stop gating: drain() builds a list of live channels for that purpose. That branch is unreachable, because pad = force or bool(self._ended) switches to "pad to the longest buffer" as soon as any channel is closed. With two channels this is harmless, since one live channel is left. With three or more channels it isn't.

Changes

  • _ChannelMixer.drain() gates on the shortest buffer among the channels that are still open. Closed channels never gate, and are zero-padded when they are shorter.
  • On the final force flush, and once every channel is closed, all buffers are still padded up to the longest.
  • The drain() docstring describes this behavior.

Behavior with two channels, and for flush(), is unchanged.

Testing

Added test_mixer_close_channel_keeps_gating_on_live_channels. With channels mic (100 ms), system (100 ms) and room (50 ms) buffered, closing mic and draining must send only the 800 samples that both live channels cover. On master it sends 1600 samples, padding room with 50 ms of silence.

python -m pytest tests/unit/test_streaming_dual_channel.py
python -m pytest tests/unit
ruff format --check --config ./ruff.toml assemblyai/streaming/v3/extras.py tests/unit/test_streaming_dual_channel.py

The full unit suite passes.

@RaphaelFakhri

Copy link
Copy Markdown
Author

Closing to keep the number of open PRs here manageable. Happy to reopen if it is still useful.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant