Skip to content

FIX: Move streaming PCM file work off the event loop in PromptNormalizer - #3778

Open
Lanre Shittu (Shizoqua) wants to merge 1 commit into
microsoft:mainfrom
Shizoqua:fix/streaming-pcm-off-event-loop
Open

Lanre Shittu (Shizoqua) wants to merge 1 commit into
microsoft:mainfrom
Shizoqua:fix/streaming-pcm-off-event-loop

Conversation

@Shizoqua

Copy link
Copy Markdown
Contributor

Description

Fixes #3720.

PromptNormalizer.convert_audio_async wrote the incoming PCM to a temp WAV, read the converted WAV back and deleted the temp file with synchronous tempfile, wave and pathlib calls on the event loop. In a realtime streaming session this runs for every audio turn that has request converters, so other async work stalled while it ran.

The write, the read and the cleanup now run through asyncio.to_thread. The cleanup stays in the finally block, and the worker thread completes the removal even if the task is cancelled during that await.

One case needed extra handling. A worker thread cannot be stopped, so if the caller is cancelled while the temp WAV is still being written, the thread goes on to create the file but the caller never gets its path. _write_pcm_to_temp_wav_async shields the write and, when the caller is cancelled, removes the file once the write finishes, so no temp file is left behind. The converted PCM, the format mismatch check and the no-converter fast path are unchanged.

Tests and Documentation

Added tests in tests/unit/prompt_normalizer/test_prompt_normalizer.py:

  • The temp WAV write, the read back and the cleanup each run on a thread other than the event loop thread.
  • Cancelling during conversion still removes the temp WAV.
  • Cancelling while the temp WAV is still being written removes the file once the write finishes, and conversion never starts.

No docs or notebooks changed, so JupyText was not needed.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Streaming PCM conversion performs synchronous disk I/O on the event loop

1 participant