Skip to content

fix(tts): load v3 references without TorchCodec and preserve failures - #238

Merged
Lucas1479 merged 3 commits into
mainfrom
codex/fix-235-tts-reference
Oct 19, 2026
Merged

Lucas1479 merged 3 commits into
mainfrom
codex/fix-235-tts-reference

Conversation

@Lucas1479

@Lucas1479 Lucas1479 commented Oct 17, 2026 •

Copy link
Copy Markdown
Member

What and why

Fixes #235.

The Windows ROCm profile uses TorchAudio 2.9, whose load() requires TorchCodec. A swallowed reference-cache failure let v3 synthesis continue without its semantic prompt; the semantic guard then misread the decoder's reference-free index as an empty result, and streaming reported synthetic silence as success.

  • Decode local v3 references with the existing SoundFile dependency and reuse only complete reference caches.
  • Slice generated tokens after the actual prompt prefix, preserving valid v1/v2 reference-free synthesis and the existing bounded semantic guard.
  • Propagate live preparation and streaming errors instead of yielding silent success.
  • Advance the first-sentence synthesis revision to v5 so existing silent v4 entries are ignored, including legacy asset-layout entries with or without metadata. Existing files remain on disk; audio regenerates on demand.
  • Keep optional emotion-reference warmup best-effort for ordinary exceptions, with an invalid status and warning; default synthesis remains available.
  • Integrate main at 559e9da0c63a899b40eef1727634a1177332c850. The ROCm workflow retains both fix(tts): load v3 references without TorchCodec and preserve failures #238 reference/semantic suites and fix(tts): serialize shared playback stop and initialization #242 playback recovery coverage in its triggers and test command.

Change class

  • Routine fix

Owning layers: local GPT-SoVITS reference preparation and semantic-result consumption, the existing derived-audio cache revision, and the optional reference-pack startup boundary.

Compatibility: no dependency, public API, configuration, or model-format change. First-sentence derived audio is regenerated once under the new synthesis revision. Live failures retain their original exception.

Evidence

Current head: 87e6c00faf23e4217995d86211d83345f385ff94.

Windows / Python 3.12.10; torch and torchaudio 2.6.0+cu124 using CPU tensors, SoundFile 0.13.1, librosa 0.11.0:

AMADEUS_E2E_NO_TTS=1 TTS_DEVICE=cpu WORK_WORKTREE_ISOLATION=0 HF_HUB_OFFLINE=1
python -B -X utf8 -m pytest -q -p no:cacheprovider
  tests/test_local_tts_semantic_guard.py tests/test_local_tts_reference_cache.py
  tests/test_t2s_semantic_history.py tests/test_tts_semantic_stability.py
  tests/test_tts_emotion_references.py tests/test_local_tts_punctuation.py
  tests/test_tts_contract.py tests/test_gpt_sovits_sidecar_backend.py
  tests/test_first_sentence_audio_cache.py tests/test_playback_device_recovery.py
  • 174 passed, 1 skipped; punctuation tests are gated off in model-less mode.
  • The seven new/extended cache and optional-warmup cases failed before the follow-up fixes and pass afterward. Coverage includes both cache identities, entries with/without metadata, regeneration and subsequent hits, and IndexError/ImportError during optional warmup followed by successful default synthesis.
  • Original reference/semantic regression evidence at unchanged 7b97e21c: 25 failed, 7 passed; the fixed source passes all 32 cases.
  • Reference tests use synthetic audio with real CPU decoding, resampling and mel extraction. Neural network outputs are stubbed; no checkpoints or character assets are required.
  • Ruff on the four follow-up Python files, git diff --check, and merged workflow YAML validation passed.

All 16 PR checks passed on this head, including Electron build, Windows packaging/startup smoke, the full model-less aggregate, Linux/macOS voice, cu124, Windows/Linux cu128, Linux T4 CPU, and macOS MPS.

Not yet verified: real Windows AMD ROCm 7.2.1 local v3 inference, GPU/driver behavior, or speech quality. #241's successful AMD Fish Audio playback report validates a different backend path; it does not validate this local v3 inference change. Hardware feedback remains welcome and is not required for merging this bounded code fix after current-head CI passes.

Final check

  • One coherent defect and its cache/startup consequences; no unrelated feature work.
  • Existing contracts reused; incomplete-cache fallbacks removed.
  • No secrets, local state, model weights, or voice assets included.
  • Vendored model code and third-party provenance unchanged.

@Lucas1479
Lucas1479 marked this pull request as ready for review October 17, 2026 20:54
Lucas1479 added a commit that referenced this pull request Oct 18, 2026
## What and why

Related to #241.

`StreamPlayer.stop()` cleared `is_playing` and drained queued writes
before acquiring the device lock. A concurrent `initialize()` could
reopen the output and publish `is_playing=True`; the unfinished stop
would then detach that new stream, leaving `is_playing=True,
stream=None`. Stopping while a native open was in progress could leave
the same inconsistent state. A subsequent write can raise `audio stream
is not initialized`.

Serialize the stop's flag change, queue drain, and stream
detachment/close under the existing reentrant device lock, matching
`initialize()`. No new state, retry, dependency, or platform branch is
added.

The defect is in the shared player and does not require AMD hardware to
reproduce. **This is a verified upstream race, but we have not
established that it is the exact cause of the reporter's installation
failure.** It is independent of the GPT-SoVITS reference-loading changes
in #238.

## Change class

- [x] Routine fix

Owning layer: shared TTS output-stream lifecycle (`tts/playback.py`).

User-visible effect: concurrent stop/open operations leave either a
stopped player or a usable replacement stream, rather than a playing
flag with no stream. Existing playback-epoch invalidation still rejects
stale audio.

Compatibility/migration: none; public APIs, device selection, backend
routing, and bounded output-open recovery are unchanged.

## Evidence

Windows / Python 3.12.10, CPU/model-less execution, synthetic PCM and
mocked audio devices; no physical playback:

- Deterministic concurrency regressions against unmodified upstream
`7b97e21c63b680d3ad53b1a14198dd52de882593`: **6 failed, 10 passed** in
`tests/test_playback_device_recovery.py`. Both stop/open orderings
reproduce the inconsistent state. The fixed implementation passes all 16
cases.
- Added cases cover reopening at the same or a different sample rate
during stop, stop during cold/replacement open, sync/queued playback, an
in-flight native write with epoch invalidation, and reentrant stop from
a window callback. Thread events and observed lock acquisition control
the interleavings; there are no scheduling sleeps.
- Focused regressions: **125 passed, no skips**, with
`AMADEUS_E2E_NO_TTS=1 TTS_DEVICE=cpu WORK_WORKTREE_ISOLATION=0`:
  ```text
python -B -X utf8 -m pytest -q -p no:cacheprovider
tests/test_playback_device_recovery.py tests/test_pyaudio_lifecycle.py
tests/test_playback_idle_device.py tests/test_playback_mouth_timing.py
tests/test_tts_handler.py tests/test_interrupt_flow.py
tests/test_tts_contract.py tests/test_shared_presentation_alignment.py
tests/test_turn_playback_complete.py
  ```
- `python -B -m ruff check tts/playback.py
tests/test_playback_device_recovery.py` and `git diff --check`: passed.
- The expanded existing suite is already exercised by Windows full CI /
T2, Linux voice, and macOS voice jobs; it is now also explicitly
included in the Windows ROCm candidate job.
- [Full Windows model-less
CI](https://github.com/Code-Amadeus/Amadeus/actions/runs/39101448220):
all three Python shards passed, **5,953 passed / 35 skipped** summed
from the runner summaries.
- [Windows ROCm candidate
CI](https://github.com/Code-Amadeus/Amadeus/actions/runs/39101448236):
**143 passed / 1 skipped** on Python 3.12.10 with the official
torch/torchaudio 2.9.1+rocm7.2.1 packages. This includes the expanded
playback suite, using mocked audio devices.
- All **16 PR checks passed** at
`8b8ad82a771e064a79fd7decccdcd2eb2a676435`, including Linux/macOS voice
checks, Windows/Linux cu128, macOS MPS, the cu124 single-environment
ladder, build, and desktop smoke checks. These do not constitute real
headset or GPU playback validation.

**Remaining validation:** no real AMD/NVIDIA GPU, reporter headset, or
Fish Audio end-to-end run was performed. Please test first-utterance
playback, subsequent turns, interruption followed by a new reply, and
streaming with the normal PyAudio path. The reporter's exact source
revision/package and any local audio-library edits still need
confirmation before attributing the whole report to this race.

## Final check

- [x] One coherent lifecycle defect; no unrelated cleanup.
- [x] Existing synchronization contract reused; no speculative fallback
or platform exception.
- [x] No private logs, credentials, model weights, voice assets, or
local paths included.
- [x] Third-party notices and provenance preserved.
@Lucas1479
Lucas1479 merged commit 7094366 into main Oct 19, 2026
16 checks passed
@Lucas1479
Lucas1479 deleted the codex/fix-235-tts-reference branch October 19, 2026 14:11
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Windows ROCm v3 TTS: missing TorchCodec causes false empty_candidate and first-sentence failure

1 participant