Skip to content

fix(core): release decoded preview audio once no clip uses its source - #6178

Queued
miguel-heygen wants to merge 2 commits into
mainfrom
dslivers/audio-decode-release
Queued

miguel-heygen wants to merge 2 commits into
mainfrom
dslivers/audio-decode-release

Conversation

@miguel-heygen

Copy link
Copy Markdown
Collaborator

What

The preview audio transport now lets go of decoded audio once no clip in the film still uses that source.

Why

When an audio clip falls back to the decoded path (for example cross-origin audio without a crossorigin attribute), the transport caches the decoded AudioBuffer by source URL. Nothing ever removed an entry except destroy(), so in a long editing session every source the film ever used stayed decoded in memory: each scene swap or replaced clip added buffers that were never released.

Repro: a composition with decoded-route audio clips, play, pause, change the clips' sources (or swap scenes in place), play again. Before this change the cache holds the old and new sources; after it, only the sources still in the document.

Related work

None.

How

  • WebAudioTransport.releaseDecodedExcept(els) drops cached buffers whose source none of the given elements points at. The cache key is read through one helper (decodeKey) so the release and the lookup cannot disagree.
  • scheduleWebAudioForActiveClips calls it with the same element list it is about to schedule. That pass is the only caller of decodeAudioElement, so a source it no longer visits can never be asked for again until it reappears.

Why this rule and not a playhead window or an LRU cap: each pass schedules every future clip at once, and each scheduled AudioBufferSourceNode already holds its own buffer. While playing, the decoded set for clips in the film is the working set, so evicting by playhead distance or by a byte cap would only add re-fetch and re-decode on the next resume or seek back (audio entering late). Dropping a cache entry never touches a playing source, because the node keeps its own reference.

Test plan

  • Unit test added in init.test.ts ("decoded audio across edits"): 8 decoded clips, then an edit swaps 4 of them to new sources and playback resumes.
    • Before this change: the cache holds 12 sources (all 8 originals plus 4 new).
    • With this change: 8 (the 4 untouched plus the 4 new), and fetch is called 12 times in total, so the untouched clips are not re-decoded.
    • Mutation check: releasing every entry makes the fetch count 16 and the test fails.
  • init.test.ts, webAudioTransport.test.ts, webAudioTransportIdle.test.ts, init.swapScenes.test.ts: 510 passed. New test passed 3 runs in a row.
  • tsc --noEmit (core), oxfmt and oxlint on the changed files.
  • Manual playback on a real audio device was not run.
  • Comments follow CONTRIBUTING.md "Comments"

The preview transport cached every decoded audio buffer by source until the player was destroyed. Each scheduling pass now drops cached buffers whose source no clip in the document still uses (scene edits, swapped sources), and keeps the rest so playback never re-decodes a clip still in the film.
@github-actions

github-actions Bot commented Oct 24, 2026 •

Copy link
Copy Markdown
Contributor

Edit accuracy: accurate 271 (base branch 271), smooth 153 of those

The gate passes.
Smoothness is reported in the artifact, not gated. A case fails only if it fails 2 of 3 runs.

Quarantined, measured but not gated (0)

Unstable (1)

  • seqplainkeys-none-center-r0-root-z100: tracking 0.06, pressJump 0, drop 0, reload 0, render 0.03, renderKey -, undo false, teleport true / tracking 0.06, pressJump 0, drop 0, reload 0, render 0.03, renderKey -, undo true, teleport true / tracking 0.06, pressJump 0, drop 0, reload 0, render 0.03, renderKey -, undo true, teleport true

@heygengenesis heygengenesis Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Verdict: APPROVE · 0 blocker · 0 important · 1 nit · head bb4b8df9

  • nit: a decode still in flight from an older pass can land its buffer after releaseDecodedExcept ran (webAudioTransport.ts:306), so a just-removed source can sit cached until the next pass. Bounded to one pass, not worth gating.
Scope

Read the full diff plus decodeAudioElement, scheduleWebAudioForActiveClips and webAudioMediaIn. decodeAudioElement has exactly one caller and it iterates the same clips list passed to the release, so a dropped key can only come back if the source reappears. Hidden and skipped clips stay in the live set, so release is conservative. Both sides read the key through decodeKey, so they cannot disagree. Playing AudioBufferSourceNodes keep their own buffer reference. All Actions runs on this head are green. No prior reviews.

@miguel-heygen
miguel-heygen added this pull request to the merge queue Oct 24, 2026
Any commits made after this event will not be merged.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant