Skip to content

fix(cli): audit audio slots at their authored playback speed - #5300

Merged
jrusso1020 merged 1 commit into
heygen-com:mainfrom
user-github-me:fix/check-audio-playback-duration
Oct 24, 2026
Merged

jrusso1020 merged 1 commit into
heygen-com:mainfrom
user-github-me:fix/check-audio-playback-duration

Conversation

@user-github-me

Copy link
Copy Markdown
Contributor

What

Make the audio slot-fit audit in check and validate compare the slot with playable timeline seconds after trimming and playback speed, including a rate automation lane.

Why

A one-second tone at data-playback-rate=".5" fills a two-second slot in the actual render, but check --strict rejects it and recommends cutting the slot to one second. At rate 2, the same tone becomes silent halfway through a one-second slot, but the audit reports no shortfall. Comparing raw source length produces both errors.

Related work

No overlapping audio slot/rate validation PR found in current open PR files or concept searches. #4049 and #4087 concern video animation after source EOF; this audit continues to inspect audio only.

How

Serialize the audio's timing attributes from the browser and pass them to the existing shared natural-media-duration reader on the host. This follows constant and automated playback speed and the shared trim-offset precedence. Looping and unavailable-media handling remain in the existing fit audit.

Test plan

  • Six regressions fail on unchanged main with three passing controls. All 129 related tests pass across the new browser-function media-fit suite, validation, check browser integration, and check command tests.
  • Actual source and built CLI browser checks: half-speed fixture passes strict mode with zero warnings; double-speed fixture exits 1 with one clip_media_fit warning reporting 0.50 seconds. Static lint, runtime errors, layout, and contrast checks are clean for both fixtures.
  • Actual main renders with a local 48 kHz one-second sine tone: the slow render remains audible at 1.5–1.9 seconds (RMS −24.12 dBFS); the fast render is audible at 0.1–0.4 seconds (−24.05 dBFS) and silent at 0.65–0.9 seconds. FFprobe confirms 64×64, 10 fps, 20/10 frames and 2/1-second outputs.
  • Full workspace build, CLI rebuild/typecheck, root lint, formatting, test reachability, comment checks, and normal commit hooks pass. Fallow finds no issues in the changed files.
  • Comments follow CONTRIBUTING.md.

To reproduce, generate tone.wav with ffmpeg -f lavfi -i "sine=frequency=440:sample_rate=48000:duration=1" tone.wav. Put this in index.html and run hyperframes check <dir> --strict, then render with --fps 10. For the fast case, change both durations to 1 and the playback rate to 2.

<!doctype html>
<html>
  <head><style>html,body{margin:0}#root{width:64px;height:64px;background:#204a80}</style></head>
  <body>
    <div id="root" data-composition-id="main" data-width="64" data-height="64"
      data-duration="2" data-no-timeline="true">
      <audio id="voice" class="clip" src="tone.wav" data-start="0"
        data-duration="2" data-playback-rate=".5"></audio>
    </div>
  </body>
</html>

@jrusso1020 jrusso1020 left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Approve. The fix is correct and small, and it reuses the runtime's own duration math, so the audit can't drift from what the render does.

What I checked:

  • packages/cli/src/commands/validate.ts:215-219 now sends data-media-start, data-playback-start, data-playback-rate and data-automation out of the page, and :237 passes them to resolveNaturalMediaTimelineDuration from core. That's the same reader the runtime uses (packages/core/src/runtime/playbackRate.ts:42), so the precedence rules carry over: data-playback-start before the legacy data-media-start, and a rate lane before the constant rate. Before this change the audit used duration - data-media-start and ignored both rate and data-playback-start.
  • Confirmed the six "fails on main" cases by hand against the old formula: half speed, trimmed half speed, both rate-lane cases, canonical-trim-over-legacy and double speed all produce the wrong answer with duration - mediaStart. The three controls (normal speed, normal-speed shortfall, looping) pass either way. The test exercises the real browser function through window.eval, which is a good way to cover serialized page code.
  • The shim's getAttribute only answers the four attributes it forwards. I checked packages/parsers/src/mediaDuration.ts and those are the only ones the reader asks for. The fallback rate becomes 1 rather than the element's defaultPlaybackRate, which only matters if a script set that property, so it's fine.
  • The mediaSeconds === null branch can't be reached after the existing finite-duration guard. It's harmless.

Not run locally: I couldn't run the suite cheaply in my checkout. I'm relying on green CI plus reading the code.

Scope note for readers: the PR says it doesn't touch #4049 (video tweens after source EOF), and it doesn't. The audit is still audio-only. It doesn't overlap with #5528, which carries playback rate through export discovery. The two complement each other.

— Rames

@jrusso1020
jrusso1020 added this pull request to the merge queue Oct 24, 2026
Merged via the queue into heygen-com:main with commit 0e0f7f8 Oct 24, 2026
80 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants