Skip to content

fix(studio-server): project history no longer copies video, audio or huge files - #6171

Queued
miguel-heygen wants to merge 5 commits into
mainfrom
fix/history-no-media-blobs
Queued

miguel-heygen wants to merge 5 commits into
mainfrom
fix/history-no-media-blobs

Conversation

@miguel-heygen

@miguel-heygen miguel-heygen commented Oct 24, 2026 •

Copy link
Copy Markdown
Collaborator

What

Project history copied every tracked file into ~/.cache/hyperframes/history/<id>/blobs, video and audio included. A project with a large camera file, a long recording and a render output being written grew its history by tens of gigabytes in minutes and filled the disk, so the render's own write failed. Each media file was stored at least once next to the original, and the current copies were never counted against the budget.

Now:

  • Video and audio files, and any file over 64 MiB, are hashed where they are (streamed, no copy) and recorded by path and hash. uncopied.ts is the one place that decides which files these are.

  • When history opens, copies an older version already made of such files are removed, but only when every reference to that content is a file of that kind. Text, small images and every edit history of HTML, CSS and JS stay as they are.

  • Bringing back an earlier version of such a file is not offered: undo, restore and checkout of a change that needs it fail up front, before anything is written, naming the file ("An earlier version of clip.mp4 is not kept"). Everything else in history behaves as before.

  • hyperframes clean does the same for every history in the cache, not only the ones Studio opens: it takes each history that no running process holds, applies the same rule, and --dry-run lists the bytes ("Video and audio copies in project history").

Not changed: the budget still counts stored bytes beyond what the current files need (a project larger than its budget keeps its history; a test pins that). With media gone, the current copies left are text and files under 64 MiB each, which cannot be shed anyway.

Evidence

  • A 50 MiB video present when history opens, and 25 MiB of video and audio written and rewritten while it is open: blob bytes stay under 1 KiB (they were the full file sizes).

  • A copy planted the way an older version left it is removed on open; the text blob next to it is kept and still readable.

  • Hashing in place costs no more than before: a file whose size and times have not moved is not read again (the existing stat cache), so reopening a project or saving a text edit does not rehash a large video. With a 1 GiB video, first adoption took 2.8 s instead of 5.8 s, reopening 18 ms (9 ms before), and a text-edit snapshot 29 ms (24 ms before); blob bytes 1 instead of 1,073,741,825.

  • hyperframes clean --dry-run on a planted history listed the 2.9 MB media copy and left it; hyperframes clean removed it and kept the text blob.

Tests

reclaimUncopied.test.ts (dry run lists and keeps; a real run removes the media copy and keeps text) and uncopied.test.ts (four cases): no copy for media present at open, none for media written or replaced later, a text edit is still copied while the media half of the same change cannot be undone, and reclaim on open. All four fail on main. The history suite (218 tests) and the clean command's tests is green three runs in a row. In projectHistory.test.ts the stand-in for "large media" is now a .bin file, since .mp4 is no longer copied.

@github-actions

github-actions Bot commented Oct 24, 2026 •

Copy link
Copy Markdown
Contributor

Edit accuracy: accurate 271 (base branch 271), smooth 142 of those

The gate passes.
Smoothness is reported in the artifact, not gated. A case fails only if it fails 2 of 3 runs.

Quarantined, measured but not gated (0)

Unstable (2)

  • seqheldtwice-tlhold-px-r0-root-z100: tracking 120.32, pressJump 312.3, drop 0, reload 0, render 0.02, renderKey -, undo true, teleport false / tracking 0.13, pressJump 0.11, drop 0.07, reload 0.07, render 0.02, renderKey -, undo true, teleport true / tracking 0.13, pressJump 0.11, drop 0.07, reload 0.07, render 0.02, renderKey -, undo true, teleport true
  • seqplainkeys-none-pct-r0-root-z100: tracking 0.06, pressJump 0, drop 0, reload 0, render 0.03, renderKey -, undo false, teleport true / tracking 0.06, pressJump 0, drop 0, reload 0, render 0.03, renderKey -, undo true, teleport true / tracking 0.06, pressJump 0, drop 0, reload 0, render 0.03, renderKey -, undo true, teleport true

@heygengenesis heygengenesis Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Verdict: REQUEST CHANGES · 1 blocker · 0 important · 0 nits · head e5f71bff

One blocker (inline): .ts and .mts are also TypeScript source extensions, and this change treats every file with those extensions as media.

Scope

Read the full diff, plus the projectHistory.ts paths it touches: firstOpen/adopt/recordAdopted, observe/storeIfPresent, hearWrites, restore and checkout.

  • Uncopied media never enters the store: reference() hashes without storing, so sizes and the budget never count those files, and prune can't touch them.
  • Restore and checkout refuse up front: both check blobs.has before anything is written.
  • Busy histories are skipped: clean takes ownership with timeout 0, so a history another process holds is left alone.

CI was still running when I reviewed.

".mts",
".m2ts",
".ts",
".mp3",

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

blocker: .ts (and .mts below) are TypeScript source extensions as well as MPEG-TS. History tracks every non-hidden project file, and helpers/projectSignature.ts lists .ts/.tsx as source text. So with this list, any TypeScript file in a project (a config, a helper script, a registry block's source) is treated as media, which causes two problems:

  1. Its edits are recorded but never copied, so undoing one fails with "An earlier version of foo.ts is not kept".
  2. reclaimUncopied on open, and hyperframes clean, delete every copy an older version already kept of those files. That loses saved history and can't be undone.

That contradicts "every edit history of HTML, CSS and JS stay as they are". Dropping .ts and .mts from the set is enough. A real transport stream large enough to matter is already caught by the 64 MiB rule, and .m2ts stays. A test with an edited .ts file that still undoes would pin it.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

✅ Fixed in 9baed70

@miguel-heygen

Copy link
Copy Markdown
Collaborator Author

Head follows this comment (rebased on main).

  • Blocker (.ts / .mts treated as media): fixed. Both are dropped from the media list; a transport stream large enough to matter is caught by the 64 MiB rule and .m2ts stays. New test: edited .ts and .mts files keep their copies, survive reclaim on open, and the edit still undoes. It fails on the previous head.

@miguel-heygen
miguel-heygen force-pushed the fix/history-no-media-blobs branch from e5f71bf to 9baed70 Compare October 24, 2026 02:30

@heygengenesis heygengenesis Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Verdict: REQUEST CHANGES — 1 resolved, 0 open, 2 new · e5f71bff → 9baed70a

The .ts/.mts fix is correct and pinned by a test. CI is red on this head, though, and both failures come from this PR:

  • blocker — Test (support): src/routes/history.test.ts:454 ("undo Studio's overwrite of media whose first copy was still running") still expects undoing a clip.mp4 overwrite to restore the original. This PR deliberately stops keeping that version, so the step now refuses and the assertion gets false. Switch that test to a .bin stand-in, as projectHistory.test.ts already does, and/or add a test asserting the refusal.
  • blocker — Fallow audit: reclaimUncopied.ts:20, where reclaimUncopiedBlobs has cognitive complexity 39 against a threshold of 15. Moving the per-history body (log read, size map, reclaim) into its own function should bring it under.

@heygengenesis heygengenesis Bot left a comment •

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Verdict: REQUEST CHANGES — 2 resolved, 0 open, 1 new · 9baed70a → 5ea5730c

Both earlier reds are fixed: Fallow audit and Test (support) pass on this head. The route test now uses a .bin stand-in, and the split of reclaimUncopiedBlobs behaves exactly as before (same hashes, sizes and reclaim rule). One new lint error (inline) fails CI's Lint job and the Preflight (lint + format) step in Windows render verification and preview-regression.

const hashes = new Set<string>(log.baseline.values());
for (const entry of log.entries)
for (const file of entry.files)
for (const hash of [file.before, file.after]) hash && hashes.add(hash);

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

blocker: oxlint no-unused-expressions ("Expected expression to be used") fails Lint on this line, because hash && hashes.add(hash) is used as a statement.

Suggested change
for (const hash of [file.before, file.after]) hash && hashes.add(hash);
for (const hash of [file.before, file.after]) if (hash) hashes.add(hash);

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

✅ Fixed in 51f914c

@heygengenesis heygengenesis Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Verdict: APPROVE — 1 resolved, 0 open, 0 new · 5ea5730c → 51f914c9

The lint fix is equivalent (it collects the same hashes), and every workflow is green on this head.

@miguel-heygen
miguel-heygen added this pull request to the merge queue Oct 24, 2026
Any commits made after this event will not be merged.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant