Repository navigation
fix(studio-server): project history no longer copies video, audio or huge files - #6171
miguel-heygen wants to merge 5 commits into
Conversation
Edit accuracy: accurate 271 (base branch 271), smooth 142 of thoseThe gate passes. Quarantined, measured but not gated (0) Unstable (2)
|
There was a problem hiding this comment.
Verdict: REQUEST CHANGES · 1 blocker · 0 important · 0 nits · head e5f71bff
One blocker (inline): .ts and .mts are also TypeScript source extensions, and this change treats every file with those extensions as media.
Scope
Read the full diff, plus the projectHistory.ts paths it touches: firstOpen/adopt/recordAdopted, observe/storeIfPresent, hearWrites, restore and checkout.
- Uncopied media never enters the store:
reference()hashes without storing, sosizesand the budget never count those files, andprunecan't touch them. - Restore and checkout refuse up front: both check
blobs.hasbefore anything is written. - Busy histories are skipped:
cleantakes ownership with timeout 0, so a history another process holds is left alone.
CI was still running when I reviewed.
| ".mts", | ||
| ".m2ts", | ||
| ".ts", | ||
| ".mp3", |
There was a problem hiding this comment.
blocker: .ts (and .mts below) are TypeScript source extensions as well as MPEG-TS. History tracks every non-hidden project file, and helpers/projectSignature.ts lists .ts/.tsx as source text. So with this list, any TypeScript file in a project (a config, a helper script, a registry block's source) is treated as media, which causes two problems:
- Its edits are recorded but never copied, so undoing one fails with "An earlier version of foo.ts is not kept".
reclaimUncopiedon open, andhyperframes clean, delete every copy an older version already kept of those files. That loses saved history and can't be undone.
That contradicts "every edit history of HTML, CSS and JS stay as they are". Dropping .ts and .mts from the set is enough. A real transport stream large enough to matter is already caught by the 64 MiB rule, and .m2ts stays. A test with an edited .ts file that still undoes would pin it.
|
Head follows this comment (rebased on main).
|
e5f71bf to
9baed70
Compare
There was a problem hiding this comment.
Verdict: REQUEST CHANGES — 1 resolved, 0 open, 2 new · e5f71bff → 9baed70a
The .ts/.mts fix is correct and pinned by a test. CI is red on this head, though, and both failures come from this PR:
- blocker —
Test (support):src/routes/history.test.ts:454("undo Studio's overwrite of media whose first copy was still running") still expects undoing aclip.mp4overwrite to restore the original. This PR deliberately stops keeping that version, so the step now refuses and the assertion getsfalse. Switch that test to a.binstand-in, asprojectHistory.test.tsalready does, and/or add a test asserting the refusal. - blocker —
Fallow audit:reclaimUncopied.ts:20, wherereclaimUncopiedBlobshas cognitive complexity 39 against a threshold of 15. Moving the per-history body (log read, size map, reclaim) into its own function should bring it under.
…oute test at a non-media file
There was a problem hiding this comment.
Verdict: REQUEST CHANGES — 2 resolved, 0 open, 1 new · 9baed70a → 5ea5730c
Both earlier reds are fixed: Fallow audit and Test (support) pass on this head. The route test now uses a .bin stand-in, and the split of reclaimUncopiedBlobs behaves exactly as before (same hashes, sizes and reclaim rule). One new lint error (inline) fails CI's Lint job and the Preflight (lint + format) step in Windows render verification and preview-regression.
| const hashes = new Set<string>(log.baseline.values()); | ||
| for (const entry of log.entries) | ||
| for (const file of entry.files) | ||
| for (const hash of [file.before, file.after]) hash && hashes.add(hash); |
There was a problem hiding this comment.
blocker: oxlint no-unused-expressions ("Expected expression to be used") fails Lint on this line, because hash && hashes.add(hash) is used as a statement.
| for (const hash of [file.before, file.after]) hash && hashes.add(hash); | |
| for (const hash of [file.before, file.after]) if (hash) hashes.add(hash); |
What
Project history copied every tracked file into
~/.cache/hyperframes/history/<id>/blobs, video and audio included. A project with a large camera file, a long recording and a render output being written grew its history by tens of gigabytes in minutes and filled the disk, so the render's own write failed. Each media file was stored at least once next to the original, and the current copies were never counted against the budget.Now:
Video and audio files, and any file over 64 MiB, are hashed where they are (streamed, no copy) and recorded by path and hash.
uncopied.tsis the one place that decides which files these are.When history opens, copies an older version already made of such files are removed, but only when every reference to that content is a file of that kind. Text, small images and every edit history of HTML, CSS and JS stay as they are.
Bringing back an earlier version of such a file is not offered: undo, restore and checkout of a change that needs it fail up front, before anything is written, naming the file ("An earlier version of clip.mp4 is not kept"). Everything else in history behaves as before.
hyperframes cleandoes the same for every history in the cache, not only the ones Studio opens: it takes each history that no running process holds, applies the same rule, and--dry-runlists the bytes ("Video and audio copies in project history").Not changed: the budget still counts stored bytes beyond what the current files need (a project larger than its budget keeps its history; a test pins that). With media gone, the current copies left are text and files under 64 MiB each, which cannot be shed anyway.
Evidence
A 50 MiB video present when history opens, and 25 MiB of video and audio written and rewritten while it is open: blob bytes stay under 1 KiB (they were the full file sizes).
A copy planted the way an older version left it is removed on open; the text blob next to it is kept and still readable.
Hashing in place costs no more than before: a file whose size and times have not moved is not read again (the existing stat cache), so reopening a project or saving a text edit does not rehash a large video. With a 1 GiB video, first adoption took 2.8 s instead of 5.8 s, reopening 18 ms (9 ms before), and a text-edit snapshot 29 ms (24 ms before); blob bytes 1 instead of 1,073,741,825.
hyperframes clean --dry-runon a planted history listed the 2.9 MB media copy and left it;hyperframes cleanremoved it and kept the text blob.Tests
reclaimUncopied.test.ts(dry run lists and keeps; a real run removes the media copy and keeps text) anduncopied.test.ts(four cases): no copy for media present at open, none for media written or replaced later, a text edit is still copied while the media half of the same change cannot be undone, and reclaim on open. All four fail on main. The history suite (218 tests) and the clean command's tests is green three runs in a row. InprojectHistory.test.tsthe stand-in for "large media" is now a.binfile, since.mp4is no longer copied.