feat: offline asset ledger and vendor preflight CLI by mvanhorn · Pull Request #3757 · heygen-com/hyperframes

mvanhorn · 2026-09-08T01:40:55Z

Summary

Agents and authors routinely pull CDN scripts and remote media into compositions, which silently breaks deterministic and offline renders. This PR adds an agent-readable asset graph and a one-command localization step:

hyperframes ledger [dir] [--json] [--strict-offline] — static inventory of every declared asset reference (scripts, stylesheets, fonts, images, audio, video, iframes, text tracks) across all project HTML, each classified remote | local | data | missing. No browser, no network. Under --strict-offline it exits non-zero while any remote reference remains — the CI gate for deterministic renders.
hyperframes vendor [dir] [--out assets/vendor] [--dry-run] [--strict-offline] — downloads each unique remote URL (scheme allowlist: http/https only; protocol-relative upgraded to https), saves it as <basename>-<sha256-prefix><ext>, rewrites every HTML reference to a path relative to the declaring file, and records provenance (url, bytes, sha256, contentType) in vendor-manifest.json. Remote iframes are deliberately never vendored — a live page cannot be made deterministic — so they still fail the strict gate.
Core module @hyperframes/core/asset-ledger — the pure extraction/classification engine (linear tag scan + CSS url()/@import/@font-face scanning), reusing @hyperframes/parsers/asset-resolution for on-disk resolution. New subpath export synced via package-subpaths.json.
Lint rule remote_script_not_vendored (info) — nudges toward hyperframes vendor when a remote <script src> is detected, without breaking the documented CDN quick-start path.
Docs — reference/cli-ledger (flags, JSON schema, exit codes) and guides/offline-deterministic-renders (workflow guide), both wired into the Mintlify nav.

Scope note: this does not implement Google Fonts crawling (#3583) — only declared URLs are inventoried/vendored; font binaries nested inside remote CSS are documented as out of scope.

Walkthrough video (rendered by HyperFrames itself)

The walkthrough is not a screen recording — it is a HyperFrames HTML composition, storyboarded first and rendered to MP4 by the CLI. Every terminal/JSON panel shows the real captured output of running the new commands on the demo project, and the demo is self-referential: the composition was authored with GSAP on jsDelivr, hyperframes vendor downloaded it to assets/vendor/gsap.min-0274614511.js and rewrote the <script src>, hyperframes ledger --strict-offline passed with zero remote references, and the video was then rendered with the vendored file.

▶ Walkthrough MP4 (public): https://raw.githubusercontent.com/mvanhorn/hyperframes/walkthrough-assets/offline-asset-ledger-walkthrough.mp4

(1920×1080, 30 s @ 30 fps, 2.2 MB — HyperFrames CLI render output)

Render command:

npx hyperframes render   # → renders/offline-ledger-demo_2026-09-08_01-32-43.mp4 (rendered in 32.9s)

Full demo pipeline (all real, exit codes verified):

npx hyperframes ledger --json            # counts: { total: 1, remote: 1, … } — flags the jsDelivr GSAP script
npx hyperframes vendor                   # ↓ …gsap.min.js → assets/vendor/gsap.min-0274614511.js (71.1 KB, sha256 c174bfce…)
npx hyperframes ledger --strict-offline  # 1 local · 0 remote — exit 0
npx hyperframes lint && npx hyperframes check   # 0 errors; 38/38 WCAG AA text checks pass
npx hyperframes render

Test plan

packages/core/src/assets/ledger.test.ts — 13 tests: URL classification (remote/data/local/skip, unknown schemes, templating placeholders), extraction (script tags, link rel/as, CSS @import/@font-face/backgrounds, srcset, poster, <source> parent-kind mapping, entity decoding, comment/script-string immunity), ledger assembly (counts, dedupe, resolver), and on-disk project scanning with file-relative resolution. Fixture HTML uses jsDelivr + local media; no browser required.
packages/cli/src/commands/ledger.test.ts — 5 tests: --json output shape, --strict-offline exit codes (1 with remotes, 0 when fully local), human output, error paths.
packages/cli/src/commands/vendor.test.ts — 10 tests: scheme allowlist, stable hashed filenames, content-type extension fallback, relative-path rewriting (incl. prefix-safe longest-first replacement), full command flow with mocked fetch (download + rewrite + manifest), --dry-run no-op guarantee, failed-download exit 1, --strict-offline failure on non-vendorable remotes.
packages/lint/src/rules/runtimeNetworkFetch.test.ts — 3 tests: info finding with vendor fixHint, URL dedupe, silence on local/inline scripts.
Full suites pass: @hyperframes/cli (vitest, exit 0), @hyperframes/core (exit 0), @hyperframes/lint (604 tests, exit 0). bun run build, root bun run lint (incl. check:package-subpaths), typecheck, oxlint/oxfmt, and the fallow audit all pass.
End-to-end verification against the live jsDelivr CDN is shown in the walkthrough video above.

AI disclosure

This PR was implemented by an AI agent (Claude Fable 5, running as a Cursor cloud agent), including the code, tests, docs, storyboard, and the HyperFrames composition used for the walkthrough video. A human (mvanhorn) directed scope and requirements.

- @hyperframes/core/asset-ledger: static asset-graph builder classifying every declared script/stylesheet/font/image/audio/video/iframe/track reference as remote | local | data | missing (no browser, no network) - hyperframes ledger [dir] --json --strict-offline: agent-readable inventory; non-zero exit when remote refs remain under strict mode - hyperframes vendor [dir] --out assets/vendor: downloads http(s) remotes, rewrites HTML references to relative paths, writes a provenance manifest - lint: remote_script_not_vendored info rule nudging toward vendoring

…ers guide

+  return value
+    .replace(/&amp;/gi, "&")


+const FONT_FACE_BLOCK_RE = /@font-face\s*\{[^}]*\}/gi;
+
+function stripCssComments(css: string): string {
+  return css.replace(/\/\*[\s\S]*?\*\//g, (m) => " ".repeat(m.length));


+          const { body, contentType } = await downloadAsset(fetchUrl, timeoutMs);
+          const fileName = vendorFileName(fetchUrl, contentType);
+          const vendorPath = posix.join(outRel, fileName);
+          writeFileSync(join(outDir, fileName), body);


- ledger: single-pass HTML entity decoder so entities decode exactly once (no double-unescape; &lt; -> <, never <) - ledger: replace regex CSS comment stripping with a linear O(n) scanner (no backtracking on adversarial input) - vendor: path traversal guard pinning every write under the vendor dir, --out containment check under the project root, per-download size cap (--max-bytes, default 100 MB), and a comment documenting that the network-to-file write is the command's intended purpose - tests for single-decode semantics, comment stripping, traversal guard, size cap, and --out containment

mvanhorn · 2026-09-08T04:07:56Z

Pushed fixes for the CodeQL findings (0a2710a):

Double-unescape in decodeHtmlEntities (packages/core/src/assets/ledger.ts) — replaced the chained .replace() calls with a single-pass decoder driven by one regex and an entity map. Each entity now decodes exactly once, so a decoded & can never recombine with following text into a second entity: &lt; decodes to <, never <. Added unit tests pinning the single-decode contract and that unknown entities are left untouched.

ReDoS risk in stripCssComments — replaced the /\/\*[\s\S]*?\*\//g regex with a linear O(n) indexOf-based scanner (same shape as the existing maskHtmlComments), still length-preserving. Added a test that a 50k-character hostile comment run completes quickly and comments still mask correctly.

Network→file in vendor.ts — the download-to-disk flow is this command's entire purpose (localize remote assets for offline renders), so the flagged flow is intentional; it is now explicitly documented at the write site and hardened:

http(s)-only scheme allowlist + URL validation retained (toFetchableUrl)
new resolveVendorTarget traversal guard: every write is resolved and verified to stay under the vendor directory before writeFileSync
--out itself is now verified to resolve inside the project root
new per-download size cap (--max-bytes, default 100 MB), enforced against both content-length and the actual body

No repo-idiomatic CodeQL suppression exists, so no suppression comment was added — just real guards plus the intent comment.

Tests: packages/core ledger suite (16 passed) and packages/cli vendor + ledger suites (19 passed); changed files pass oxlint/oxfmt.

+          // (vendorFileName), a traversal check pinning every write under the
+          // vendor directory (resolveVendorTarget, with outDir itself pinned
+          // under the project root), and a per-download size cap.
+          writeFileSync(resolveVendorTarget(outDir, fileName), body);


cursoragent added 2 commits September 8, 2026 01:18

docs: asset ledger & vendor reference plus offline deterministic rend…

f9048e8

…ers guide

github-advanced-security Bot found potential problems Sep 8, 2026

View reviewed changes

Comment thread packages/core/src/assets/ledger.ts Fixed

Comment thread packages/core/src/assets/ledger.ts Fixed

Comment thread packages/cli/src/commands/vendor.ts Fixed

cursor Bot pushed a commit to mvanhorn/hyperframes that referenced this pull request Sep 8, 2026

chore: add offline-asset-ledger walkthrough video (PR heygen-com#3757)

9def9ff

github-advanced-security Bot found potential problems Sep 8, 2026

View reviewed changes

Navigation Menu

Uh oh!

feat: offline asset ledger and vendor preflight CLI - #3757

feat: offline asset ledger and vendor preflight CLI#3757
mvanhorn wants to merge 3 commits into
heygen-com:mainfrom
mvanhorn:feat/offline-asset-ledger-vendor

mvanhorn commented Sep 8, 2026 •

edited

Loading

Uh oh!

Uh oh!

Uh oh!

Uh oh!

mvanhorn commented Sep 8, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Uh oh!

Conversation

mvanhorn commented Sep 8, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

Walkthrough video (rendered by HyperFrames itself)

Test plan

AI disclosure

Uh oh!

Uh oh!

Uh oh!

Uh oh!

mvanhorn commented Sep 8, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

mvanhorn commented Sep 8, 2026 •

edited

Loading