diff --git a/quest/m0/README.md b/quest/m0/README.md index 3bd519d..eb29ff6 100644 --- a/quest/m0/README.md +++ b/quest/m0/README.md @@ -104,6 +104,11 @@ repository. - [Published independently](/quest/m0/publish-independently.md) - a source opts out of the program and publishes its own track for services that place it themselves - [Web studio](/quest/m0/web.md) - the full studio in wasm on WebGPU and WebCodecs, served from this repository - [Virtual camera](/quest/m0/vcam/README.md) - a canvas's program exposed as a system camera on Linux, Windows, and macOS +- [Clean quit](/quest/m0/clean-quit.md) - quitting finishes every track and closes the relay session, so the broadcast's tail arrives +- [Recover crashed recordings](/quest/m0/recording-repair.md) - a recording a crash left unfinished is repaired beside the original at launch +- [Premultiplied source sampling](/quest/m0/premultiply.md) - scaled transparent sources draw without edge fringes +- [Deterministic browser source startup](/quest/m0/cef-startup.md) - CEF on the CPU path paints its first frame without the GPU fallback hang +- [A browser renderer that dies on its first page](/quest/m0/browser-first-crash.md) - a renderer lost during the first navigation fails its source - [Signing identities are available](/quest/m0/signing.md) - the Apple and Windows code-signing credentials packaging and the macOS camera need - [Packaging](/quest/m0/packaging.md) - signed installers for macOS and Windows and a Flatpak for Linux - [The persistable portal restore token is pinned](/quest/m0/pin-portal-token.md) - moq-video's restore token, for Wayland sources diff --git a/quest/m0/browser-first-crash.md b/quest/m0/browser-first-crash.md new file mode 100644 index 0000000..a02d1a0 --- /dev/null +++ b/quest/m0/browser-first-crash.md @@ -0,0 +1,22 @@ +# [S] A browser renderer that dies on its first page + +## Goal + +A browser source whose renderer dies during its first navigation shows as +failed, as one that crashes later does, instead of staying Starting forever. +`a_crash_drops_only_its_source` passes on NixOS as well as Ubuntu. + +## Plan + +From [moq-dev/studio#8](https://github.com/moq-dev/studio/pull/8) and +[moq-dev/studio#29](https://github.com/moq-dev/studio/pull/29). + +- A renderer crashed by `chrome://crash` as the first URL fires neither + `on_render_process_terminated` (`rs/moq-scene/src/browser/host.rs`) nor + `on_load_error`, so the page stays Starting. Checked with debug prints on + #29. +- Find the signal CEF does give for it (for example the browser's process + exit or a load that never commits) and fail the source on it, without a + timeout: fail loud on an observed event. +- Split the test: one crash during the first navigation, and one after the + page commits. diff --git a/quest/m0/cef-startup.md b/quest/m0/cef-startup.md new file mode 100644 index 0000000..690380b --- /dev/null +++ b/quest/m0/cef-startup.md @@ -0,0 +1,32 @@ +# [S] Deterministic browser source startup + +## Goal + +A browser source paints its first frame within a bounded, repeatable time +when its frames come back over the CPU path, and the browser source tests +no longer take 4 to 33 seconds or hit their budget. + +## Plan + +From [moq-dev/studio#8](https://github.com/moq-dev/studio/pull/8). + +On the CPU path (no shared GPU texture, and CI under `just render-sw`), this +build's CEF cannot load GL. Chromium walks its GPU fallback chain, and in the +slow runs a fallback GPU process fails "Failed to create shared context for +virtualization", then hangs until Chromium's GPU watchdog kills it, 4 to 11 +seconds later, sometimes twice. + +Decided 2026-11-04 with the maintainer (/quest-plan): + +- ✅ **Pass `--disable-gpu` on the CPU path.** The frames come back on the + CPU anyway, so software raster gives deterministic startup. WebGL pages + run slower on that path. + Rejected: packaging GL and EGL for CEF (keeps WebGL accelerated, but adds + packaging and driver variance). + +Also check whether CEF starts without an X display under this flag +(`--ozone-platform=headless` started before, but its GPU process failed +EGL), and record the result in the quest's PR. + +Verify by running `a_click_reaches_the_page` many times under +`just render-sw`, and report the startup time's spread before and after. diff --git a/quest/m0/clean-quit.md b/quest/m0/clean-quit.md new file mode 100644 index 0000000..7b6e99e --- /dev/null +++ b/quest/m0/clean-quit.md @@ -0,0 +1,27 @@ +# [S] Clean quit + +## Goal + +Quitting the studio delivers everything it published before the process +exits: the program's video and audio and each source's own track end +cleanly, and the relay session closes gracefully, so a viewer or a recording +on the relay receives the tail of the broadcast instead of a cut. +A worker that fails to start rolls back what it already started, rather +than leaving threads or tracks running. + +## Plan + +From [moq-dev/studio#23](https://github.com/moq-dev/studio/pull/23) and +[moq-dev/studio#22](https://github.com/moq-dev/studio/pull/22). + +- `Engine::stop` (`rs/moq-studio/src/engine.rs`) drains and finishes the + program video, but `main` then returns and drops the runtime, so the tail + may never reach the relay. +- `moq_tokio::Connection::close` drains, bounded at one second, once every + track is finished: program audio, and each source's track. Source threads + are not stopped at exit today, so stop them and finish their tracks first. +- A worker that fails partway through starting (the stop signal raised on + #10 and #22) stops the parts it already started before reporting the + error. +- Test on a paused `kio::clock`: quit while live to a local relay, and a + subscriber receives the last group of every track and then the end. diff --git a/quest/m0/premultiply.md b/quest/m0/premultiply.md new file mode 100644 index 0000000..f8bd901 --- /dev/null +++ b/quest/m0/premultiply.md @@ -0,0 +1,23 @@ +# [XS] Premultiplied source sampling + +## Goal + +A scaled source with transparency, such as a PNG logo, draws without a dark +or light fringe at its edges, because the compositor samples it +premultiplied. + +## Plan + +From [moq-dev/studio#17](https://github.com/moq-dev/studio/pull/17), raised +by CodeRabbit on `rs/moq-scene/src/effect.wgsl`. It predates that PR. + +- Source pictures are sampled bilinearly straight (not premultiplied), so a + transparent texel's color bleeds into its neighbours when scaled. + Premultiply before interpolation: upload premultiplied, or premultiply each + tap in the shader, whichever adds no GPU copy. The effect pass's + `premultiplied` flag is not the fix: `fetch()` in `effect.wgsl` interpolates + first and `straight()` divides afterwards, so the flag only describes the + input. +- A compositor test scales a transparent picture with a colored transparent + region and checks that edge texels carry no color from it. +- Measure frame time before and after. diff --git a/quest/m0/recording-repair.md b/quest/m0/recording-repair.md new file mode 100644 index 0000000..3a26f8b --- /dev/null +++ b/quest/m0/recording-repair.md @@ -0,0 +1,33 @@ +# [S] Recover crashed recordings + +## Goal + +When the studio starts and finds a recording that a crash or power loss left +unfinished, it repairs the file without asking, writes the result as +`.repaired.mp4` beside the original, and reports the repair in Health +and the log. The original stays untouched. + +## Plan + +From [moq-dev/studio#18](https://github.com/moq-dev/studio/pull/18). + +Decided 2026-11-04 with the maintainer (/quest-plan): + +- ✅ **Repair automatically and keep the original.** The repaired copy is + written to a new file, so a repair that goes wrong never costs the only + copy, and no prompt blocks startup. + Rejected: repair in place (risks the only copy), ask first (a prompt at + every launch after a crash). + +Implementation: + +- `moq_mux::container::mp4::repair` rebuilds a file from its fragments. +- The recording directory and file naming are in `rs/moq-studio/src/record.rs`. + Mark a file as in progress with a sidecar (for example `.mp4.part`) + written when it starts and deleted when it finishes. Launch repairs each + file with a sidecar, then deletes the sidecar once `.repaired.mp4` is + written, so the same file is never repaired twice and the original keeps + its name. +- Repair each source's file as well as the program's. +- A test writes a recording, stops before finishing it, and checks that + launch produces a playable `.repaired.mp4` holding the samples written. diff --git a/quest/m1/README.md b/quest/m1/README.md index 5f8a276..a739aa7 100644 --- a/quest/m1/README.md +++ b/quest/m1/README.md @@ -1,10 +1,12 @@ -# m1: viewers compose +# m1: after the studio ships ## Goal -A studio broadcast can be composed by its viewers and by a server from the -sources' own tracks and a layout track, matching the program the studio -composites, so a viewer can toggle and prioritize elements. +The work held until m0 ships. A studio broadcast can be composed by its +viewers and by a server from the sources' own tracks and a layout track, +matching the program the studio composites, so a viewer can toggle and +prioritize elements. The studio gains features m0 left out: overlay assets +and restarts, global hotkeys, and isolated guest recordings. ## Plan @@ -15,6 +17,14 @@ default (decided 2026-11-03 on encoding every source for viewers costs more encoder sessions than a consumer GPU has. Viewer-side composition waits until m0 ships. +Broadened 2026-11-04 (maintainer, /quest-plan): m1 also holds studio +features planned as follow-ups of m0 work but not needed to ship it, rather +than opening an m2 for them. + ## Required - [Viewer-side composition](/quest/m1/compose/README.md) - sources as their own tracks plus a layout track, composed by viewers and by a server +- [Overlays restart on air](/quest/m1/overlay-restart.md) - an overlay's CSS timeline restarts at each cut unless set to keep running +- [Overlay assets](/quest/m1/overlay-assets.md) - overlays use fonts and images from data: URLs or files beside the scene +- [Hotkeys and editor preferences](/quest/m1/hotkeys.md) - system-wide hotkeys, arrow nudge, aspect lock, and persisted editor settings +- [Isolated guest recordings](/quest/m1/guest-recording.md) - record-each-source also writes each guest's tracks to its own file diff --git a/quest/m1/guest-recording.md b/quest/m1/guest-recording.md new file mode 100644 index 0000000..2354728 --- /dev/null +++ b/quest/m1/guest-recording.md @@ -0,0 +1,28 @@ +# [M] Isolated guest recordings + +## Goal + +Recording each source also writes a file for each guest and other broadcast +source, so an editor gets every guest's camera and microphone on its own. +The tracks are copied as received, with no re-encode. + +## Plan + +From [moq-dev/studio#18](https://github.com/moq-dev/studio/pull/18). + +Decided 2026-11-04 with the maintainer (/quest-plan): + +- ✅ **Extend the existing record-each-source option**, rather than adding a + separate guest toggle: one switch gives an editor every isolated track. + +Implementation: + +- `rs/moq-studio/src/record.rs` records each source's own track from the + studio's local broadcast. A guest's tracks come from the relay origin the + studio subscribes through, so read them from there and name the file after + the guest. +- A guest that leaves and rejoins mid-recording continues in a new file, or + in the same one if its timeline continues; pick the one moq-mux supports + and say which in `README.md`. +- Test with a guest published to a local relay: its file holds the guest's + samples unchanged. diff --git a/quest/m1/hotkeys.md b/quest/m1/hotkeys.md new file mode 100644 index 0000000..8c02bf7 --- /dev/null +++ b/quest/m1/hotkeys.md @@ -0,0 +1,42 @@ +# [L] Hotkeys and editor preferences + +## Goal + +The operator binds keys to studio actions, and they fire system-wide, even +while another app such as a game has focus: +- replay +- save a clip +- cut, transition, and select a scene +- start and stop recording and going live + +In the focused preview, arrow keys nudge the selected item, and holding a +modifier while resizing locks its aspect ratio. +Snap and grid settings and the hotkey bindings persist in an app settings +store, apart from the scene file. + +## Plan + +From [moq-dev/studio#15](https://github.com/moq-dev/studio/pull/15) and +[moq-dev/studio#20](https://github.com/moq-dev/studio/pull/20). + +Decided 2026-11-04 with the maintainer (/quest-plan): + +- ✅ **Global hotkeys in this quest**, as OBS has, since an operator fires a + replay from a game. Rejected: focused-only first with global later. +- ✅ **No default bindings.** Every action starts unbound, like OBS, so a + global key never collides with a game. Arrow-key nudge is always on but + works only in the focused preview. + +Implementation: + +- Per platform: the XDG GlobalShortcuts portal on Wayland (falling back to + focused-only where the portal is missing, said in the UI), `RegisterHotKey` + on Windows, and an event tap on macOS, which needs the Input Monitoring + grant; ask for it when the first key is bound. Prefer a maintained crate + that covers these over writing each. +- The settings store lives in the platform config directory, not the scene + file, so a scene file moves between machines without carrying one + operator's keys. +- A Hotkeys panel binds and clears keys, and shows a conflict. +- Aspect lock: the default for video sources, which have a fixed shape. +- `README.md` documents the panel, the macOS grant, and the Wayland portal. diff --git a/quest/m1/overlay-assets.md b/quest/m1/overlay-assets.md new file mode 100644 index 0000000..6f92fe6 --- /dev/null +++ b/quest/m1/overlay-assets.md @@ -0,0 +1,27 @@ +# [S] Overlay assets + +## Goal + +An overlay uses its own fonts and images, such as a logo or a brand face, +given as `data:` URLs or as files beside the scene file. Any other URL is +refused with an error naming it. + +## Plan + +From [moq-dev/studio#34](https://github.com/moq-dev/studio/pull/34). + +Decided 2026-11-04 with the maintainer (/quest-plan): + +- ✅ **Local files and `data:` URLs only.** With no network, what goes on air + is deterministic and private, and an asset can't fail or change mid-show. + Rejected: also fetching over https. + +Implementation: + +- Give Blitz a `NetProvider` that serves only `data:` URLs and files under + the scene file's directory, rejecting `..` escapes and symlinks out of it. +- Widen `overlay::check` (`rs/moq-scene/src/overlay.rs`) to accept + `@font-face` and `` / `background-image` that resolve that way. +- Load assets before the overlay first paints, so its first frame on program + has them. +- `README.md` documents where assets go. diff --git a/quest/m1/overlay-restart.md b/quest/m1/overlay-restart.md new file mode 100644 index 0000000..09e6194 --- /dev/null +++ b/quest/m1/overlay-restart.md @@ -0,0 +1,27 @@ +# [XS] Overlays restart on air + +## Goal + +An overlay's CSS timeline restarts each time it goes on program, so a lower +third plays its entrance at every cut, unless the overlay is set to keep +running. + +## Plan + +From [moq-dev/studio#34](https://github.com/moq-dev/studio/pull/34). + +Decided 2026-11-04 with the maintainer (/quest-plan): + +- ✅ **A per-overlay `restart` flag, true by default**, like OBS's "refresh + when the scene becomes active". A clock or ticker overlay turns it off so + it keeps running across cuts. + Rejected: always restart (no way to keep a running clock). + +Implementation: + +- `rs/moq-scene/src/overlay.rs` already starts an overlay's CSS timeline at a + `start` on the program clock; with `restart`, reset `start` when the item's + scene goes on program, matching when keyframes start. +- A scene in preview restarts its overlays when it goes into preview, as + keyframes do, so the preview shows the entrance. +- The flag is in the scene file, the overlay's add form, and `README.md`.