Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 4 additions & 4 deletions quest/m0/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,7 @@
A native, cross-platform streaming studio (`rs/moq-studio`) for creators and
teams who rely on OBS today, plus the same app built to wasm. It composites
cameras, screens, media, browser sources, and remote MoQ guests into scenes
on the GPU with moq-video and moq-audio, across several canvases at once
on the GPU with moq-video and moq-audio, publishing several scenes at once
(landscape and portrait), and goes live to a MoQ relay and directly to RTMP,
RTMPS, and WHIP ingests.

Expand Down Expand Up @@ -37,7 +37,7 @@ Decided 2026-10-22 with the maintainer (/quest-plan):
destination itself. It remuxes its own broadcast through a UI-free egress
library that a hosted service can also run server-side. There is no
gateway service and no moq-cli dependency, because a studio user does not
run a server. Destinations that take the same canvas and audio mix with
run a server. Destinations that take the same picture scene and audio scene with
identical encode settings share one encoder. The studio picks codecs per destination: Opus by default, AAC
where an ingest requires it, from platform encoders only.
- **Extensibility:** first-party features plus browser sources (CEF
Expand Down Expand Up @@ -93,13 +93,13 @@ repository.
- [Wayland display and window sources](/quest/m0/wayland-sources.md) - portal-picked sources that reopen without the picker
- [ML video effects](/quest/m0/effects.md) - background removal and other ML effects, baked into their source's track
- [Destinations](/quest/m0/destinations.md) - RTMP, RTMPS, and WHIP push with per-destination codecs and shared encoders, on a UI-free egress library a hosted service can reuse
- [Multiple canvases](/quest/m0/multi-canvas.md) - landscape and portrait canvases composite at once from shared sources
- [Composable scenes](/quest/m0/scenes/README.md) - scenes nest scenes and the program, any scene publishes at its own size, and mixes become audio scenes
- [The join page is live](/quest/m0/join-page-live.md) - `join.studio.moq.dev` serves the join page `--join-page` defaults to
- [Clap-to-measure sync delay](/quest/m0/sync-clap.md) - a clap sets a microphone's delay against its camera
- [Audio filters](/quest/m0/audio-filters.md) - per-source noise suppression, gate, compressor, limiter, and plugins, configured from the studio
- [Browser sources on macOS and Windows](/quest/m0/browser-source.md) - the out-of-process CEF pages Linux has, with shared GPU textures on the other desktops
- [Web studio](/quest/m0/web.md) - the full studio in wasm on WebGPU and WebCodecs, served from this repository
- [Virtual camera](/quest/m0/vcam/README.md) - a canvas's program exposed as a system camera on Linux, Windows, and macOS
- [Virtual camera](/quest/m0/vcam/README.md) - the program or a published scene exposed as a system camera on Linux, Windows, and macOS
- [Browser sources without an X display](/quest/m0/cef-headless.md) - CEF runs headless, so Wayland sessions and tests need no X server
- [Signing identities are available](/quest/m0/signing.md) - the Apple and Windows code-signing credentials packaging and the macOS camera need
- [Packaging](/quest/m0/packaging.md) - signed installers for macOS and Windows and a Flatpak for Linux
Expand Down
4 changes: 2 additions & 2 deletions quest/m0/destinations.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@

A creator adds any number of RTMP, RTMPS, and WHIP destinations alongside
MoQ, each with its own codec and bitrate, and goes live to all of them at
once. Destinations that take the same canvas and audio mix with identical
once. Destinations that take the same picture scene and audio scene with identical
encode settings share one encoder. Each destination's status and errors show
in the UI.

Expand Down Expand Up @@ -54,7 +54,7 @@ Decided 2026-10-22 with the maintainer:
- **Benchmarks:**
- Sweep concurrent broadcasts against destinations per broadcast in the
library.
- Sweep distinct encoder-sharing keys (canvas, mix, and settings) against
- Sweep distinct encoder-sharing keys (picture scene, audio scene, and settings) against
destinations per key in the studio. Finding a destination's encoder must
not scan every encoder group, and identical keys must never spawn a
duplicate encoder.
Expand Down
27 changes: 0 additions & 27 deletions quest/m0/multi-canvas.md

This file was deleted.

56 changes: 56 additions & 0 deletions quest/m0/scenes/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,56 @@
# Composable scenes

## Goal

Scenes form a graph. A scene can draw another scene, or the live program, as
a source. Any scene can be published as its own broadcast at its own size, and
the old audio mixes are audio-only scenes that nest the same way. This
replaces separate canvases: a portrait broadcast is a published 1080x1920
scene that draws the program, cropped, with its own overlays.

This epic's own work is an end-to-end test: one studio publishes the program
at `<--broadcast>` and a portrait scene drawing it at
`<--broadcast>/<scene>`, each carrying its picked audio scene, and a viewer of
each sees and hears the same cut at the same moment.

## Plan

Decided 2026-11-05 with the maintainer (/quest-plan), replacing the canvases
of [moq-dev/studio#42](https://github.com/moq-dev/studio/pull/42) and closing
[moq-dev/studio#54](https://github.com/moq-dev/studio/pull/54):

- ✅ **Published scenes + scene source** rather than canvases with a layout
per scene and linked or unlinked switching. One concept (a scene) instead of
two, and no unlinking. Rejected: published scenes that never follow cuts;
keeping canvases.
- ✅ **A scene source draws any scene, or `program`** (whatever is on air,
transitions included), scaled and cropped like any picture. Rejected: a
program-only source.
- ✅ **The program stays one switcher at `<--broadcast>`**: Preview, Program
and Cut work as today. Other published scenes are always live at
`<--broadcast>/<scene>` and follow cuts only through a scene source.
Rejected: every published scene switchable on its own.
- ✅ **Every scene has a size**, defaulting to the program's. The switcher
offers only scenes of the program's size.
- ✅ **Mixes become audio scenes** that hold only sounds and nest, so a
"Mics" scene can feed "Stream" and "Headphones". Audio stays independent of
cuts, as today. Each published output pairs a picture scene with an audio
scene, defaulting to the one on program. Rejected: scenes carrying their own
audio, so a cut switches audio; keeping `[mixes]` separate.
- ✅ **UI:** the Scenes panel lists picture and audio scenes, tagged by kind.
The Audio panel stays a mixer (fader, pan, meter, and sends to audio scenes)
for live work, as pro switchers do. A graph view of the same data is
[its own quest](/quest/m1/scene-graph.md).
- Nothing is released, so the scene file breaks freely: `[canvases]` and
`[mixes]` go away, with no migration.

Carried over from #54: Health grades each published broadcast; a recording
picks a picture scene and an audio scene; a scene no panel shows and no output
reads skips drawing. Per-scene cost grows only with what that scene draws;
`compositor::bench` measures it.

## Required

- [Scene source](/quest/m0/scenes/scene-source.md) - a scene draws another scene, or the live program, as a source
- [Published scenes](/quest/m0/scenes/published.md) - any scene publishes as its own broadcast at its own size, replacing canvases
- [Audio scenes](/quest/m0/scenes/audio.md) - mixes become audio-only scenes that nest
29 changes: 29 additions & 0 deletions quest/m0/scenes/audio.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,29 @@
# [M] Audio scenes

## Goal

The audio mixes become audio-only scenes. An audio scene holds sounds
(sources' audio, with gain, pan, mute, and delay) and other audio scenes, so a
"Mics" scene can feed both "Stream" and "Headphones". **On program** picks the
audio scene the broadcast carries; each published scene and each recording
picks one. Audio stays independent of cuts.

## Plan

Decided 2026-11-05 with the maintainer (/quest-plan); see the
[epic](/quest/m0/scenes/README.md).

- Replaces `[mixes]` and `[audio.<source>]` in the scene file with no
migration; nothing is released.
- An audio scene cannot contain itself, directly or through others.
- The Audio panel stays a mixer: a strip per sound with fader, pan, meter,
and a send to each audio scene. Listen and On program work as today.
- The guest return feed (a mix-minus) and the sync delay keep working; a
guest hears the audio scene on program without their own sound.
- Switching or editing an audio scene never drops or repeats audio, as with
mixes today.
- `README.md` documents audio scenes.

## Required

- [Scene source](/quest/m0/scenes/scene-source.md) - shares the scene graph and its cycle check
32 changes: 32 additions & 0 deletions quest/m0/scenes/published.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,32 @@
# [L] Published scenes

## Goal

Any scene can be published: ticked **Publish**, it goes live as its own
broadcast at `<--broadcast>/<scene>`, at its own size, with the audio scene it
picks (the one on program by default). This replaces canvases: further
canvases, their per-scene layouts, and the canvas picker are removed, and a
portrait broadcast is a published portrait scene that draws the program
through a [scene source](/quest/m0/scenes/scene-source.md).

## Plan

Decided 2026-11-05 with the maintainer (/quest-plan); see the
[epic](/quest/m0/scenes/README.md).

- Every scene has a size, defaulting to the program's; the switcher offers
only scenes of the program's size.
- The program keeps one switcher at `<--broadcast>`. A published scene is
always live and never switched.
- Each published scene gets the program's two sizes (full and half) and the
bandwidth grant shares across all of them, as further canvases did.
- Carried from [moq-dev/studio#54](https://github.com/moq-dev/studio/pull/54):
Health grades each published broadcast with its viewer feedback; a
recording picks a picture scene and an audio scene; a scene no panel shows
and no output reads skips drawing.
- Remove `[canvases]` from the scene file with no migration; nothing is
released. `README.md` documents published scenes.

## Required

- [Scene source](/quest/m0/scenes/scene-source.md) - a portrait scene follows the program through it
23 changes: 23 additions & 0 deletions quest/m0/scenes/scene-source.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,23 @@
# [M] Scene source

## Goal

A new source kind, `scene`, draws another scene, or `program` (whatever is on
air, including a transition in progress), as a picture that an item places,
scales, crops, and filters like any other. A scene cannot contain itself,
directly or through other scenes; such an edit is refused with an error.

## Plan

Decided 2026-11-05 with the maintainer (/quest-plan); see the
[epic](/quest/m0/scenes/README.md) for the model.

- A nested scene renders at its own size into a texture the compositor
samples, once per frame however many items draw it, and only while
something that is drawn reads it.
- `program` reads the frame the program output draws, so a scene drawing it
shows the same cut on the same frame, with no extra composite.
- Keyframes, stingers, and restarts in a nested scene start when the scene
that draws it goes on program, as its items would.
- Measure frame time and GPU copies with and without a nested scene.
- `README.md` documents the source kind.
4 changes: 2 additions & 2 deletions quest/m0/vcam/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,9 +2,9 @@

## Goal

The studio exposes a canvas's program as a system camera, so video-call and
The studio exposes the program or a published scene as a system camera, so video-call and
conferencing apps, which offer no ingest API, can use it. The studio has one
toggle per canvas, and each OS backend lands on its own.
toggle per published scene, and each OS backend lands on its own.

## Plan

Expand Down
2 changes: 1 addition & 1 deletion quest/m0/vcam/linux.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@

## Goal

The studio publishes a canvas's program as a PipeWire video node that
The studio publishes the program or a published scene as a PipeWire video node that
PipeWire-aware apps (browsers and conferencing apps) list as a camera. It
falls back to an existing v4l2loopback device when one is present.
The studio's Devices panel leaves out its own virtual camera, so picking it
Expand Down
2 changes: 1 addition & 1 deletion quest/m0/vcam/macos.md
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@
## Goal

The macOS app bundle carries a CoreMediaIO Camera Extension. Once the user
approves it, apps list a camera that carries a canvas's program. An app
approves it, apps list a camera that carries the program or a published scene. An app
update installs the new extension without a reboot.

## Required
Expand Down
2 changes: 1 addition & 1 deletion quest/m0/vcam/windows.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@

## Goal

On Windows 11, the studio registers a canvas's program as a session-lifetime
On Windows 11, the studio registers the program or a published scene as a session-lifetime
camera through `MFCreateVirtualCamera`, backed by a software camera source.
Apps list it as a camera while the studio runs.

Expand Down
1 change: 1 addition & 0 deletions quest/m1/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -28,3 +28,4 @@ than opening an m2 for them.
- [Overlay assets](/quest/m1/overlay-assets.md) - overlays use fonts and images from data: URLs or files beside the scene
- [Hotkeys and editor preferences](/quest/m1/hotkeys.md) - system-wide hotkeys, arrow nudge, aspect lock, and persisted editor settings
- [Isolated guest recordings](/quest/m1/guest-recording.md) - record-each-source also writes each guest's tracks to its own file
- [Scene graph view](/quest/m1/scene-graph.md) - a node view of picture and audio scenes, their routing, volume, pan, and waveforms
19 changes: 19 additions & 0 deletions quest/m1/scene-graph.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,19 @@
# [M] Scene graph view

## Goal

A Graph panel shows the scene graph: picture and audio scenes as nodes, the
sources and scenes each one draws or sends to as edges, with each sound's
volume, pan, and a live waveform on its node. Edges and node settings can be
edited there, as in the Scenes and Audio panels.

## Plan

Decided 2026-11-05 with the maintainer (/quest-plan): the live controls stay
a mixer, as pro switchers keep them, and this view is for building and
inspecting routing, as node tools (TouchDesigner, Fusion) do. It reads and
edits the same data, so it adds no second model.

## Required

- [Composable scenes](/quest/m0/scenes/README.md) - the graph this view shows