feat(studio)!: further canvases with their own layouts, drawn from shared sources, each in its own broadcast

README.md

@@ -90,17 +90,17 @@ Every panel is a tab you can drag, split, or stack.
 
 | Panel | Use it to |
 | --- | --- |
-| Preview, Program | See the scene you are editing and the one that is live, and drag items in the preview to place them. **Cut** puts the preview on program at once; **Transition** takes it there by the transition picked beside it, over the time set there (see [Transitions](#transitions)). **Interact** passes the mouse over the preview to the browser source under it instead, and the keyboard to the one last clicked. |
+| Preview, Program | See the scene you are editing and the one that is live, and drag items in the preview to place them. With more than one canvas, pick the canvas they show, and that the Items panel edits, beside the preview. **Cut** puts the preview on program at once; **Transition** takes it there by the transition picked beside it, over the time set there (see [Transitions](#transitions)). **Interact** passes the mouse over the preview to the browser source under it instead, and the keyboard to the one last clicked. |
 | Scenes | Pick the scene in preview, and add, rename (double-click), reorder, or remove one. A scene on program, or still leaving it in a transition, cannot be removed, even by an undo, until program cuts away from it. |
 | History | See every edit, and undo or redo back to any of them. |
-| Items | Place, show, hide, and order what the preview scene or the top layer draws, back to front. Select an item to filter or copy it. |
+| Items | Place, show, hide, and order what the preview scene or the top layer draws on the canvas shown, back to front. Select an item to filter or copy it. |
 | Sources | Add a source by hand, see whether each and its own track are live and where it is shown, clone one, and remove one. Select a source to filter it. A camera or broadcast without a new picture, or a microphone or system audio without new sound, for a second reads as stalled. A browser source shows whether its page is copied on the GPU or, as a fallback, uploaded from memory (CPU), in place of a track, and **Reload** restarts its page. A media file reads as ended once it plays to its end without looping, **Play** plays it from its start, and **Reload** reads the file again. |
 | Devices | List this machine's cameras, displays, windows, and audio inputs, and add one as a source. |
 | Guests | Invite a guest, show their join link as a QR code, see who joined, and take a guest out of every scene. |
 | Filters | Crop the selected source or item, add, order, and set its filters, set an item's blend mode, and copy or paste filters. |
 | Keyframes | Animate the selected item: add, time, ease, and remove its keyframes, and set how far each moves, resizes, crops, and filters it (see [Keyframes](#keyframes)). **Play** plays the preview scene's keyframes again. |
 | Audio | Add, rename (double-click), and remove mixes, pick the one on program and the one to listen to and on which output, and set each audio source's gain, mute, delay, and the mixes it feeds, with a meter on every mix and source. |
-| Output | Start and stop recording, pick the mix it records, set the canvas size and frame rate, and see the broadcast path, whether the program is encoding, the encoder sessions open, and each rendition's encoder work. |
+| Output | Start and stop recording, pick the mix it records, set the canvas size and frame rate, add, resize, and remove further canvases, and see the broadcast path, whether the program is encoding, the encoder sessions open, and each canvas's renditions' encoder work. |
 | Health | See how the stream reaches viewers, relay by relay and track by track, and the cause of each problem. |
 | Replay | Replay the last moments of the program or a source in a replay source, slowed down if you like, or save them as a clip. |
 
@@ -109,6 +109,25 @@ Items panel shows. The **top layer** is drawn over every scene, in preview and
 on program, and stays on through a cut or transition: a downstream keyer for a
 logo or a lower third.
 
+### Canvases
+
+Beside the main canvas, the studio can draw further canvases at once, such as
+a 1080x1920 portrait one for a vertical stream beside a 1920x1080 landscape
+main canvas. Add one by name in the Output panel. Each scene has its own
+layout on each canvas, and each canvas its own top layer; a scene without a
+layout on a canvas shows only that canvas's top layer there. Every canvas
+shares the main canvas's frame rate, and the canvases are linked: one scene is
+in preview and one on program on all of them, and a cut or transition takes
+every canvas together. An item's keyframes play on every canvas from when its
+scene goes on, as on the main one. A source captures or decodes once however many canvases
+show it, and a canvas draws only the items it places.
+
+Each further canvas publishes a broadcast of its own beneath the main one, at
+`<--broadcast>/<canvas name>`, with renditions and encoders of its own (see
+[Output](#output)). Its catalog lists the main broadcast's audio rather than
+encoding it again. Removing a canvas ends its broadcast, and removes its top
+layer and every scene's layout on it.
+
 ### Transitions
 
 | Transition | What it does |
@@ -415,6 +434,11 @@ width = 1920
 height = 1080
 framerate = 30
 
+[canvases.portrait] # a further canvas, at the main one's frame rate
+width = 1080
+height = 1920
+top = [{ source = "logo", rect = { x = 780, y = 40, width = 280, height = 120 } }]
+
 [transition]
 kind = "slide"      # fade, slide, wipe, or move
 duration = "300ms"
@@ -485,6 +509,10 @@ items = [
     { source = "guest", rect = { x = 1280, y = 720, width = 640, height = 360 }, filters = [{ kind = "drop_shadow", x = 8, y = 8, blur = 12, color = "#00000080" }] },
     { source = "logo", rect = { x = 0, y = 0, width = 1920, height = 1080 }, blend = "multiply", filters = [{ kind = "opacity", amount = 0.3 }] },
 ]
+canvases.portrait = [ # the same scene laid out on the portrait canvas
+    { source = "camera", rect = { x = 0, y = 0, width = 1080, height = 1920 }, crop = { left = 0.34, right = 0.34 } },
+    { source = "guest", rect = { x = 0, y = 1312, width = 1080, height = 608 } },
+]
 
 [[scenes]]
 name = "Break"
@@ -512,6 +540,11 @@ file, which no item places, but which grades what a `lut` filter names it in),
 and `replay`, where the Replay panel plays replays. The Devices panel fills in
 ids. `[[top]]` items make up the top layer.
 
+`[canvases.<name>]` adds a further canvas, with its `width` and `height` and
+its own `top` layer, and a scene's `canvases.<name>` lists its items on that
+canvas; leave it out, rather than empty, for a scene that shows nothing there.
+A canvas name is one segment of its broadcast path, so it holds no `/`.
+
 `[mixes]` names every mix, in the order the Audio panel lists them, the one
 on `program`, and the one a `recording` holds, the mix on program without it;
 without `[mixes]`, the collection has one mix, `Program`.
@@ -657,17 +690,20 @@ encode engine for them all. The Output panel counts the sessions open:
 |---|---|
 | The program's canvas size, live | 1, or none without `--connect` |
 | Its half size | 1 while someone watches it |
+| Each further canvas's canvas size and half size | as the program's |
 | The recording's own canvas size | 1 while recording |
 | Each captured source or media file | 1 while on program or recorded |
 
 So a live scene showing a camera, a display, and a window holds 4 sessions, 5
-with the half size watched, and 6 while recording too. When the program
+with the half size watched, and 6 while recording too. A portrait canvas beside
+it adds 1 more, 2 with its half size watched. When the program
 encodes on hardware, so do the sources, and a
 session the GPU refuses fails that source's track with the encoder's reason in
 the Sources panel rather than falling back to software. The source keeps
 drawing in the studio's own scenes. The program opens its session before any
 source, even with `--on-demand`, so the sources past the cap fail rather than
-the program. When the program encodes in software, a source takes hardware
+the program; so does each canvas in the scene file at startup. A canvas added
+later, past the cap, draws without video and says why in the Output panel. When the program encodes in software, a source takes hardware
 where it can and software otherwise.
 
 ### Recording

quest/m0/multi-canvas.md

@@ -9,13 +9,19 @@ captures or decodes once however many canvases use it.
 
 ## Plan
 
-Each canvas publishes its own broadcast, because different content needs a
-different name. Each canvas has its own top layer, which the collection's
-`top` holds for the one canvas today. Transitions are per canvas, so a scene change cuts every
-canvas together unless the creator unlinks one. Reuse the one canvas's resize
-path per canvas: the compositor reallocates on a new canvas, and the program
-video publishes a rendition at a time, ending one and listing the next.
+Further canvases, their layouts and top layers, linked scene switching, and a
+broadcast per canvas at `<--broadcast>/<name>` landed in
+[moq-dev/studio#42](https://github.com/moq-dev/studio/pull/42). What remains:
 
-Benchmark source count against canvas count, including sparse reuse. Per-canvas
-cost must grow only with the sources that canvas uses, never with the whole
-source table.
+- Unlinking: a canvas the creator unlinks keeps its own preview and program
+  scene, so a scene change no longer cuts it with the others. Transitions are
+  per canvas, so an unlinked canvas takes its own.
+- Outputs per canvas: a recording, and each destination, picks the canvas it
+  carries, as each picks its mix. Only the main canvas records today.
+- Health per canvas: the Health panel grades each canvas's broadcast, with
+  viewer feedback beneath it, not only the main one's.
+- Preview cost: a canvas no panel shows skips drawing its preview, so an
+  unseen canvas costs only its program.
+
+Per-canvas cost must grow only with the sources that canvas uses, never with
+the whole source table; `compositor::bench` measures it.

rs/moq-scene/src/clip.rs

@@ -295,7 +295,7 @@ mod tests {
 	fn pastes_linked_within_a_collection() {
 		let collection = collection();
 		let clip = Clip::items(&collection, collection.scenes[0].items.clone(), Path::new(DIR));
-		let pasted = paste(&collection, &Layer::Top, &clip.to_toml().unwrap(), DIR);
+		let pasted = paste(&collection, &Layer::top(), &clip.to_toml().unwrap(), DIR);
 		assert_eq!(pasted.sources, collection.sources, "no source added");
 		assert_eq!(pasted.top, collection.scenes[0].items);
 	}
@@ -311,14 +311,14 @@ mod tests {
 		}];
 		let clip = Clip::items(&collection, collection.scenes[0].items.clone(), Path::new(DIR));
 		let text = clip.to_toml().unwrap();
-		let pasted = paste(&collection, &Layer::Scene("Scene".into()), &text, DIR);
+		let pasted = paste(&collection, &Layer::scene("Scene"), &text, DIR);
 		assert_eq!(pasted.scenes[0].items[1], collection.scenes[0].items[0]);
 
 		let Clip::Items(items) = Clip::from_toml(&text).unwrap() else {
 			panic!("not items")
 		};
 		let mut draft = collection.draft("Paste");
-		draft.paste(&Layer::Top, &items, Path::new(DIR)).unwrap();
+		draft.paste(&Layer::top(), &items, Path::new(DIR)).unwrap();
 		assert!(matches!(draft.finish(), Err(Error::Invalid(_))));
 	}
 
@@ -333,7 +333,7 @@ mod tests {
 
 		let mut other = Collection::new();
 		other.sources.insert("cam".into(), color([0, 255, 0, 255]));
-		let pasted = paste(&other, &Layer::Scene("Scene".into()), &text, DIR);
+		let pasted = paste(&other, &Layer::scene("Scene"), &text, DIR);
 		assert_eq!(pasted.sources["cam"], color([0, 255, 0, 255]), "left alone");
 		assert_eq!(pasted.sources["cam 2"], Source::Camera { id: None });
 		assert_eq!(pasted.sources["cam faded"], Source::Clone { source: "cam 2".into() });
@@ -367,7 +367,7 @@ mod tests {
 			.to_toml()
 			.unwrap();
 
-		let pasted = paste(&Collection::new(), &Layer::Top, &text, DIR);
+		let pasted = paste(&Collection::new(), &Layer::top(), &text, DIR);
 		let expected = crate::Audio {
 			exclude: ["Program".to_string()].into(),
 			..audio
@@ -393,7 +393,7 @@ mod tests {
 			.to_toml()
 			.unwrap();
 
-		let home = paste(&collection, &Layer::Top, &text, DIR);
+		let home = paste(&collection, &Layer::top(), &text, DIR);
 		assert_eq!(home.sources, collection.sources, "linked to the same image");
 
 		// However the scene file spells the path.
@@ -404,7 +404,7 @@ mod tests {
 				path: "/shows/one/logo.png".into(),
 			},
 		);
-		let home = paste(&absolute, &Layer::Top, &text, DIR);
+		let home = paste(&absolute, &Layer::top(), &text, DIR);
 		assert_eq!(home.sources, absolute.sources, "linked to the same image");
 
 		let mut other = Collection::new();
@@ -414,7 +414,7 @@ mod tests {
 				path: "logo.png".into(),
 			},
 		);
-		let away = paste(&other, &Layer::Top, &text, "/shows/two");
+		let away = paste(&other, &Layer::top(), &text, "/shows/two");
 		assert_eq!(away.sources["logo"], other.sources["logo"], "another file, left alone");
 		assert_eq!(
 			away.sources["logo 2"],
@@ -462,7 +462,7 @@ mod tests {
 
 		let mut other = Collection::new();
 		other.sources.insert("matte".into(), color([0, 255, 0, 255]));
-		let pasted = paste(&other, &Layer::Top, &text, "/shows/two");
+		let pasted = paste(&other, &Layer::top(), &text, "/shows/two");
 		assert_eq!(pasted.top[0].filters, [mask("matte 2")]);
 		assert_eq!(
 			pasted.sources["grade"],