feat(studio): rename and reorder scenes, set the fade time, and resize the canvas live

README.md

@@ -76,26 +76,25 @@ canvas and no sources. The proposed scene is the first edit in the history, so
 undoing it leaves that empty scene.
 
 Setup writes an ordinary scene file. Change its sources and scenes in their
-panels, and its canvas in the file, which takes a restart. The Output panel
-shows the canvas in use, and the Health panel shows how the uplink holds up
-once you are live.
+panels, and its canvas in the Output panel. The Health panel shows how the
+uplink holds up once you are live.
 
 ## Panels
 
 Every panel is a tab you can drag, split, or stack.
 
 | Panel | Use it to |
 | --- | --- |
-| Preview, Program | See the scene you are editing and the one that is live, and drag items in the preview to place them. **Cut** puts the preview on program at once; **Fade** blends over the collection's fade time. |
-| Scenes | Pick the scene in preview, add one, or remove one. |
+| Preview, Program | See the scene you are editing and the one that is live, and drag items in the preview to place them. **Cut** puts the preview on program at once; **Fade** blends over the fade time set beside it. |
+| Scenes | Pick the scene in preview, and add, rename (double-click), reorder, or remove one. A scene on program, or still fading out, cannot be removed, even by an undo, until program cuts away from it. |
 | History | See every edit, and undo or redo back to any of them. |
 | Items | Place, show, hide, and order what the preview scene or the top layer draws, back to front. Select an item to filter or copy it. |
 | Sources | Add a source by hand, see whether each and its own track are live and where it is shown, clone one, and remove one. Select a source to filter it. |
 | Devices | List this machine's cameras, displays, windows, and audio inputs, and add one as a source. |
 | Guests | Invite a guest, show their join link as a QR code, see who joined, and take a guest out of every scene. |
 | Filters | Crop the selected source or item, add, order, and set its filters, set an item's blend mode, and copy or paste filters. |
 | Audio | Set each audio source's gain and mute, and watch its meter and the program's. |
-| Output | See the broadcast path, whether the program is encoding, the canvas, and the encoder's work. |
+| Output | Set the canvas size and frame rate, and see the broadcast path, whether the program is encoding, and the encoder's work. |
 | Health | See how the stream reaches viewers, relay by relay and track by track, and the cause of each problem. |
 
 A new source, from the Sources or Devices panel, is placed over whatever the
@@ -304,7 +303,7 @@ leaving half of it, and a collection that has not changed is not rewritten, so
 a hand-written file keeps its comments and layout until the next edit in the
 studio. A failed write shows in the status bar and
 is retried every second; until one succeeds, a crash loses every edit since the
-last write. A new canvas size or frame rate takes a restart.
+last write.
 
 The file can be edited by hand, or by another studio, while the studio runs.
 The studio looks at it every second and before each write. With no edits of
@@ -333,6 +332,14 @@ with Opus stereo audio, under the `--broadcast` path. Timestamps come from the
 studio's own clock, so a source that stalls repeats its last picture instead of
 holding up the stream.
 
+A new canvas size or frame rate, from the Output panel, an undo, or a reload,
+takes effect live. A track never changes what it carries, so the old
+rendition's track ends and a new one is listed in its place in a catalog
+update, on the same timeline, and viewers switch to it; the video pauses while
+the new encoder opens. Items keep their place in canvas pixels. A canvas the
+encoder cannot take stops the video, saying why, until the canvas changes
+again.
+
 The program encodes whether or not anyone watches, so a viewer who arrives
 late can fetch the recent past. `--on-demand` copies and encodes the video only
 while someone watches it, which saves that work but leaves nothing from before

quest/m0/README.md

@@ -83,7 +83,6 @@ repository.
 - [Studio on Windows](/quest/m0/app-windows.md) - the studio runs on Windows, not only Linux
 - [Viewer-side composition](/quest/m0/compose/README.md) - sources as their own tracks plus a layout track, composed by viewers, with a server-composed program as the fallback
 - [The program encoder drains](/quest/m0/program-drain.md) - a hardware encoder never loses the program's last frames at exit or on pause
-- [Scene editing](/quest/m0/scene-editing.md) - rename and reorder scenes, and resize the canvas while live through a new rendition
 - [Broadcast sources play in sync](/quest/m0/broadcast-sync.md) - subscribed broadcasts and guests play audio and video on the publisher's timestamps, with `moq play`'s delay
 - [Wayland display and window sources](/quest/m0/wayland-sources.md) - portal-picked sources that reopen without the picker
 - [Live encode control](/quest/m0/encode-control.md) - simulcast rungs that follow the bandwidth grant, live bitrate and resolution, keyframes on request

quest/m0/encode-control.md

@@ -26,4 +26,3 @@ track's share of the allocator.
 
 - [Simulcast rung disable](https://github.com/moq-dev/moq/blob/main/quest/m1/simulcast-rung-disable.md) - grant-driven rung disable this builds on
 - [moq-audio follows the grant](https://github.com/moq-dev/moq/blob/main/quest/m2/2848-follow-the-bandwidth-grant-in-moq-audio-instead-of.md) - the audio half
-- [Scene editing](/quest/m0/scene-editing.md) - canvas resize through a new rendition

quest/m0/multi-canvas.md

@@ -12,7 +12,9 @@ captures or decodes once however many canvases use it.
 Each canvas publishes its own broadcast, because different content needs a
 different name. Each canvas has its own top layer, which the collection's
 `top` holds for the one canvas today. Transitions are per canvas, so a scene change cuts every
-canvas together unless the creator unlinks one.
+canvas together unless the creator unlinks one. Reuse the one canvas's resize
+path per canvas: the compositor reallocates on a new canvas, and the program
+video publishes a rendition at a time, ending one and listing the next.
 
 Benchmark source count against canvas count, including sparse reuse. Per-canvas
 cost must grow only with the sources that canvas uses, never with the whole

quest/m0/program-drain.md

@@ -3,7 +3,8 @@
 ## Goal
 
 The program's encoder drains its buffered pictures before it stops at exit
-and before the studio publishes the gap marker of an on-demand pause, so a
+or when a new canvas ends its rendition, and before the studio publishes the
+gap marker of an on-demand pause, so a
 pipelined hardware encoder never loses the program's last frames.
 
 ## Plan

rs/moq-scene/src/collection.rs

@@ -298,7 +298,7 @@ impl Canvas {
 		moq_video::FrameRate::new(self.framerate, 1).expect("validated canvas framerate")
 	}
 
-	fn validate(&self) -> Result<(), Error> {
+	pub(crate) fn validate(&self) -> Result<(), Error> {
 		if self.width == 0 || self.height == 0 || !self.width.is_multiple_of(2) || !self.height.is_multiple_of(2) {
 			return Err(Error::Invalid(format!(
 				"canvas {}x{} must be non-zero and even",

rs/moq-scene/src/compositor.rs

@@ -1,7 +1,7 @@
 //! Draw scenes onto the canvas on the GPU: a preview, a program, and the fade
 //! between them.
 
-use std::collections::HashMap;
+use std::collections::{BTreeMap, HashMap};
 use std::time::Duration;
 
 use kio::clock::Instant;
@@ -90,15 +90,10 @@ impl effect::Library for Library<'_> {
 impl Compositor {
 	/// A compositor drawing `canvas`-sized pictures on `device`.
 	///
-	/// Fails if the device cannot hold a texture that large.
+	/// Fails if the canvas is invalid or the device cannot hold a texture that
+	/// large.
 	pub fn new(device: &wgpu::Device, queue: &wgpu::Queue, canvas: Canvas) -> Result<Self, Error> {
-		let limit = device.limits().max_texture_dimension_2d;
-		if canvas.width > limit || canvas.height > limit {
-			return Err(Error::Invalid(format!(
-				"canvas {}x{} exceeds this GPU's {limit}px texture limit",
-				canvas.width, canvas.height
-			)));
-		}
+		fits(device, canvas)?;
 
 		let effects = Effects::new(device, queue, canvas);
 		let target = |label| Target::new(device, &effects, canvas, label);
@@ -122,6 +117,26 @@ impl Compositor {
 		self.canvas
 	}
 
+	/// Draw `canvas` from the next [`render`](Self::render) on. A new size
+	/// replaces the [`preview`](Self::preview) and [`program`](Self::program)
+	/// textures, and every other texture sized from the canvas, with ones of
+	/// that size, blank until drawn; a fade under way carries on.
+	///
+	/// Fails, changing nothing, if the canvas is invalid or the device cannot
+	/// hold a texture that large.
+	pub fn set_canvas(&mut self, canvas: Canvas) -> Result<(), Error> {
+		fits(&self.device, canvas)?;
+		self.effects.set_canvas(canvas);
+		if canvas.size() != self.canvas.size() {
+			let target = |label| Target::new(&self.device, &self.effects, canvas, label);
+			self.preview = target("moq-scene preview");
+			self.program = target("moq-scene program");
+			self.incoming = target("moq-scene incoming");
+		}
+		self.canvas = canvas;
+		Ok(())
+	}
+
 	/// The scene in preview, as of the last [`render`](Self::render).
 	pub fn preview(&self) -> &wgpu::Texture {
 		&self.preview.texture
@@ -154,11 +169,51 @@ impl Compositor {
 		self.fade.is_some()
 	}
 
+	/// The scenes program draws: the one on program, and the one a fade under
+	/// way leaves.
+	pub fn on_air(&self) -> impl Iterator<Item = &str> {
+		let leaving = self.fade.as_ref().and_then(|fade| fade.from.as_deref());
+		self.program_scene.as_deref().into_iter().chain(leaving)
+	}
+
 	/// Put the scene called `scene` in preview, or nothing.
 	pub fn set_preview(&mut self, scene: Option<String>) {
 		self.preview_scene = scene;
 	}
 
+	/// Follow each renamed scene to its new name, wherever it is: in preview,
+	/// on program, or fading out, and drop each removed one, so a scene that
+	/// later takes its name does not go on program without a cut. `renames`
+	/// maps each old name to its new one, or `None`, as
+	/// [`Collection::renames`] reads them, and each scene is looked up once, so
+	/// a name one scene gave up and another took is not followed twice.
+	pub fn rename(&mut self, renames: &BTreeMap<String, Option<String>>) {
+		for scene in self.scenes() {
+			if let Some(to) = scene.as_ref().and_then(|name| renames.get(name)) {
+				*scene = to.clone();
+			}
+		}
+	}
+
+	/// Drop each scene `collection` does not have from preview, program, and a
+	/// fade under way, such as one a reloaded file left out, so a scene that
+	/// later takes its name does not go on program without a cut.
+	pub fn retain(&mut self, collection: &Collection) {
+		for scene in self.scenes() {
+			if scene.as_deref().is_some_and(|name| collection.scene(name).is_none()) {
+				*scene = None;
+			}
+		}
+	}
+
+	/// The scenes followed by name: in preview, on program, and fading out.
+	fn scenes(&mut self) -> impl Iterator<Item = &mut Option<String>> {
+		let fading = self.fade.as_mut().map(|fade| &mut fade.from);
+		[&mut self.preview_scene, &mut self.program_scene]
+			.into_iter()
+			.chain(fading)
+	}
+
 	/// Put the preview on program at once, ending any fade.
 	pub fn cut(&mut self) {
 		self.fade = None;
@@ -284,6 +339,20 @@ impl Compositor {
 	}
 }
 
+/// Refuse an invalid canvas, or one larger than `device` can hold as a
+/// texture.
+fn fits(device: &wgpu::Device, canvas: Canvas) -> Result<(), Error> {
+	canvas.validate()?;
+	let limit = device.limits().max_texture_dimension_2d;
+	if canvas.width > limit || canvas.height > limit {
+		return Err(Error::Invalid(format!(
+			"canvas {}x{} exceeds this GPU's {limit}px texture limit",
+			canvas.width, canvas.height
+		)));
+	}
+	Ok(())
+}
+
 /// One render's draws onto its targets.
 struct Draw<'a> {
 	collection: &'a Collection,
@@ -1158,6 +1227,110 @@ mod tests {
 		.await;
 	}
 
+	/// A resize draws the next frame on textures of the new size, with items
+	/// where their pixels say, a blend mode and a blur reaching the canvas's
+	/// new width, and a canvas the GPU cannot hold changes nothing.
+	#[tokio::test]
+	#[ignore = "needs a Vulkan device; run by just rs render-sw"]
+	async fn a_resize_draws_the_new_canvas() {
+		local(async {
+			let mut collection = collection();
+			collection.sources.insert(
+				"tint".into(),
+				Source::Color {
+					color: Rgba([0, 0, 255, 255]),
+				},
+			);
+			let mut studio = Studio::new(collection).await;
+			studio.compositor.set_preview(Some("red".into()));
+			studio.compositor.cut();
+			studio.render(Instant::now());
+			let old = studio.compositor.program().clone();
+
+			let wide = Canvas {
+				width: 16,
+				..studio.collection.canvas
+			};
+			studio.compositor.set_canvas(wide).unwrap();
+			assert_ne!(studio.compositor.program(), &old, "a new texture");
+			studio.collection.canvas = wide;
+			let fill = Rect::fill(&wide);
+			studio.collection.scenes[0].items = vec![
+				Item::new("red", Rect { width: 8.0, ..fill }),
+				Item {
+					blend: Blend::Screen,
+					filters: vec![Filter::Blur { radius: 1.0 }],
+					..Item::new("tint", fill)
+				},
+			];
+			studio.render(Instant::now());
+			let program = studio.program();
+			assert_eq!(program.len(), 16 * 8 * 4);
+			let pixel = |x: usize, y: usize| -> [u8; 4] {
+				let at = (y * 16 + x) * 4;
+				program[at..at + 4].try_into().unwrap()
+			};
+			assert!(
+				near(pixel(4, 4), [255, 0, 255, 255]),
+				"screened over red: {:?}",
+				pixel(4, 4)
+			);
+			assert!(
+				near(pixel(12, 4), BLUE),
+				"screened past the old width: {:?}",
+				pixel(12, 4)
+			);
+
+			let limit = studio.compositor.device.limits().max_texture_dimension_2d;
+			let huge = Canvas {
+				width: limit + 2,
+				..wide
+			};
+			assert!(matches!(studio.compositor.set_canvas(huge), Err(Error::Invalid(_))));
+			let empty = Canvas { width: 0, ..wide };
+			assert!(matches!(studio.compositor.set_canvas(empty), Err(Error::Invalid(_))));
+			assert_eq!(studio.compositor.canvas(), wide);
+		})
+		.await;
+	}
+
+	/// A renamed scene stays in preview, on program, and in the fade under way,
+	/// even when another scene takes its old name in the same edit, and a scene
+	/// the collection lost leaves them.
+	#[tokio::test]
+	#[ignore = "needs a Vulkan device; run by just rs render-sw"]
+	async fn a_renamed_scene_stays_where_it_is() {
+		local(async {
+			let mut studio = Studio::new(collection()).await;
+			let start = Instant::now();
+			studio.compositor.set_preview(Some("red".into()));
+			studio.compositor.cut();
+			studio.compositor.set_preview(Some("blue".into()));
+			studio.compositor.fade(Duration::from_secs(1), start);
+
+			// `blue` takes the name `red` gives up, which is followed once.
+			let mut draft = studio.collection.draft("Rename");
+			draft.rename_scene(0, "crimson").unwrap();
+			draft.rename_scene(1, "red").unwrap();
+			let edit = draft.finish().unwrap();
+			studio.compositor.rename(&studio.collection.renames(&edit));
+			studio.collection.apply(&edit).unwrap();
+			assert_eq!(studio.compositor.preview_scene(), Some("red"));
+			assert_eq!(studio.compositor.program_scene(), Some("red"));
+			studio.render(start + Duration::from_millis(500));
+			let actual = pixel(&studio.program(), 1, 1);
+			assert!(near(actual, [128, 0, 127, 255]), "still fading from red: {actual:?}");
+
+			// A scene the collection no longer has, as a reload can leave, is
+			// dropped, so one that later takes its name is not drawn unasked.
+			studio.collection.scenes.retain(|scene| scene.name != "red");
+			studio.compositor.retain(&studio.collection);
+			assert_eq!(studio.compositor.preview_scene(), None);
+			assert_eq!(studio.compositor.program_scene(), None);
+		})
+		.await;
+	}
+
 	/// A blend mode mixes an item with what is under it by the CSS formula,
 	/// weighted by the item's opacity.
 	#[tokio::test]