feat(scene): HTML and CSS overlays drawn by Blitz on the program clock

README.md

@@ -1,8 +1,8 @@
 # MoQ Studio
 
 `moq-studio` is a native streaming studio for Linux, macOS, and Windows. It
-composes cameras, screens, windows, images, colors, web pages, and broadcasts
-from a relay into scenes on the GPU, mixes their audio, and publishes the result to a
+composes cameras, screens, windows, images, colors, web pages, HTML and CSS
+overlays, and broadcasts from a relay into scenes on the GPU, mixes their audio, and publishes the result to a
 [MoQ](https://github.com/moq-dev/moq) relay. It is early and not packaged yet,
 so build it from source. The moq crates come from the `moq` submodule:
 
@@ -154,6 +154,38 @@ stream carries on. A crash of the browser itself fails every browser source.
 windows (`window.open` and new-tab links do nothing), and its dropdown lists,
 such as a `<select>`'s, are not drawn yet.
 
+An overlay is a title, lower third, or other graphic written as HTML and CSS,
+drawn by the studio itself (through [Blitz](https://github.com/DioxusLabs/blitz))
+rather than a browser, on every platform. Its CSS animations and transitions
+run on the program clock, so they land on the same frame in the program and in
+a recording or replay of it, whatever the studio's load. It runs no scripts
+and loads nothing: no images, stylesheets, or fonts from anywhere. Its text is
+set in the bundled Noto Sans, regular, bold, or italic, whatever `font-family`
+names, so it looks the same on every machine. An overlay outside what
+overlays draw fails to start, naming the first element, attribute, rule, or
+property it refused:
+
+- Elements: text and the boxes around it (`div`, `span`, `p`, headings,
+  lists, tables, and the like), plus `style`, `title`, and `meta`. Not
+  `script`, `img`, `link`, `iframe`, `svg`, `canvas`, or form controls.
+- Attributes: `id`, `class`, `style`, `lang`, `dir`, `title`, `hidden`,
+  table and list numbering ones, and `data-*` and `aria-*`.
+- CSS rules: style rules, nested ones included, `@media`, and `@keyframes`.
+  Not `@import` or `@font-face`, nor any `url()` or `image-set()`.
+- CSS properties: box layout, flexbox, grid, positioning, margins and
+  padding, colors, opacity, gradients, borders and their radii, `box-shadow`,
+  outlines, transforms, `clip-path`, fonts and text layout, text decoration,
+  `content`, and animations and transitions, by their usual shorthands too.
+  Not yet `filter`, `backdrop-filter`, `mix-blend-mode`, `text-shadow`,
+  `mask`, `text-overflow`, or multi-column layout; the list grows as Blitz
+  draws more.
+
+An overlay's animation starts when the overlay does: when the studio opens
+it, and again whenever it is edited. One that no preview or program shows
+costs nothing until it is shown again, where its animation has carried on.
+Overlays publish no track of their own, so viewers see them only in the
+program.
+
 ## Guests
 
 Guests and phones join a room the studio owns on its relay, from a web page,
@@ -271,6 +303,20 @@ url = "https://example.com/alerts"
 width = 800 # the page's own size, whatever its items stretch it to
 height = 600
 
+[sources.name]
+kind = "overlay"
+width = 1920 # the overlay's own size, like a page's
+height = 1080
+html = '''
+<style>
+  .third { position: absolute; left: 80px; bottom: 120px; padding: 16px 32px;
+    background: linear-gradient(90deg, #1e3a8a, #2563eb); border-radius: 12px;
+    font: bold 48px sans-serif; color: white; animation: in 400ms ease-out both; }
+  @keyframes in { from { transform: translateX(-40px); opacity: 0 } }
+</style>
+<div class="third">Ada Lovelace</div>
+'''
+
 [audio.guest]
 gain = 0.5
 
@@ -315,7 +361,7 @@ Source kinds are `camera`, `display`, and `microphone` (each with an optional
 `id`, the default device without one), `window` (with an `id`), `system_audio`
 (macOS), `image` (PNG or JPEG), `color` (`#rrggbb` or `#rrggbbaa`),
 `broadcast`, `browser` (a `url`, and a `width` and `height` up to 4096),
-`clone` (with the `source` it shows), `lut` (a 3D Adobe or Resolve `.cube`
+`overlay` (its `html`, and a `width` and `height` up to 4096), `clone` (with the `source` it shows), `lut` (a 3D Adobe or Resolve `.cube`
 file, which no item places, but which grades what a `lut` filter names it in),
 and `replay`, where the Replay panel plays replays. The Devices panel fills in
 ids. `[[top]]` items make up the top layer.
@@ -435,7 +481,7 @@ colors publish once, as the image file and as a one-pixel PNG. Removing or
 editing a source ends its track. A track carries the plain capture, before any
 filter, and a clone publishes none, since it shows its source's capture.
 Microphones and system audio do not publish tracks of their own yet. A browser
-source publishes none, so viewers see its page only in the program, and a
+source or overlay publishes none, so viewers see it only in the program, and a
 broadcast source is already its publisher's.
 
 Every captured source opens an encode session as it starts, and consumer GPUs

quest/m0/README.md

@@ -94,7 +94,6 @@ repository.
 - [Broadcast sources play in sync](/quest/m0/broadcast-sync.md) - subscribed broadcasts and guests play audio and video on the publisher's timestamps, with `moq play`'s delay
 - [Wayland display and window sources](/quest/m0/wayland-sources.md) - portal-picked sources that reopen without the picker
 - [Live encode control](/quest/m0/encode-control.md) - simulcast rungs that follow the bandwidth grant, live bitrate and resolution, keyframes on request
-- [HTML/CSS overlays](/quest/m0/overlays.md) - script-free HTML and CSS graphics through Blitz natively and the DOM in the browser
 - [Animation and transitions](/quest/m0/animation.md) - move, slide, wipe, and stinger transitions, and keyframed item properties
 - [ML video effects](/quest/m0/effects.md) - background removal and other ML effects, baked into their source's track
 - [Destinations](/quest/m0/destinations.md) - RTMP, RTMPS, and WHIP push with per-destination codecs and shared encoders, on a UI-free egress library a hosted service can reuse

rs/moq-scene/Cargo.toml

@@ -32,8 +32,23 @@ pipewire = ["capture", "moq-video/pipewire", "moq-audio/pipewire"]
 cef = ["dep:cef"]
 
 [dependencies]
+# Overlays: Blitz lays HTML and CSS out and paints it into a Vello scene, which
+# Vello renders on the compositor's device. Pinned to the beta whose Vello and
+# wgpu match moq-video's: the stable 0.2 line is on wgpu 26, which cannot share
+# the device. Fonts and network access are left off, so an overlay loads nothing.
+anyrender_vello = "0.15"
+blitz-dom = { version = "=0.3.0-beta.4", default-features = false }
+blitz-html = "=0.3.0-beta.4"
+blitz-paint = { version = "=0.3.0-beta.4", default-features = false }
+blitz-traits = "=0.3.0-beta.4"
 bytemuck = { version = "1", features = ["derive"] }
 bytes = { workspace = true }
+# Reads the declarations an overlay's CSS checks, as Stylo serializes them.
+cssparser = "0.38"
+# The faces every overlay's text is set in: Noto Sans, regular, bold, and italic.
+notosans = "0.1"
+# Blitz's text layout, to register those faces as every family's.
+parley = { version = "0.12", default-features = false, features = ["std"] }
 # The catalog's rendition configs, which a broadcast source follows.
 hang = { workspace = true }
 humantime-serde = { workspace = true }
@@ -48,10 +63,14 @@ moq-net = { workspace = true }
 moq-video = { workspace = true, features = ["render"] }
 serde = { workspace = true }
 serde_with = { workspace = true }
+# Stylo, Blitz's style engine, which parses an overlay's CSS to check it.
+style = { package = "stylo", version = "0.22" }
 thiserror = { workspace = true }
 tokio = { workspace = true, features = ["macros", "sync", "time"] }
 toml = { workspace = true }
 tracing = { workspace = true }
+url = { workspace = true }
+vello = "0.11"
 
 # The helper process a browser source runs CEF in, and the socket between them.
 [target.'cfg(target_os = "linux")'.dependencies]

rs/moq-scene/src/collection.rs

@@ -847,6 +847,20 @@ pub enum Source {
 		/// The page's viewport height in pixels.
 		height: u32,
 	},
+	/// A graphic written as HTML and CSS without scripts, such as a title or
+	/// lower third. Its CSS animations run on the program clock, so they land
+	/// on the same frame in the program and everything made from it. It loads
+	/// nothing, and HTML or CSS outside what overlays draw fails the source,
+	/// naming it.
+	Overlay {
+		/// The document, its CSS in `<style>` elements and `style` attributes.
+		html: String,
+		/// The overlay's viewport width in pixels, independent of where it is
+		/// placed.
+		width: u32,
+		/// The overlay's viewport height in pixels.
+		height: u32,
+	},
 	/// Another source's picture, captured once, with [`Collection::video`]
 	/// filters of its own. Its sound is not mixed a second time.
 	Clone {
@@ -902,16 +916,21 @@ impl Source {
 	}
 
 	fn validate(&self, name: &str) -> Result<(), Error> {
-		if let Self::Browser { url, width, height } = self {
-			if url.trim().is_empty() {
-				return Err(Error::Invalid(format!("browser source `{name}` has no url")));
-			}
-			let max = crate::browser::MAX_SIDE;
-			if *width == 0 || *height == 0 || *width > max || *height > max {
-				return Err(Error::Invalid(format!(
-					"browser source `{name}` is {width}x{height}; each side must be 1 to {max} pixels"
-				)));
+		let (kind, width, height) = match self {
+			Self::Browser { url, width, height } => {
+				if url.trim().is_empty() {
+					return Err(Error::Invalid(format!("browser source `{name}` has no url")));
+				}
+				("browser", width, height)
 			}
+			Self::Overlay { width, height, .. } => ("overlay", width, height),
+			_ => return Ok(()),
+		};
+		let max = crate::browser::MAX_SIDE;
+		if *width == 0 || *height == 0 || *width > max || *height > max {
+			return Err(Error::Invalid(format!(
+				"{kind} source `{name}` is {width}x{height}; each side must be 1 to {max} pixels"
+			)));
 		}
 		Ok(())
 	}