Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
39 changes: 31 additions & 8 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -94,11 +94,11 @@ Every panel is a tab you can drag, split, or stack.

| Panel | Use it to |
| --- | --- |
| Preview, Program | See the scene you are editing and the one that is live, and drag items in the preview to place them. With more than one canvas, pick the canvas they show, and that the Items panel edits, beside the preview. **Cut** puts the preview on program at once; **Transition** takes it there by the transition picked beside it, over the time set there (see [Transitions](#transitions)). **Interact** passes the mouse over the preview to the browser source under it instead, and the keyboard to the one last clicked. |
| Preview, Program | See the scene you are editing and the one that is live, and drag items in the preview to place them. With more than one canvas, pick the canvas they show, and that the Items panel edits, beside the preview. **Cut** puts the preview on program at once; **Transition** takes it there by the transition picked beside it, over the time set there, or for a stinger by the media source and cut point set there (see [Transitions](#transitions)). **Interact** passes the mouse over the preview to the browser source under it instead, and the keyboard to the one last clicked. |
| Scenes | Pick the scene in preview, and add, rename (double-click), reorder, or remove one. A scene on program, or still leaving it in a transition, cannot be removed, even by an undo, until program cuts away from it. |
| History | See every edit, and undo or redo back to any of them. |
| Items | Place, show, hide, and order what the preview scene or the top layer draws on the canvas shown, back to front. Select an item to filter or copy it. |
| Sources | Add a source by hand, see whether each and its own track are live and where it is shown, clone one, and remove one. Select a source to filter it. A camera or broadcast without a new picture, or a microphone or system audio without new sound, for a second reads as stalled. A browser source shows whether its page is copied on the GPU or, as a fallback, uploaded from memory (CPU), in place of a track, and **Reload** restarts its page. A media file reads as ended once it plays to its end without looping, **Play** plays it from its start, and **Reload** reads the file again. |
| Sources | Add a source by hand, see whether each and its own track are live and where it is shown, clone one, and remove one. Select a source to filter it. A camera or broadcast without a new picture, or a microphone or system audio without new sound, for a second reads as stalled. A browser source shows whether its page is copied on the GPU or, as a fallback, uploaded from memory (CPU), in place of a track, and **Reload** restarts its page. A media file reads as ended once it plays to its end without looping, **Play** plays it from its start, **Stop** stops it at once, blank and silent, until played again, and **Reload** reads the file again. |
| Devices | List this machine's cameras, displays, windows, and audio inputs, and add one as a source. |
| Guests | Invite a guest, show their join link as a QR code, see who joined, and take a guest out of every scene. |
| Filters | Crop the selected source or item, add, order, and set its filters, set an item's blend mode, and copy or paste filters. |
Expand Down Expand Up @@ -140,12 +140,28 @@ layer and every scene's layout on it.
| Slide | The incoming scene pushes the outgoing one off the canvas, both moving left, right, up, or down. |
| Wipe | An edge sweeps across the canvas left, right, up, or down, uncovering the incoming scene behind it. |
| Move | Each item showing a source both scenes place glides from its place in the outgoing scene to its place in the incoming one, resizing, recropping, and mixing its filters' settings on the way. If its blend mode or kinds of filter differ, it changes look halfway. The rest of the outgoing scene fades out and the rest of the incoming scene fades in. Items stack as the outgoing scene has them until halfway, then as the incoming one does. A source placed more than once pairs up in order, back to front. |
| Stinger | A media source's clip plays over the program from its start, and the program switches scenes under it at the cut point, the time into the clip when it covers the canvas. It lasts as long as the clip. |

All but a fade ease in and out. A transition is drawn on the program clock, so
the stream and a recording show the same frames of it, and a second
A slide, wipe, or move eases in and out. A transition is drawn on the program
clock, so the stream and a recording show the same frames of it, and a second
**Transition** while one runs starts from the scene it was heading to. A
transition set to 0 ms is a cut.

A stinger's clip fills the canvas, cropped evenly to the canvas's shape rather
than stretched, through its source's crop and filters, so a clip without an
alpha channel can be keyed. The cut point counts from the clip's first picture
or sound, however long the file takes to start decoding. Its sound joins the
mixes as its source's does. Its source waits blank and silent for the first
stinger rather than playing once when it opens, and rests ended between
stingers, which the Health panel does not count unless a scene on air places
it. A clip played to its end holds its last picture, as any media file does,
and a looping one stops, blank and silent, after one pass. A
clip that cannot play, or that is played again from the Sources panel or
reloaded while a stinger plays it, cuts at once; one stopped there ends the
stinger at once. A **Cut**, or another kind of transition, while a stinger
runs stops its clip, sound included; another stinger plays the clip again from
its start.

### Keyframes

An item's keyframes animate it: an item can fly in, fade in, or blur into
Expand Down Expand Up @@ -317,8 +333,11 @@ as ended in Sources and, while the program shows or plays it, in Health.
**Play** in Sources plays it from its start again, as does going on program
with `restart`, in a scene on air or the top layer, when it was not on program
before. Either clears its picture at once, so the end of the last play never
shows on program. A file that cannot be read or decoded fails, with the reason.
Its pictures and its sound each have a track of their own (see [Source
shows on program. **Stop** in Sources stops it at once: blank, silent, sound
already queued included, and reading as ended until played again. A file that
cannot be read or decoded fails, with the reason.
The source a [stinger](#transitions) plays waits, blank and silent, for the
first stinger instead of playing as it opens. Its pictures and its sound each have a track of their own (see [Source
tracks](#source-tracks)), its pictures on one timeline across plays. Each is
used apart, as a camera's and a microphone's are: the pictures while a scene on
air shows it, the sound while the mix on program plays it, and either while
Expand Down Expand Up @@ -445,9 +464,13 @@ height = 1920
top = [{ source = "logo", rect = { x = 780, y = 40, width = 280, height = 120 } }]

[transition]
kind = "slide" # fade, slide, wipe, or move
duration = "300ms"
kind = "slide" # fade, slide, wipe, move, or stinger
duration = "300ms" # all but a stinger
direction = "left" # slide and wipe: left, right, up, or down
# A stinger names a media source and its cut point instead:
# kind = "stinger"
# source = "sting"
# cut = "500ms"

[sources.camera]
kind = "camera"
Expand Down
1 change: 0 additions & 1 deletion quest/m0/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -91,7 +91,6 @@ repository.
- [Studio on macOS with capture granted](/quest/m0/app-macos.md) - camera, screen, and audio capture run on a Mac, not only Linux
- [Studio on Windows](/quest/m0/app-windows.md) - the studio runs on Windows, not only Linux
- [Wayland display and window sources](/quest/m0/wayland-sources.md) - portal-picked sources that reopen without the picker
- [Stinger transitions](/quest/m0/stinger.md) - a clip played over the program that switches scenes at its cut point
- [ML video effects](/quest/m0/effects.md) - background removal and other ML effects, baked into their source's track
- [Destinations](/quest/m0/destinations.md) - RTMP, RTMPS, and WHIP push with per-destination codecs and shared encoders, on a UI-free egress library a hosted service can reuse
- [Multiple canvases](/quest/m0/multi-canvas.md) - landscape and portrait canvases composite at once from shared sources
Expand Down
16 changes: 0 additions & 16 deletions quest/m0/stinger.md

This file was deleted.

93 changes: 81 additions & 12 deletions rs/moq-scene/src/collection.rs
Original file line number Diff line number Diff line change
Expand Up @@ -204,9 +204,9 @@ impl Collection {
}

/// Check what the types cannot: the canvas can be encoded, names are unique,
/// every item, clone, audio, and video entry names a source that exists,
/// and every crop and filter's settings are in range, each filter naming a
/// source of the kind it reads.
/// every item, clone, audio, and video entry names a source that exists, a
/// stinger names a media source, and every crop and filter's settings are
/// in range, each filter naming a source of the kind it reads.
pub fn validate(&self) -> Result<(), Error> {
self.canvas.validate()?;
for (name, extra) in &self.canvases {
Expand Down Expand Up @@ -323,6 +323,14 @@ impl Collection {
self.validate_chain(&video.filters, &on)?;
}

if let Some(name) = self.transition.stinger()
&& !matches!(self.sources.get(name), Some(Source::Media { .. }))
{
return Err(Error::Invalid(format!(
"the stinger plays `{name}`, which is not a media source"
)));
}

self.mixes.validate()?;
for (name, audio) in &self.audio {
if !self.sources.contains_key(name) {
Expand Down Expand Up @@ -487,9 +495,9 @@ impl Extra {

/// How the program changes when the preview is taken to it, short of a cut.
///
/// A fade runs at an even pace; the others ease in and out, so motion starts
/// and lands gently. One that takes no time is a cut.
#[derive(Clone, Copy, Debug, PartialEq, Eq, Serialize, Deserialize)]
/// A fade runs at an even pace; the slide, wipe, and move ease in and out, so
/// motion starts and lands gently. One that takes no time is a cut.
#[derive(Clone, Debug, PartialEq, Eq, Serialize, Deserialize)]
#[serde(tag = "kind", rename_all = "snake_case", deny_unknown_fields)]
pub enum Transition {
/// The incoming scene fades in over the outgoing one.
Expand Down Expand Up @@ -524,26 +532,49 @@ pub enum Transition {
#[serde(with = "humantime_serde")]
duration: Duration,
},
/// A media source's clip plays over the program from its start, and the
/// program switches scenes under it at `cut` into the clip, while it
/// covers the canvas. It lasts as long as the clip's first play, filling
/// each canvas it is drawn on, cropped to the canvas's shape, through the
/// source's own crop and filters.
Stinger {
/// The [`Source::Media`] whose clip plays.
source: String,
/// How far into the clip the scenes switch.
#[serde(with = "humantime_serde")]
cut: Duration,
},
}

impl Transition {
/// How long it takes.
pub fn duration(&self) -> Duration {
/// How long it takes, or `None` for a stinger, which lasts as long as its
/// clip.
pub fn duration(&self) -> Option<Duration> {
match *self {
Self::Fade { duration }
| Self::Slide { duration, .. }
| Self::Wipe { duration, .. }
| Self::Move { duration } => duration,
| Self::Move { duration } => Some(duration),
Self::Stinger { .. } => None,
}
}

/// How long it takes, to change.
pub fn duration_mut(&mut self) -> &mut Duration {
/// How long it takes, to change, or `None` for a stinger.
pub fn duration_mut(&mut self) -> Option<&mut Duration> {
match self {
Self::Fade { duration }
| Self::Slide { duration, .. }
| Self::Wipe { duration, .. }
| Self::Move { duration } => duration,
| Self::Move { duration } => Some(duration),
Self::Stinger { .. } => None,
}
}

/// The media source a stinger plays.
pub fn stinger(&self) -> Option<&str> {
match self {
Self::Stinger { source, .. } => Some(source),
_ => None,
}
}

Expand Down Expand Up @@ -1902,6 +1933,44 @@ rect = { x = 0, y = 680, width = 1280, height = 40 }
assert_eq!(collection.top[0].source, "bg");
}

/// A stinger names its media source and its cut point, and round-trips.
#[test]
fn reads_a_stinger() {
let text = r#"
version = 1

[canvas]
width = 1280
height = 720
framerate = 30

[transition]
kind = "stinger"
source = "sting"
cut = "800ms"

[sources.sting]
kind = "media"
path = "sting.webm"
"#;
let collection = Collection::from_toml(text).unwrap();
let stinger = Transition::Stinger {
source: "sting".into(),
cut: Duration::from_millis(800),
};
assert_eq!(collection.transition, stinger);
assert_eq!(collection.transition.stinger(), Some("sting"));
assert_eq!(collection.transition.duration(), None);
assert_eq!(
Collection::from_toml(&collection.to_toml().unwrap()).unwrap(),
collection
);

let missing = text.replace("source = \"sting\"", "source = \"nothing\"");
let err = Collection::from_toml(&missing).unwrap_err().to_string();
assert!(err.contains("`nothing`, which is not a media source"), "{err}");
}

#[test]
fn refuses_another_version() {
let text = example().to_toml().unwrap().replacen("version = 1", "version = 2", 1);
Expand Down
Loading