Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
25 commits
Select commit Hold shift + click to select a range
e29c4ce
chore(quest): claim quest/m0/replay.md
kixelated Nov 3, 2026
d3fe75e
Merge remote-tracking branch 'origin/main' into quest/m0/replay
kixelated Nov 3, 2026
49192cb
wip: replay takes and the replay source
kixelated Nov 3, 2026
30a1e02
wip: replay panel, program and source targets, clips
kixelated Nov 3, 2026
e0fffeb
feat(studio): instant replay and clips from the tracks' held groups
kixelated Nov 3, 2026
fb9a48c
fix(scene): a take copies what its tracks hold when taken
kixelated Nov 3, 2026
19268d2
Merge remote-tracking branch 'origin/main' into quest/m0/replay
kixelated Nov 3, 2026
7bd314b
fix(scene): a take copies only what is readable, without waiting
kixelated Nov 3, 2026
c1cd335
Merge remote-tracking branch 'origin/main' into quest/m0/replay
kixelated Nov 3, 2026
e3d03cc
test(scene): hold the paused clock while moq-video's threads work
kixelated Nov 3, 2026
4bd70a9
Merge remote-tracking branch 'origin/main' into quest/m0/replay
kixelated Nov 3, 2026
57c1f6b
Merge remote-tracking branch 'origin/main' into quest/m0/replay
kixelated Nov 3, 2026
49bcb76
test(scene): move the paused clock with a pending timer, not advance
kixelated Nov 3, 2026
f48e7e9
Merge remote-tracking branch 'origin/main' into quest/m0/replay
kixelated Nov 3, 2026
918393a
Merge remote-tracking branch 'origin/main' into iterate-replay
kixelated Nov 4, 2026
35f60f6
fix(replay): take without waiting, in group order, and mark replay gaps
kixelated Nov 4, 2026
9165739
fix(replay): release a finished take, and keep its sound to the end
kixelated Nov 4, 2026
6dc050c
Merge remote-tracking branch 'origin/main' into iterate-replay
kixelated Nov 4, 2026
bae57e0
style(replay): format the replay driver
kixelated Nov 4, 2026
ce1add6
Merge remote-tracking branch 'origin/main' into iterate-replay
kixelated Nov 4, 2026
db4f620
fix(replay): clear only a replay that is cut off, not an idle source
kixelated Nov 4, 2026
5b04b13
fix(replay): a clip shows nothing after its end
kixelated Nov 4, 2026
d473f25
Merge remote-tracking branch 'origin/main' into iterate-replay
kixelated Nov 4, 2026
8f74528
docs(replay): state how early a clip with B-frames can end
kixelated Nov 4, 2026
9974622
Merge remote-tracking branch 'origin/main' into iterate-replay
kixelated Nov 4, 2026
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions Cargo.lock

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

38 changes: 35 additions & 3 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -96,6 +96,7 @@ Every panel is a tab you can drag, split, or stack.
| Audio | Set each audio source's gain and mute, and watch its meter and the program's. |
| Output | Set the canvas size and frame rate, and see the broadcast path, whether the program is encoding, and the encoder's work. |
| Health | See how the stream reaches viewers, relay by relay and track by track, and the cause of each problem. |
| Replay | Replay the last moments of the program or a source in a replay source, slowed down if you like, or save them as a clip. |

A new source, from the Sources or Devices panel, is placed over whatever the
Items panel shows. The **top layer** is drawn over every scene, in preview and
Expand Down Expand Up @@ -264,10 +265,10 @@ rect = { x = 1600, y = 40, width = 280, height = 120 }
Source kinds are `camera`, `display`, and `microphone` (each with an optional
`id`, the default device without one), `window` (with an `id`), `system_audio`
(macOS), `image` (PNG or JPEG), `color` (`#rrggbb` or `#rrggbbaa`),
`broadcast`, `clone` (with the `source` it shows), and `lut` (a 3D Adobe or
`broadcast`, `clone` (with the `source` it shows), `lut` (a 3D Adobe or
Resolve `.cube` file, which no item places, but which grades what a `lut`
filter names it in). The Devices panel fills in ids. `[[top]]` items make up the top
layer.
filter names it in), and `replay`, where the Replay panel plays replays. The
Devices panel fills in ids. `[[top]]` items make up the top layer.

`[video.<source>]` sets a source's `crop` and `filters`, wherever it is shown,
and an item's `crop`, `filters`, and `blend` its own. A `crop` trims a fraction
Expand Down Expand Up @@ -403,6 +404,37 @@ macOS. Every other source, every source on Windows or on a GPU that cannot
import, and every still image delivers its frames from the CPU, which the
compositor uploads.

## Replay

Add a `replay` source and place it like any other, such as full screen in a
scene of its own. In the Replay panel, pick the program or a source, a length
up to 30 seconds, and a speed of 100%, 50%, or 25%, then **Replay** plays that
many seconds up to the click in the replay source, which is transparent before
and after. **Cut** to its scene to put it on program. Sound replays only at
100%, through the replay source's strip in the Audio panel. A new replay cuts
off the one playing, and a stretch the source was offline or paused replays
as the gap it was.

**Save clip** writes the same seconds, in real time, beside the scene file as a
regular MP4: `show-clip-1.mp4` for `show.toml`, then `show-clip-2.mp4`, and so
on. Nothing is re-encoded, so a clip starts at the keyframe before those
seconds, up to one keyframe interval earlier. It shows nothing after them, so
video with B-frames can end early: the pictures shown before the end that refer
to one shown after it are left out, along with everything decoded after that
one. That is usually a fraction of a second, but it can reach back as far as
the keyframe for a slow or deeply reordered source. The panel says where once
it is written, shortly after the click.

Nothing records in the background for this. The program and each source's
track keep their last 30 seconds for late viewers anyway, and a replay copies
from those, so a replay needs `--connect` and goes back at most that far. With
`--on-demand`, the program has no past from before someone watched it. A
broadcast source replays from what the studio received of it, and only over a
relay connection whose protocol version can address a track by time; an older
one refuses with the reason in the panel. An image or a color has no past to
replay. A replay source publishes each replay as its own track while it plays,
like a camera, so viewers composing the scene themselves see it too.

## Health

The Health panel grades the stream every second on the thresholds moq's
Expand Down
1 change: 0 additions & 1 deletion quest/m0/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -87,7 +87,6 @@ repository.
- [Wayland display and window sources](/quest/m0/wayland-sources.md) - portal-picked sources that reopen without the picker
- [Live encode control](/quest/m0/encode-control.md) - simulcast rungs that follow the bandwidth grant, live bitrate and resolution, keyframes on request
- [Audio sources report stalls](/quest/m0/audio-stall.md) - a dead microphone or system audio reads stalled like a camera
- [Instant replay and DVR](/quest/m0/replay.md) - replays and clips from held groups, and viewer scrubbing through the archive
- [HTML/CSS overlays](/quest/m0/overlays.md) - script-free HTML and CSS graphics through Blitz natively and the DOM in the browser
- [Animation and transitions](/quest/m0/animation.md) - move, slide, wipe, and stinger transitions, and keyframed item properties
- [ML video effects](/quest/m0/effects.md) - background removal and other ML effects, baked into their source's track
Expand Down
2 changes: 1 addition & 1 deletion quest/m0/compose/sources.md
Original file line number Diff line number Diff line change
Expand Up @@ -28,7 +28,7 @@ in its ML effects, and a composing viewer mixes the levels itself.

Encode like every first-party publisher (decided 2026-10-28, from
[#7101](https://github.com/moq-dev/moq/issues/7101)): an unwatched source
still encodes, so a late viewer or a [replay](/quest/m0/replay.md)
still encodes, so a late viewer or a replay
finds its recent past, but costs no upload.

✅ 2026-11-01 (maintainer, on
Expand Down
34 changes: 0 additions & 34 deletions quest/m0/replay.md

This file was deleted.

4 changes: 2 additions & 2 deletions rs/moq-scene/Cargo.toml
Original file line number Diff line number Diff line change
Expand Up @@ -44,12 +44,12 @@ moq-video = { workspace = true, features = ["render"] }
serde = { workspace = true }
serde_with = { workspace = true }
thiserror = { workspace = true }
tokio = { workspace = true, features = ["macros", "time"] }
tokio = { workspace = true, features = ["macros", "sync", "time"] }
toml = { workspace = true }
tracing = { workspace = true }

[dev-dependencies]
# An encoder on every machine, for the source track tests.
# An encoder on every machine, for the source track and replay tests.
moq-video = { workspace = true, features = ["openh264"] }
tempfile = { workspace = true }
tokio = { workspace = true, features = ["macros", "rt", "test-util"] }
Expand Down
6 changes: 5 additions & 1 deletion rs/moq-scene/src/collection.rs
Original file line number Diff line number Diff line change
Expand Up @@ -839,6 +839,10 @@ pub enum Source {
/// The file, relative to the scene file's directory unless absolute.
path: PathBuf,
},
/// Where instant replays play: the recent past of the program or another
/// source, on demand through [`Feed::replay`](crate::source::Feed::replay).
/// Transparent while nothing plays.
Replay,
}

impl Source {
Expand Down Expand Up @@ -873,7 +877,7 @@ impl Source {
pub fn has_audio(&self) -> bool {
matches!(
self,
Self::Microphone { .. } | Self::SystemAudio | Self::Broadcast { .. }
Self::Microphone { .. } | Self::SystemAudio | Self::Broadcast { .. } | Self::Replay
)
}
}
Expand Down
5 changes: 5 additions & 0 deletions rs/moq-scene/src/lib.rs
Original file line number Diff line number Diff line change
Expand Up @@ -33,6 +33,7 @@ mod edit;
mod effect;
mod lut;
pub mod publish;
pub mod replay;
pub mod source;

pub use clip::Clip;
Expand Down Expand Up @@ -85,6 +86,10 @@ pub enum Error {
#[error("unsupported source: {0}")]
Unsupported(String),

/// The source stopped, with the reason, so it takes nothing more.
#[error("the source stopped: {0}")]
Stopped(String),

/// Capturing, decoding, or drawing video failed.
#[error(transparent)]
Video(#[from] moq_video::Error),
Expand Down
85 changes: 71 additions & 14 deletions rs/moq-scene/src/publish.rs
Original file line number Diff line number Diff line change
Expand Up @@ -218,7 +218,7 @@ impl Drop for Listing<'_> {
}
}

/// A captured source's video, encoded frame by frame as it is shown.
/// A captured or replayed source's video, encoded frame by frame as it is shown.
pub(crate) struct Video {
/// Taken only to abort or finish it, as the track ends.
producer: Option<moq_video::encode::Producer>,
Expand All @@ -237,10 +237,6 @@ impl Video {
/// to learn exactly what the real one will write, so a machine that cannot
/// encode another stream, such as a GPU at its hardware session cap, fails
/// here with the reason.
#[cfg_attr(
not(feature = "capture"),
allow(dead_code, reason = "only capture sources encode video")
)]
pub(crate) async fn new(
listing: &Listing<'_>,
size: moq_video::Size,
Expand All @@ -267,10 +263,6 @@ impl Video {
}

/// Encode and publish `frame`, restamping it onto the studio clock.
#[cfg_attr(
not(feature = "capture"),
allow(dead_code, reason = "only capture sources encode video")
)]
pub(crate) fn encode(&mut self, frame: &mut moq_video::Frame) -> Result<(), Error> {
let captured = frame.timestamp.as_micros();
let epoch = *self
Expand Down Expand Up @@ -299,6 +291,15 @@ impl Video {
Ok(())
}

/// Mark a break, such as a gap in a replay: what follows does not continue
/// what came before, and a viewer arriving during it waits for what follows
/// rather than being served the last picture as if it were live.
pub(crate) fn discontinuity(&mut self) -> Result<(), Error> {
self.drain()?;
self.producer.as_mut().expect("open until it ends").discontinuity()?;
Ok(())
}

/// Publish what the encoder still holds and close it, so a pipelined
/// backend's last pictures are not lost when the encoder is replaced or the
/// track ends.
Expand All @@ -310,10 +311,6 @@ impl Video {
}

/// End the track, failed with `err` so viewers see why.
#[cfg_attr(
not(feature = "capture"),
allow(dead_code, reason = "only capture sources encode video")
)]
pub(crate) fn abort(mut self, err: &Error) {
if let Some(producer) = self.producer.take() {
producer.abort(moq_net::Error::Transport(err.to_string()));
Expand Down Expand Up @@ -392,7 +389,7 @@ mod tests {

use super::*;
use crate::collection::{Collection, Rgba, Source};
use crate::source::test::{context, local, start};
use crate::source::test::{context, local, start, until, until_time, video};
use crate::source::{Sources, Status};

/// A studio broadcast and catalog to publish into, and a reader of it.
Expand Down Expand Up @@ -617,4 +614,64 @@ mod tests {
})
.await;
}

/// A replay source publishes each replay as it shows, so viewers composing
/// the scene themselves see it too, gaps included.
#[tokio::test(start_paused = true)]
async fn a_replay_is_published_as_it_plays() {
local(async {
let dir = tempfile::tempdir().unwrap();
let (publish, catalog, consumer) = studio();
let (mut context, _origin) = context(dir.path());
context.publish = Some(publish);
let mut collection = Collection::new();
collection.sources.insert("replay".into(), Source::Replay);
let mut sources = Sources::new(context);
let drivers = sources.sync(&collection);
start(&sources, drivers).await;

// The broadcast left out its second second, a gap in the span.
let (broadcast, video, _live) = video(Some(1)).await;
let target = crate::replay::Target {
broadcast: broadcast.consume(),
video: Some(video),
audio: None,
end: moq_net::Timestamp::from_micros(2_500_000).unwrap(),
};
let take = crate::replay::Take::new(&target, std::time::Duration::from_millis(2_500)).unwrap();
let feed = sources.get("replay").unwrap();
let begin = kio::clock::now();
feed.replay(take, 1.0).unwrap();
// The first picture is due at once, so the clock stays put while
// the encoder opens.
let listed = until(|| {
let listed = catalog.snapshot().ext.studio.sources.remove("replay");
listed.filter(|listed| !listed.video.renditions.is_empty())
})
.await;
assert_eq!(feed.track(), Some(Status::Live));

// Into the gap, before the replay ends, a viewer reads the break.
let (name, config) = listed.video.renditions.iter().next().unwrap();
let track = consumer
.track(hang::catalog::Reference::track(config, name))
.unwrap()
.subscribe(moq_net::track::Subscription::default().with_max_delay(None))
.await
.unwrap();
let container = moq_mux::catalog::hang::Container::try_from(config).unwrap();
let mut track = moq_mux::container::Consumer::new(track, container);
until_time(begin + std::time::Duration::from_millis(1_500)).await;
let now = kio::Waiter::noop();
until(|| match track.poll_read(&now) {
std::task::Poll::Ready(read) => match read.unwrap().expect("open while the replay plays") {
moq_mux::container::Read::End { .. } | moq_mux::container::Read::Gap { .. } => Some(()),
_ => None,
},
std::task::Poll::Pending => None,
})
.await;
})
.await;
}
}
Loading