Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
45 changes: 42 additions & 3 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -13,8 +13,9 @@ cargo run --release -p moq-studio --features capture,openh264 -- \
show.toml --connect https://cdn.moq.dev/anon --broadcast my-show
```

- `show.toml` is the scene collection. It is created if it does not exist, and
changes are saved to it a second after you stop editing, with no save step.
- `show.toml` is the scene collection. If it does not exist yet,
[guided setup](#first-launch) proposes one and creates it. Changes are saved
to it a second after you stop editing, with no save step.
- Without `--connect`, the studio composes and previews but publishes nothing,
and broadcast sources stay offline.
- `capture` adds camera, display, window, microphone, and system-audio sources.
Expand All @@ -30,6 +31,44 @@ cargo run --release -p moq-studio --features capture,openh264 -- \
- The `--connect-*`, `--quic-*`, and `--log-level` flags are
[moq-cli's](https://doc.moq.dev/bin/cli); see `moq-studio --help`.

## First launch

Opening a scene file that does not exist yet starts guided setup instead of
the studio. It runs three checks, and a check that fails shows its reason:

- **Devices:** the cameras, displays, and audio inputs this machine has.
- **Encoders:** for each canvas setup can propose (1080p and 720p at 60 or 30
fps, and 360p at 30 fps), the encoder the program would pick, whether it
reads the GPU, the bitrate it encodes at, and how long a frame takes over a
short burst of moving frames. An encoder keeps up when a frame takes at most
three quarters of the frame period, leaving the rest for sources and the
compositor.
- **Uplink:** with `--connect`, setup publishes a probe at the `--broadcast`
path for about six seconds and subscribes to it through the relay, so the
relay pulls it. The congestion controller's send-rate estimate is the
measurement. A connection that reports no estimate, such as WebSocket, fails
the check.

Setup then proposes a scene named Main and a canvas, explaining each choice.
The scene has the first display filling the canvas, the first camera in the
bottom-right corner over it (or filling the canvas alone), and the first audio
input. Swap any of them for another device or leave it out. The canvas is the
largest one whose encoder keeps up and whose video and audio fit in two thirds
of the measured uplink, the most the Health panel grades healthy. Without a
measurement, it is the default 1080p at 30 fps. If no encoder keeps up, it is
the canvas closest to keeping up.

Nothing is written until you start the studio, so quitting during setup runs
it again on the next launch. **Start with this scene** and **Start with an
empty scene** both create the scene file at once, the second with the chosen
canvas and no sources. The proposed scene is the first edit in the history, so
undoing it leaves that empty scene.

Setup writes an ordinary scene file. Change its sources and scenes in their
panels, and its canvas in the file, which takes a restart. The Output panel
shows the canvas in use, and the Health panel shows how the uplink holds up
once you are live.

## Panels

Every panel is a tab you can drag, split, or stack.
Expand All @@ -44,7 +83,7 @@ Every panel is a tab you can drag, split, or stack.
| Devices | List this machine's cameras, displays, windows, and audio inputs, and add one as a source. |
| Filters | Add, order, and set the filters of the selected source or item, and copy or paste them. |
| Audio | Set each audio source's gain and mute, and watch its meter and the program's. |
| Output | See the broadcast path, whether the program is encoding, and the encoder's work. |
| Output | See the broadcast path, whether the program is encoding, the canvas, and the encoder's work. |
| Health | See how the stream reaches viewers, track by track, and the cause of each problem. |

A new source, from the Sources or Devices panel, is placed over whatever the
Expand Down
2 changes: 1 addition & 1 deletion justfile
Original file line number Diff line number Diff line change
Expand Up @@ -48,7 +48,7 @@ render-sw:
icd=("$mesa"/share/vulkan/icd.d/lvp_icd.*.json)
export LD_LIBRARY_PATH="$loader/lib" VK_DRIVER_FILES="${icd[0]}"
cargo nextest run --locked -p moq-scene --lib --run-ignored only -E 'test(/^compositor::tests::/)'
cargo nextest run --locked -p moq-studio --features openh264 --run-ignored only -E 'test(=engine::tests::the_program_reaches_a_relay)'
cargo nextest run --locked -p moq-studio --features openh264 --run-ignored only -E 'test(=engine::tests::the_program_reaches_a_relay) | test(=setup::tests::times_every_canvas)'

# Move the moq submodule to moq's latest main (or a given commit) and refresh Cargo.lock.
upgrade-moq rev="origin/main":
Expand Down
1 change: 0 additions & 1 deletion quest/m0/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -96,7 +96,6 @@ repository.
- [Animation and transitions](/quest/m0/animation.md) - move, slide, wipe, and stinger transitions, and keyframed item properties
- [Video effects](/quest/m0/effects.md) - a curated effect set with CSS meanings, and ML effects baked into their source
- [Snapping and guides](/quest/m0/snapping.md) - preview moves snap to edges, centers, and other items
- [Guided setup](/quest/m0/guided-setup.md) - first launch detects devices, measures the uplink, and proposes settings
- [Destinations](/quest/m0/destinations.md) - RTMP, RTMPS, and WHIP push with per-destination codecs and shared encoders, on a UI-free egress library a hosted service can reuse
- [Multiple canvases](/quest/m0/multi-canvas.md) - landscape and portrait canvases composite at once from shared sources
- [Guests](/quest/m0/guests.md) - remote guests and phone cameras join through a moq-room link or QR code and appear as sources
Expand Down
19 changes: 0 additions & 19 deletions quest/m0/guided-setup.md

This file was deleted.

52 changes: 47 additions & 5 deletions rs/moq-studio/src/autosave.rs
Original file line number Diff line number Diff line change
Expand Up @@ -118,7 +118,8 @@ struct Written {

impl Autosave {
/// Read the scene file at `path` and its history, or start a new
/// collection if there is no scene file yet; the first save creates it.
/// collection if there is no scene file yet; the first save creates it, and
/// fails rather than replace one another program created meanwhile.
pub fn open(path: PathBuf) -> anyhow::Result<(Self, Loaded)> {
let (collection, written) = match std::fs::read_to_string(&path) {
Ok(text) => {
Expand Down Expand Up @@ -151,6 +152,12 @@ impl Autosave {
&self.path
}

/// Whether the scene file exists: it was read when opened, or a save has
/// created it since.
pub fn exists(&self) -> bool {
self.written.lock().unwrap().collection.is_some()
}

/// Where the studio records its edits to the collection.
pub fn edits(&self) -> Edits {
self.edits.clone()
Expand All @@ -171,11 +178,13 @@ impl Autosave {
self.edits.0.lock().take();
let result = serialize().and_then(|text| {
if written.collection.as_ref() != Some(&text.collection) {
write(&self.path, &text.collection).with_context(|| format!("writing {}", self.path.display()))?;
let new = written.collection.is_none();
write(&self.path, &text.collection, new).with_context(|| format!("writing {}", self.path.display()))?;
written.collection = Some(text.collection);
}
if written.history != text.history {
write(&self.history, &text.history).with_context(|| format!("writing {}", self.history.display()))?;
write(&self.history, &text.history, false)
.with_context(|| format!("writing {}", self.history.display()))?;
written.history = text.history;
}
Ok(())
Expand Down Expand Up @@ -263,9 +272,13 @@ fn load_history(path: &Path, collection: Collection) -> anyhow::Result<Loaded> {

/// Replace the file at `path` with `text` all at once, so a crash leaves
/// either the old file or the new one, never part of either.
fn write(path: &Path, text: &str) -> std::io::Result<()> {
///
/// A `new` file must not exist yet: one another program created since the
/// studio found none is left alone rather than replaced.
fn write(path: &Path, text: &str, new: bool) -> std::io::Result<()> {
let path = resolve(path)?;
let existing = match std::fs::metadata(&path) {
Ok(_) if new => return Err(created_since()),
Ok(metadata) => {
// Refused as a write in place would be, so a read-only scene file
// is not replaced.
Expand Down Expand Up @@ -302,13 +315,27 @@ fn write(path: &Path, text: &str) -> std::io::Result<()> {
// On disk before the rename, so a power loss cannot leave the new name
// pointing at an empty file.
file.as_file().sync_all()?;
file.persist(&path)?;
match new {
// Fails if the file appeared since the check above.
true => file.persist_noclobber(&path).map_err(|err| match err.error.kind() {
std::io::ErrorKind::AlreadyExists => created_since(),
_ => err.error,
})?,
false => file.persist(&path)?,
};
// And the rename itself, so a power loss cannot undo the save.
#[cfg(unix)]
std::fs::File::open(dir)?.sync_all()?;
Ok(())
}

fn created_since() -> std::io::Error {
std::io::Error::new(
std::io::ErrorKind::AlreadyExists,
"another program created it since the studio started; restart the studio to open it",
)
}

/// The file a chain of symlinks at `path` ends in, which need not exist yet,
/// so a save replaces that file and every link survives.
fn resolve(path: &Path) -> std::io::Result<PathBuf> {
Expand Down Expand Up @@ -391,6 +418,21 @@ mod tests {
);
}

/// A scene file another program creates after the studio found none, such
/// as a second studio finishing setup first, is never replaced.
#[test]
fn leaves_a_file_created_since_it_found_none() {
let dir = tempfile::tempdir().unwrap();
let path = dir.path().join("show.toml");
let (autosave, Loaded { collection: opened, .. }) = Autosave::open(path.clone()).unwrap();

std::fs::write(&path, "elsewhere").unwrap();
let err = format!("{:#}", autosave.save(serialize(&opened)).unwrap_err());
assert!(err.contains("restart the studio"), "{err}");
assert_eq!(std::fs::read_to_string(&path).unwrap(), "elsewhere");
assert!(autosave.error().is_some(), "shown until a restart");
}

/// A new file gets the permissions any new file would, not a temporary
/// file's owner-only ones.
#[cfg(unix)]
Expand Down
83 changes: 83 additions & 0 deletions rs/moq-studio/src/devices.rs
Original file line number Diff line number Diff line change
@@ -0,0 +1,83 @@
//! The capture devices this machine has, for the Devices panel and setup.

use moq_scene::Source;

/// A device the user can add as a source.
#[derive(Clone, Debug)]
pub struct Pick {
pub name: String,
pub source: Source,
}

/// One listing of every kind of device, each of which can fail on its own.
pub struct Listing {
pub cameras: Result<Vec<Pick>, String>,
pub displays: Result<Vec<Pick>, String>,
pub windows: Result<Vec<Pick>, String>,
/// Microphones and other inputs, plus system audio where it can be captured.
pub audio: Result<Vec<Pick>, String>,
}

impl Listing {
/// Each kind with its heading, in the order the Devices panel shows them.
pub fn kinds(&mut self) -> [(&'static str, &mut Result<Vec<Pick>, String>); 4] {
[
("Cameras", &mut self.cameras),
("Displays", &mut self.displays),
("Windows", &mut self.windows),
("Audio inputs", &mut self.audio),
]
}
}

#[cfg(feature = "capture")]
pub async fn list() -> Listing {
fn picks<T>(result: Result<Vec<T>, impl std::fmt::Display>, pick: impl Fn(T) -> Pick) -> Result<Vec<Pick>, String> {
result
.map(|devices| devices.into_iter().map(pick).collect())
.map_err(|err| err.to_string())
}

let cameras = picks(moq_video::capture::cameras().await, |camera| Pick {
name: camera.name,
source: Source::Camera { id: Some(camera.id) },
});
let displays = picks(moq_video::capture::displays().await, |display| Pick {
name: display.name,
source: Source::Display { id: Some(display.id) },
});
let windows = picks(moq_video::capture::windows().await, |window| Pick {
name: format!("{} ({})", window.title, window.app),
source: Source::Window { id: window.id },
});
let mut audio = picks(moq_audio::capture::devices().await, |device| Pick {
name: format!("{} ({})", device.name, device.host),
source: Source::Microphone { id: Some(device.id) },
});
if cfg!(target_os = "macos")
&& let Ok(audio) = &mut audio
{
audio.push(Pick {
name: "System audio".into(),
source: Source::SystemAudio,
});
}

Listing {
cameras,
displays,
windows,
audio,
}
}

#[cfg(not(feature = "capture"))]
pub async fn list() -> Listing {
let unsupported = || Err("this build cannot capture; rebuild with `--features capture`".to_string());
Listing {
cameras: unsupported(),
displays: unsupported(),
windows: unsupported(),
audio: unsupported(),
}
}
53 changes: 35 additions & 18 deletions rs/moq-studio/src/engine.rs
Original file line number Diff line number Diff line change
Expand Up @@ -229,27 +229,41 @@ pub struct Engine {
}

/// The audio the program publishes.
const AUDIO_BITRATE: moq_net::bandwidth::Rate = moq_net::bandwidth::Rate::from_kbps(128);
pub const AUDIO_BITRATE: moq_net::bandwidth::Rate = moq_net::bandwidth::Rate::from_kbps(128);

/// How often the audio thread reads the bus. Short enough that a read is never
/// close to the bus's buffering, long enough to cost nothing.
const AUDIO_PERIOD: Duration = Duration::from_millis(10);

/// Everything starting the engine takes besides the scene file, which guided
/// setup holds back until the creator has settled on a collection.
#[derive(Clone)]
pub struct Launch {
pub gpu: Gpu,
/// The relay to publish to, if any.
pub connect: Option<(moq_tokio::Client, url::Url)>,
/// The broadcast path the program is published at.
pub broadcast: String,
/// Whether the program video encodes only while someone watches.
pub on_demand: bool,
}

impl Engine {
/// Open the `loaded` collection and history on `gpu`, start the program
/// clock, and save both to `autosave`'s files as they change.
/// With `connect`, publish the program at `broadcast` on that relay,
/// encoding its video only while someone watches when `on_demand`.
/// Open the `loaded` collection and history on `launch.gpu`, start the
/// program clock, and save both to `autosave`'s files as they change,
/// starting at once.
/// With `launch.connect`, publish the program at `launch.broadcast` on that
/// relay, encoding its video only while someone watches when
/// `launch.on_demand`.
///
/// Call inside the tokio runtime that serves the relay connection.
pub async fn start(
gpu: Gpu,
autosave: Autosave,
loaded: autosave::Loaded,
connect: Option<(moq_tokio::Client, url::Url)>,
broadcast: &str,
on_demand: bool,
) -> anyhow::Result<Arc<Self>> {
pub async fn start(launch: Launch, autosave: Autosave, loaded: autosave::Loaded) -> anyhow::Result<Arc<Self>> {
let Launch {
gpu,
connect,
broadcast,
on_demand,
} = launch;
let bus = moq_audio::bus::Bus::new(Default::default())?;
let subscribe = moq_tokio::origin::spawn();
let sources = subscribe.consume();
Expand All @@ -262,7 +276,7 @@ impl Engine {

let (output, origin, publish) = match connect {
Some((client, url)) => {
let (output, publish) = publish(&gpu, client, url, subscribe, broadcast, canvas, on_demand).await?;
let (output, publish) = publish(&gpu, client, url, subscribe, &broadcast, canvas, on_demand).await?;
(Some(output), None, Some(publish))
}
None => (None, Some(subscribe), None),
Expand Down Expand Up @@ -859,16 +873,19 @@ mod tests {
filters: Vec::new(),
});

let gpu = Gpu::new().await.expect("a GPU");
let launch = Launch {
gpu: Gpu::new().await.expect("a GPU"),
connect: Some((client, fixture.url.clone())),
broadcast: "studio".into(),
on_demand: false,
};
let (autosave, _) = Autosave::open(dir.path().join("show.toml")).unwrap();
let connect = Some((client, fixture.url.clone()));
let on_demand = false;
let loaded = autosave::Loaded {
collection,
history: History::new(),
dropped: 0,
};
let engine = Engine::start(gpu, autosave, loaded, connect, "studio", on_demand)
let engine = Engine::start(launch, autosave, loaded)
.await
.expect("the studio starts");
assert!(
Expand Down
Loading