Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
10 changes: 6 additions & 4 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -105,7 +105,7 @@ Every panel is a tab you can drag, split, or stack.
| Guests | Invite a guest, show their join link as a QR code, see who joined, and take a guest out of every scene. |
| Filters | Remove the selected source's background, crop the selected source or item, add, order, and set its filters, set an item's blend mode, and copy or paste filters. |
| Keyframes | Animate the selected item: add, time, ease, and remove its keyframes, and set how far each moves, resizes, crops, and filters it (see [Keyframes](#keyframes)). **Play** plays the preview scene's keyframes again. |
| Audio | Pick the audio scene on program and the one to listen to and on which output, set each sound's gain, mute, and delay, and send each sound and audio scene into audio scenes, with a meter on every one. |
| Audio | Pick the audio scene on program and the one to listen to and on which output, set each sound's gain, pan, mute, and delay, and send each sound and audio scene into audio scenes, with a meter on every one. |
| Graph | See the sources, scenes, and audio scenes as nodes, wired left to right to what draws or holds them, and wire them there (see [Scene graph](#scene-graph)). |
| Output | Start and stop recording, pick the picture and the audio scene it records, set the canvas size and frame rate and each scene's size, see the broadcast path, whether the program is encoding, the encoder sessions open, and the encoder work of the program, the recording, and each published scene, and turn the virtual camera of the program or of each published scene on. |
| Health | See how the stream reaches viewers, relay by relay and, for the program and each published scene, track by track, and the cause of each problem, and which recordings a crash left unfinished were repaired at launch. |
Expand All @@ -132,7 +132,7 @@ Drag from a node's right edge onto a scene's, the top layer's, or an audio
scene's left edge to wire them: a picture is placed filling the scene, and a
sound or audio scene is sent into the audio scene. Click a wire, then
**Remove**, to take every item of that source out of the scene, or stop the
send. Each sound's node has its strip's mute, gain, and delay and a waveform
send. Each sound's node has its strip's mute, gain, pan, and delay and a waveform
of its last few seconds, each audio scene's node **On program**, **Listen**,
and the waveform of its mix, and each scene's node **Publish** and the audio
scene it carries. Clicking a scene previews it, and clicking a source picks
Expand Down Expand Up @@ -305,7 +305,7 @@ once, and an audio scene never holds itself, even through others. Add,
rename, reorder, and remove audio scenes in the Scenes panel, where they are
tagged `audio` and listed after the scenes.

The Audio panel is a mixer. Each sound has a strip, with its gain, mute,
The Audio panel is a mixer. Each sound has a strip, with its gain, pan, mute,
delay, and meter, which apply in every audio scene holding it, and a send to
each audio scene; each audio scene has a strip with its meter and a send to
each other audio scene. A source added from the Sources or Devices panel, or
Expand Down Expand Up @@ -618,6 +618,7 @@ loop = true

[audio.guest]
gain = 0.5
pan = -0.5
delay = "80ms"

[sources.camera-faded]
Expand Down Expand Up @@ -722,7 +723,8 @@ audio `scenes` it holds. Exactly one is on `program = true`, and
`recorded = true` has recordings hold one rather than the one on program.
Without `[[audio_scenes]]`, the collection has one audio scene, `Program`, on
program and holding nothing. `[audio.<source>]` sets a source's linear `gain`,
`muted`, and `delay`, in every audio scene holding it.
`muted`, `pan` (`-1.0` hard left to `1.0` hard right, centered by
default), and `delay`, in every audio scene holding it.

`[video.<source>]` sets a source's `background`, `crop`, and `filters`,
wherever it is shown, and an item's `crop`, `filters`, and `blend` its own.
Expand Down
1 change: 0 additions & 1 deletion quest/m0/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -92,7 +92,6 @@ repository.
- [Studio on Windows](/quest/m0/app-windows.md) - the studio runs on Windows, not only Linux
- [Wayland display and window sources](/quest/m0/wayland-sources.md) - portal-picked sources that reopen without the picker
- [Destinations](/quest/m0/destinations.md) - RTMP, RTMPS, and WHIP push with per-destination codecs and shared encoders, on a UI-free egress library a hosted service can reuse
- [Pan each sound](/quest/m0/audio-pan.md) - the Audio panel and `[audio.<source>]` pan each sound wherever it plays
- [Clap-to-measure sync delay](/quest/m0/sync-clap.md) - a clap sets a microphone's delay against its camera
- [Audio filters](/quest/m0/audio-filters.md) - per-source noise suppression, gate, compressor, limiter, and plugins, configured from the studio
- [Browser sources on macOS and Windows](/quest/m0/browser-source.md) - the out-of-process CEF pages Linux has, with shared GPU textures on the other desktops
Expand Down
27 changes: 0 additions & 27 deletions quest/m0/audio-pan.md

This file was deleted.

19 changes: 18 additions & 1 deletion rs/moq-scene/src/collection.rs
Original file line number Diff line number Diff line change
Expand Up @@ -514,6 +514,12 @@ impl Collection {
if !audio.gain.is_finite() || audio.gain < 0.0 {
return Err(Error::Invalid(format!("`{name}` has gain {}", audio.gain)));
}
if !(-1.0..=1.0).contains(&audio.pan) {
return Err(Error::Invalid(format!(
"`{name}` has pan {}, outside -1 to 1",
audio.pan
)));
}
if audio.delay > moq_audio::bus::MAX_DELAY {
return Err(Error::Invalid(format!(
"`{name}` is held back {:?}, past the longest, {:?}",
Expand Down Expand Up @@ -1897,7 +1903,7 @@ impl AudioScene {
}
}

/// The gain, mute, and sync delay of one source with sound, in every audio
/// The gain, mute, pan, and sync delay of one source with sound, in every audio
/// scene that holds it.
#[derive(Clone, Debug, PartialEq, Serialize, Deserialize)]
#[serde(deny_unknown_fields)]
Expand All @@ -1907,6 +1913,10 @@ pub struct Audio {
/// Whether the source is silenced in every audio scene, keeping its gain.
#[serde(default)]
pub muted: bool,
/// Where the source sits between left (`-1.0`) and right (`1.0`), as
/// [`Strip::set_pan`](moq_audio::bus::Strip::set_pan) places it.
#[serde(default, skip_serializing_if = "is_center")]
pub pan: f32,
/// How long the source is held back in every audio scene, so a microphone lines up
/// with a camera that lags it. At most the bus's
/// [`MAX_DELAY`](moq_audio::bus::MAX_DELAY).
Expand All @@ -1919,11 +1929,16 @@ impl Default for Audio {
Self {
gain: 1.0,
muted: false,
pan: 0.0,
delay: Duration::ZERO,
}
}
}

fn is_center(pan: &f32) -> bool {
*pan == 0.0
}

/// An sRGB color with straight alpha, written `#rrggbb` or `#rrggbbaa`.
#[derive(Clone, Copy, Debug, PartialEq, Eq, Hash, SerializeDisplay, DeserializeFromStr)]
pub struct Rgba(pub [u8; 4]);
Expand Down Expand Up @@ -2034,6 +2049,7 @@ mod tests {
Audio {
gain: 0.5,
muted: true,
pan: 0.25,
delay: Duration::from_millis(80),
},
);
Expand Down Expand Up @@ -2203,6 +2219,7 @@ rect = { x = 0, y = 680, width = 1280, height = 40 }
Audio {
gain: 2.0,
muted: false,
pan: 0.0,
delay: Duration::from_millis(120),
}
);
Expand Down
2 changes: 1 addition & 1 deletion rs/moq-scene/src/edit.rs
Original file line number Diff line number Diff line change
Expand Up @@ -533,7 +533,7 @@ impl Draft {
})
}

/// Set or clear the gain, mute, and delay of the source called `name`.
/// Set or clear the gain, mute, pan, and delay of the source called `name`.
pub fn set_audio(&mut self, name: &str, audio: Option<Audio>) -> Result<(), Error> {
let before = self.collection.audio.get(name).cloned();
self.change(Change::Audio {
Expand Down
10 changes: 7 additions & 3 deletions rs/moq-scene/src/source.rs
Original file line number Diff line number Diff line change
Expand Up @@ -306,11 +306,11 @@ impl Sources {
/// stinger plays opens silent and blank, waiting for its first
/// [play](Feed::play), rather than playing once at the start.
///
/// Also applies the collection's gain, mute, and delay to every audio
/// Also applies the collection's gain, mute, pan, and delay to every audio
/// strip, routes each into the mix of every audio scene that reaches it,
/// and applies the frame rate to every web page, which never restarts a
/// source. A removed or edited source leaves every mix at once, so its
/// replacement never plays over it. Panics on a gain or delay
/// replacement never plays over it. Panics on a gain, pan, or delay
/// [`Collection::validate`] refuses.
pub fn sync(&mut self, collection: &Collection) -> Vec<Driver> {
let mixes = &self.mixes;
Expand Down Expand Up @@ -353,6 +353,8 @@ impl Sources {
let audio = collection.audio.get(name).cloned().unwrap_or_default();
strip.set_gain(audio.gain);
strip.set_muted(audio.muted);
// The bus is stereo, the one layout that refuses no pan.
strip.set_pan(audio.pan).expect("stereo bus pans");
strip.set_delay(audio.delay);
for (scene, mix) in &self.mixes {
let routed = heard
Expand Down Expand Up @@ -2414,7 +2416,7 @@ mod tests {
}

/// Editing a source restarts it, removing one stops it, and an untouched one
/// keeps running. Gain, delay, and routing changes reach the strip without
/// keeps running. Gain, pan, delay, and routing changes reach the strip without
/// restarting anything, and a source plays in every audio scene holding it
/// until it is removed.
#[tokio::test]
Expand All @@ -2436,12 +2438,14 @@ mod tests {
Audio {
gain: 0.25,
muted: true,
pan: -0.5,
delay: Duration::from_millis(120),
},
);
assert!(sources.sync(&collection).is_empty());
assert_eq!(guest.gain(), 0.25);
assert!(guest.muted());
assert_eq!(guest.pan(), -0.5);
assert_eq!(guest.delay(), Duration::from_millis(120));
let stream = sources.mix("Stream").unwrap().clone();
assert!(program.routes(&guest));
Expand Down
33 changes: 32 additions & 1 deletion rs/moq-studio/src/ui.rs
Original file line number Diff line number Diff line change
Expand Up @@ -2603,6 +2603,7 @@ impl Panels {
ui.label("");
ui.label("");
ui.label("Gain");
ui.label("Pan");
ui.label("Delay")
.on_hover_text("Hold the source back to line it up with a camera that lags it");
for scene in &scenes {
Expand Down Expand Up @@ -2635,6 +2636,7 @@ impl Panels {
ui.weak("audio scene");
ui.label("");
ui.label("");
ui.label("");
for scene in &scenes {
let mut sent = scene.scenes.contains(name);
let loops = scene.name == *name || state.collection().nests(name, &scene.name);
Expand Down Expand Up @@ -3641,7 +3643,7 @@ fn on_program(scenes: &[moq_scene::AudioScene], name: &str) -> (Vec<moq_scene::A
(next, format!("Put audio scene `{name}` on program"))
}

/// A sound's mute, gain, and delay, as its Audio panel strip and Graph node
/// A sound's mute, gain, pan, and delay, as its Audio panel strip and Graph node
/// show them, and the change made to `audio` with its label and gesture.
fn strip_controls(
ui: &mut Ui,
Expand All @@ -3664,6 +3666,35 @@ fn strip_controls(
let audio = moq_scene::Audio { gain, ..audio.clone() };
change = Some((audio, format!("Set `{name}` gain"), gesture));
}
let mut pan = audio.pan;
let response = ui
.add(
egui::Slider::new(&mut pan, -1.0..=1.0)
.custom_formatter(|pan, _| match pan {
0.0 => "C".to_string(),
pan if pan < 0.0 => format!("L{:.0}", -pan * 100.0),
pan => format!("R{:.0}", pan * 100.0),
})
.custom_parser(|text| {
let text = text.trim().to_ascii_uppercase();
let percent = |side: &str| side.trim().parse::<f64>().ok().map(|n| n / 100.0);
match text.as_str() {
"C" => Some(0.0),
_ if let Some(side) = text.strip_prefix('L') => percent(side).map(|n| -n),
_ if let Some(side) = text.strip_prefix('R') => percent(side),
_ => text.parse().ok(),
}
}),
)
.on_hover_text("Pan: place the sound between left and right; double-click to center");
let gesture = gestures.of(&response);
if response.double_clicked() {
pan = 0.0;
}
if pan != audio.pan {
let audio = moq_scene::Audio { pan, ..audio.clone() };
change = Some((audio, format!("Set `{name}` pan"), gesture));
}
let max = moq_audio::bus::MAX_DELAY.as_millis() as u64;
let mut millis = audio.delay.as_millis() as u64;
let response = ui
Expand Down