Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
38 changes: 35 additions & 3 deletions .github/workflows/nightly.yml
Original file line number Diff line number Diff line change
@@ -1,8 +1,11 @@
name: Nightly

# The gates too slow or too specialized for a pull request: compositing and
# publishing through wgpu on Mesa's software Vulkan driver, and `just ci` on
# macOS, whose capture backends Check's Linux runner never compiles.
# The gates too slow or too specialized for a pull request: compositing,
# publishing, and the virtual camera through wgpu on Mesa's software Vulkan
# driver, the camera in a private PipeWire session; the virtual camera's
# v4l2loopback fallback, a module load no pull request job should make; and
# `just ci` on macOS, whose capture backends Check's Linux runner never
# compiles.

permissions:
contents: read
Expand Down Expand Up @@ -36,6 +39,35 @@ jobs:
- name: Render
run: nix develop --command just render-sw

v4l2loopback:
name: v4l2loopback camera
runs-on: ubuntu-24.04
timeout-minutes: 30
env:
NEXTEST_PROFILE: ci

steps:
- name: Checkout
uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
with:
persist-credentials: false
submodules: true

- uses: ./.github/actions/nix

- name: Load v4l2loopback
# Out of tree, so DKMS builds it for the runner's kernel.
run: |
sudo apt-get update
sudo apt-get install -y "linux-headers-$(uname -r)" v4l2loopback-dkms
sudo modprobe v4l2loopback exclusive_caps=1
sudo udevadm settle
# The runner user is not in the video group.
sudo chmod a+rw /dev/video*

- name: Test
run: nix develop --command just v4l2loopback

macos:
name: macOS
runs-on: macos-latest
Expand Down
2 changes: 2 additions & 0 deletions Cargo.lock

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

1 change: 1 addition & 0 deletions Cargo.toml
Original file line number Diff line number Diff line change
Expand Up @@ -35,6 +35,7 @@ moq-room = { path = "moq/rs/moq-room" }
moq-scene = { path = "rs/moq-scene" }
moq-stats = { path = "moq/rs/moq-stats" }
moq-tokio = { path = "moq/rs/moq-tokio", default-features = false }
moq-v4l = { path = "moq/rs/moq-v4l" }
# Default features off, as in moq: each consumer opts into native codecs and
# rendering explicitly.
moq-video = { path = "moq/rs/moq-video", default-features = false }
Expand Down
27 changes: 24 additions & 3 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -35,8 +35,10 @@ cargo run --release -p moq-studio --features capture,openh264 -- \
a second, so viewers and recordings on the relay see the broadcast end
rather than cut off.
- `capture` adds camera, display, window, microphone, and system-audio
sources, and listening to a mix. On Linux it links ALSA; add `pipewire` for PipeWire cameras. `openh264`
is the software H.264 fallback for a machine without a hardware encoder.
sources, and listening to a mix. On Linux it links ALSA; add `pipewire` for
PipeWire cameras and the [virtual camera](#virtual-camera)'s PipeWire node.
`openh264` is the software H.264 fallback for a machine without a hardware
encoder.
- `cef` adds browser sources, on Linux only for now. The build takes CEF from
`CEF_PATH`, which the dev shell sets, or else downloads it, and the binary
loads `libcef.so` from beside itself or from that `CEF_PATH`.
Expand Down Expand Up @@ -104,7 +106,7 @@ Every panel is a tab you can drag, split, or stack.
| Filters | Remove the selected source's background, crop the selected source or item, add, order, and set its filters, set an item's blend mode, and copy or paste filters. |
| Keyframes | Animate the selected item: add, time, ease, and remove its keyframes, and set how far each moves, resizes, crops, and filters it (see [Keyframes](#keyframes)). **Play** plays the preview scene's keyframes again. |
| Audio | Add, rename (double-click), and remove mixes, pick the one on program and the one to listen to and on which output, and set each audio source's gain, mute, delay, and the mixes it feeds, with a meter on every mix and source. |
| Output | Start and stop recording, pick the mix it records, set the canvas size and frame rate, add, resize, and remove further canvases, and see the broadcast path, whether the program is encoding, the encoder sessions open, and each canvas's renditions' encoder work. |
| Output | Start and stop recording, pick the mix it records, set the canvas size and frame rate, add, resize, and remove further canvases, see the broadcast path, whether the program is encoding, the encoder sessions open, and each canvas's renditions' encoder work, and turn the virtual camera on. |
| Health | See how the stream reaches viewers, relay by relay and track by track, and the cause of each problem, and which recordings a crash left unfinished were repaired at launch. |
| Replay | Replay the last moments of the program or a source in a replay source, slowed down if you like, or save them as a clip. |
| Hotkeys | Bind keys to cut, transition, preview a scene, replay, save a clip, and start or stop recording, and clear them (see [Hotkeys](#hotkeys)). |
Expand Down Expand Up @@ -997,6 +999,25 @@ The catalog asks viewers to report under `<broadcast>/viewers/`: a viewer
publishes `<broadcast>/viewers/<name>.echo`, as `moq export --echo <name>`
does. The relay decides who may publish there, as for any broadcast.

## Virtual camera

The **Virtual camera** toggle in the Output panel shows the program as a
camera that video-call and conferencing apps can pick, at the canvas size and
frame rate. It stays on until you turn it off or quit; it is not saved in the
scene file. The program is copied for the camera only while an app reads it.
Changing the canvas size or frame rate reopens the camera at the new one, so
an app reading it opens it again. The Devices panel leaves the camera out
while it is on, so it cannot be added as a source of the program it shows.

- **Linux:** a PipeWire camera named "MoQ Studio" (needs the `pipewire`
feature), which browsers and other PipeWire-aware apps list beside your
other cameras. Without PipeWire, the studio writes to the first
[v4l2loopback](https://github.com/v4l2loopback/v4l2loopback) device that
takes the canvas size, for apps that open `/dev/video*` directly. The
studio never loads that module, since it needs root:
`sudo modprobe v4l2loopback exclusive_caps=1 card_label="MoQ Studio"`.
- **macOS and Windows:** not available yet.

## Development

Use the Nix dev shell (`direnv allow`, or `nix develop --command ...`) so
Expand Down
13 changes: 10 additions & 3 deletions justfile
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@ set shell := ["bash", "-euo", "pipefail", "-c"]

# The device, codec, and browser features off by default, which every check
# covers too.
features := "moq-scene/capture,moq-studio/capture,moq-studio/openh264,moq-studio/cef"
features := "moq-scene/capture,moq-studio/capture,moq-studio/openh264,moq-studio/pipewire,moq-studio/cef"

# List the available development commands.
default:
Expand Down Expand Up @@ -45,8 +45,9 @@ fix:
bun biome check --write .
cargo fmt --all

# Composite scenes, render browser sources, and publish a program through wgpu
# on Mesa's software Vulkan driver, so no GPU is needed.
# Composite scenes, render browser sources, publish a program, and show it as a
# PipeWire camera through wgpu on Mesa's software Vulkan driver, so no GPU is
# needed.
render-sw:
#!/usr/bin/env bash
set -euo pipefail
Expand All @@ -57,6 +58,12 @@ render-sw:
cargo nextest run --locked -p moq-scene --lib --run-ignored only -E 'test(/^compositor::tests::/) | test(/^browser::layers::tests::/)'
cargo nextest run --locked -p moq-studio --features openh264 --run-ignored only -E 'test(/^engine::tests::the_program_/) | test(=setup::tests::times_every_canvas)'
cargo nextest run --locked -p moq-studio --features cef --run-ignored only -E 'binary(browser)'
sh/pipewire-session.sh cargo nextest run --locked -p moq-studio --features pipewire --run-ignored only -E 'test(=engine::tests::the_virtual_camera_reaches_a_pipewire_app)'

# Show the program through a v4l2loopback device, which needs the module loaded
# (`sudo modprobe v4l2loopback exclusive_caps=1`) and the device writable.
v4l2loopback:
cargo nextest run --locked -p moq-studio --run-ignored only -E 'test(=vcam::v4l2::tests::a_v4l2_app_reads_the_program)'

# Move the moq submodule to moq's latest main (or a given commit) and refresh Cargo.lock and bun.lock.
upgrade-moq rev="origin/main":
Expand Down
9 changes: 8 additions & 1 deletion quest/m0/vcam/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -18,8 +18,15 @@ Decided 2026-10-22:
- **macOS:** a CoreMediaIO Camera Extension, which needs signing.
- **Order:** the first backend to land adds the studio toggle.

The Linux backend landed the toggle: **Virtual camera** in the Output panel,
for the program, not saved in the scene file. Each OS backend is a
`vcam::backend` in `rs/moq-studio`, fed tightly packed RGBA program frames on
the `vcam` thread only while an app reads the camera. Each published scene
gets a toggle of its own once [published scenes](/quest/m0/scenes/published.md)
land.

## Required

- [Linux virtual camera](/quest/m0/vcam/linux.md) - a PipeWire video node carrying the program
- [Linux virtual camera for each published scene](/quest/m0/vcam/linux.md) - a PipeWire camera per published scene, beside the program's
- [Windows virtual camera](/quest/m0/vcam/windows.md) - a Media Foundation virtual camera on Windows 11
- [macOS virtual camera](/quest/m0/vcam/macos.md) - a CoreMediaIO Camera Extension inside the app bundle
26 changes: 20 additions & 6 deletions quest/m0/vcam/linux.md
Original file line number Diff line number Diff line change
@@ -1,9 +1,23 @@
# [M] Linux virtual camera
# [S] Linux virtual camera for each published scene

## Goal

The studio publishes the program or a published scene as a PipeWire video node that
PipeWire-aware apps (browsers and conferencing apps) list as a camera. It
falls back to an existing v4l2loopback device when one is present.
The studio's Devices panel leaves out its own virtual camera, so picking it
cannot feed the program back into itself.
Each published scene has a **Virtual camera** toggle of its own, as the
program does, and shows as its own PipeWire camera named after the scene, or
else its own v4l2loopback device. An app can then pick the program or any
published scene. The Devices panel leaves out every camera the studio
publishes, so picking one cannot feed it back into itself.

## Plan

The program's camera landed in
[moq-dev/studio#9](https://github.com/moq-dev/studio/pull/9): `vcam::Camera`
in `rs/moq-studio` reads the main canvas back through moq-video's
`render::readback` and reopens when the canvas changes size or frame rate.
Its PipeWire node name (`moq-studio`) and description ("MoQ Studio") are
fixed, so a camera per scene needs a name of its own, such as
`moq-studio-<scene>`.

## Required

- [Published scenes](/quest/m0/scenes/published.md) - any scene publishes as its own broadcast at its own size, replacing canvases
14 changes: 10 additions & 4 deletions rs/moq-studio/Cargo.toml
Original file line number Diff line number Diff line change
Expand Up @@ -29,10 +29,10 @@ nvidia = ["moq-video/nvidia"]
openh264 = ["moq-video/openh264"]
# Intel/AMD VAAPI hardware codecs (Linux); its bindgen needs libclang.
vaapi = ["moq-video/vaapi"]
# PipeWire cameras, drawn without a copy, and audio through the sound server.
# Wayland screens and windows are not reachable yet: display and window sources
# capture X11 only.
pipewire = ["capture", "moq-scene/pipewire"]
# PipeWire cameras, drawn without a copy, audio through the sound server, and
# the virtual camera as a PipeWire node. Wayland screens and windows are not
# reachable yet: display and window sources capture X11 only.
pipewire = ["capture", "moq-scene/pipewire", "dep:pipewire"]
# Browser sources, rendered by CEF in a helper process (Linux). Off by default:
# the build downloads the CEF runtime (or takes it from `CEF_PATH`) and copies
# it beside the binary, where the binary looks for it when it starts.
Expand Down Expand Up @@ -92,6 +92,12 @@ winit = { version = "0.30", default-features = false }
# PipeWire capture uses, so the two share one D-Bus stack.
ashpd = { version = "0.13", default-features = false, features = ["global_shortcuts", "tokio"] }
futures-util = "0.3"
# The virtual camera's v4l2loopback fallback. Pure Rust with its kernel bindings
# checked in, and already in the graph through moq-video's capture.
moq-v4l = { workspace = true }
# The virtual camera's PipeWire node, behind `pipewire`. moq-video links the
# same release; `v0_3_34` adds the trigger a driving stream paces frames with.
pipewire = { version = "0.10", optional = true, features = ["v0_3_34"] }
# Whether the window is on Wayland, where only the portal hears keys globally.
raw-window-handle = "0.6"
# Which keysyms the X11 layout puts on each key, so a hotkey grabs the key
Expand Down
Loading