Skip to content

feat(studio): a Linux virtual camera, as a PipeWire node or a v4l2loopback device - #9

Merged
kixelated merged 7 commits into
mainfrom
quest/m0/vcam/linux
Nov 7, 2026
Merged

kixelated merged 7 commits into
mainfrom
quest/m0/vcam/linux

Conversation

@kixelated

@kixelated kixelated commented Nov 2, 2026 •

Copy link
Copy Markdown
Contributor

Problem

Video-call and conferencing apps offer no ingest, so a creator cannot use the
studio's program in them. The virtual camera epic
starts with Linux: a PipeWire video node that PipeWire-aware apps list as a
camera, falling back to an existing v4l2loopback device. The first backend to
land also adds the studio's toggle.

Approach

  • rs/moq-studio/src/vcam.rs: on each frame the program clock calls
    Camera::offer with the main canvas's program. Only while an app reads the
    camera, it reads the program back through moq-video's
    render::readback::Readback (pinned by
    moq-dev/studio#69), which
    never waits on the GPU and refuses a copy when its three staging buffers are
    all in flight; that frame is dropped and counted. A vcam thread waits for
    each copy and hands tightly packed RGBA to the backend.
  • The camera follows the canvas: a new size or frame rate closes the backend
    and opens it again for the new one on the vcam thread, so the program
    clock never waits on it. An app reading the camera opens it again.
  • vcam/pipewire.rs (the pipewire feature): a Video/Source stream with
    media.role = Camera, named moq-studio and described as "MoQ Studio".
    This is what libwebrtc (Chromium and Firefox) and moq-video's own reader
    filter on. It offers one fixed format, RGBA at the canvas size and frame
    rate, which libwebrtc maps to kABGR, so the studio does no conversion.
    The stream drives the graph: each picture triggers a cycle. It runs on its
    own PipeWire main loop, fed through a one-picture slot and a wake that is
    never queued twice. Streaming turns copying on; Paused turns it off.
  • vcam/v4l2.rs: the fallback. It takes the first /dev/videoN whose driver
    is v4l2 loopback and accepts the canvas size as tightly packed I420 (YU12,
    which the common V4L2 camera readers take). It writes one frame per
    write. A device that answers with another layout is refused, with the
    reason.
  • Engine and UI: Engine::set_camera opens and closes the camera outside the
    state lock. The Output panel gets a Virtual camera checkbox, plus where
    the camera lives, the picture size and rate, whether an app is reading it,
    frames sent and dropped, and any error. The Devices panel leaves out the
    camera while it is on, so it cannot be added to the program it shows.

Impact

  • New Virtual camera toggle in the Output panel. On Linux it adds a "MoQ
    Studio" PipeWire camera (needs the pipewire feature), or else uses an
    existing v4l2loopback device. macOS and Windows report that it is not
    available yet.
  • moq-studio's pipewire feature now also builds the virtual camera's
    PipeWire node.
  • just check/just test now cover moq-studio/pipewire, as AGENTS.md
    already says they do. Previously the justfile left it out, so this code
    would never have been linted.
  • Nightly: render-sw also runs the PipeWire camera test inside a private
    PipeWire + WirePlumber session (sh/pipewire-session.sh). A new
    v4l2loopback job builds the module with DKMS and runs just v4l2loopback.
  • README: a Virtual camera section, the Output panel row, and the feature note.
  • No public API or wire change: moq-studio is a binary, and moq-video's
    readback is used as published.
  • quest/m0/vcam/linux.md now holds what remains, a camera per published
    scene, which waits on published scenes.

Decisions

  1. Where the program readback lives. vcam.rs used to copy the program
    into staging buffers itself, the same shape as moq-video's private export
    readback.
    • ✅ (Recommended) Land a public readback in moq-dev/moq with a
      moq-side test, then a pin here, and switch vcam.rs to it before this
      merges. Decided by the maintainer on 2026-11-02 (the comment below and
      the quest/m0/pin-readback.md Plan). Landed as
      moq-dev/moq#8056
      (moq_video::render::readback), pinned by
      moq-dev/studio#69; this PR
      now uses it and drops the copy.
    • Merge as is, and move the readback later.

Choices made without asking

  • The toggle is runtime only, not saved in the scene file. It works like
    OBS's "Start Virtual Camera" button, and the scene file is meant to be
    shared.
  • The camera shows the main canvas's program. Since this PR was opened,
    main gained further canvases and planned
    published scenes
    to replace them, and the quest's Goal grew to "the program or a published
    scene". A camera per published scene needs published scenes first, so it
    stays in quest/m0/vcam/linux.md with that quest as its blocker, rather
    than a per-canvas toggle here that published scenes would replace.
  • A canvas change reopens the camera rather than failing it or refusing
    the edit, since undo, a reload, and the Output panel can all change the
    canvas while it is on.
  • The Devices panel hides the camera by the id it is open as
    (pipewire:moq-studio, or the loopback's /dev/videoN), only while it is
    on. A listing taken while the camera was off still shows nothing of it,
    since it did not exist.
  • PipeWire offers RGBA only. Chromium's and Firefox's libwebrtc, GStreamer,
    and moq-video all take it, and RGBA avoids a CPU conversion and a YUV matrix
    the app would have to guess. v4l2loopback uses I420, because V4L2 readers
    expect YUV.
  • Fallback on any PipeWire failure. When the node cannot be added (no
    session, or a build without pipewire), the studio logs a warning and tries
    v4l2loopback. If both fail, the error names why each one refused.

Verification

All on nixos (Linux, RTX 3070 Ti, desktop PipeWire session) at head
63720dc, since the PipeWire and v4l2loopback paths need Linux:

  • just ci passes (quest gate just ci: lints including
    moq-studio/pipewire, and 278 tests).
  • engine::tests::the_virtual_camera_reaches_a_pipewire_app passed against
    the desktop PipeWire session (features pipewire,openh264) and inside
    sh/pipewire-session.sh (feature pipewire, as nightly runs it). The test
    starts the engine with a red canvas, turns the camera on, and opens
    pipewire:moq-studio through moq-video's PipeWire camera reader, which
    filters on class and role as browsers do. It asserts the red picture
    arrives, reading flips on, and frames are counted; then it resizes the
    canvas to 320x180, waits for the camera to reopen at that size, and reads
    red through it again.
  • Not re-run: vcam::v4l2::tests::a_v4l2_app_reads_the_program, since
    nixos has no v4l2loopback module and loading one means changing the host's
    NixOS configuration. vcam/v4l2.rs is unchanged since it passed against a
    real v4l2loopback device, and in nightly
    run 41543325349
    (head 81c621d), which also passed the PipeWire test in the private
    session. The merged nightly's v4l2loopback job covers it from here.
  • Not tested by hand in a browser or conferencing app, nor the Devices
    panel's filter, which no test drives.

Alternatives

  • Offer I420 or NV12 to PipeWire too. Every current consumer takes RGBA,
    so a second format would be code with no reader.
  • Expose the PipeWire node through DMA-BUF to skip the readback. libwebrtc
    maps DMA-BUFs back to the CPU anyway, and modifier negotiation is a lot of
    surface for one copy at 30 fps.
  • Put the backend in moq-video. The m0 README
    names virtual cameras as studio surface, and the macOS and Windows backends
    are app-bundle and session-lifetime objects, so the backends live here.

Review

Codex (quest review), gpt-6.1-sol, medium effort, a new session after the
readback design change. Round 2 on head
63720dcc0f6c3270367d83a4b12929be3d78c740: no findings.

  • Round 1, "Move native camera export backends into moq-video"
    (vcam/pipewire.rs:139): refuted. The maintainer's 2026-11-02 decision
    moving the studio out of moq names virtual cameras as studio surface
    (quest/m0/README.md:56-58), the vcam epic's Plan places the backends here,
    and the macOS and Windows backends are app-bundle and session-lifetime
    objects. The maintainer's decision on this PR moved only the readback
    upstream, which this head uses. Round 2 accepted it.

Earlier session (head fc0d1c1, before the merge): round 1's "Wait for node
registration before reporting startup success" was fixed in fc0d1c1;
"Reuse the upstream rendering readback" is now fixed by the switch to
moq_video::render::readback.

🤖 Generated with Claude Code

(Written by Claude Opus 5.5)

kixelated and others added 2 commits November 2, 2026 14:42
Quest-Claim: 131a812fdb42e147
…pback device

The Output panel's Virtual camera toggle shows the program as a camera:
a PipeWire Video/Source node with the Camera role (the `pipewire`
feature), falling back to an existing v4l2loopback device. The program
is read back for it only while an app reads the camera.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@kixelated kixelated changed the title Linux virtual camera feat(studio): a Linux virtual camera, as a PipeWire node or a v4l2loopback device Nov 2, 2026
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@kixelated

Copy link
Copy Markdown
Contributor Author

Decision (maintainer, 2026-11-02): the readback goes upstream first. A public readback-only Exporter (RGBA, no encoder probe) lands in moq-dev/moq with a moq-side test, then a pin quest here bumps it, and vcam.rs switches to it before this PR merges. This PR stays a draft until then; the pin quest will be planned and set as a Required blocker of quest/m0/vcam/linux.md.

(Written by Claude Opus 5.5)

kixelated and others added 3 commits November 6, 2026 18:58
Adapts the virtual camera to main: it reads the main canvas back through
moq-video's render::readback (pinned by #69) instead of a copy here, and
reopens when the canvas changes size or frame rate. The Devices panel leaves
out the camera while it is on. quest/m0/vcam/linux.md keeps what remains:
a camera per published scene.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
One Linux dependency table, the camera's Output panel row after the canvases, and the lockfile entries for moq-v4l and pipewire.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@kixelated

Copy link
Copy Markdown
Contributor Author

Design change: merged main and switched the virtual camera to moq-video's public readback, as decided on 2026-11-02.

  • vcam.rs drops its own staging-buffer copy and reads the program back through moq_video::render::readback (moq-dev/moq#8056, pinned by moq-dev/studio#69).
  • Main can now change the canvas size and frame rate while running, so the camera reopens its backend for the new canvas on the vcam thread instead of failing on a mismatched copy.
  • The Devices panel leaves out the camera while it is on, as quest/m0/vcam/linux.md's Goal asks.
  • Main's quest grew to "the program or a published scene". Published scenes have not landed, so quest/m0/vcam/linux.md now holds only a camera per published scene, blocked on quest/m0/scenes/published.md.

The review restarted with quest review --new; round 2 on 63720dc is clean.

(Written by Claude Opus 5.5)

@kixelated
kixelated marked this pull request as ready for review November 7, 2026 03:20
@coderabbitai

coderabbitai Bot commented Nov 7, 2026 •

Copy link
Copy Markdown

Warning

Review limit reached

You've used all free OSS reviews for now. Wait for the free limit to reset to keep reviewing this public repository.

Next included review available in 40 minutes.

Check out review usage here.

View limit details

Limit details: You’ve used the included review currently available.

Learn how review limits work.

Review configuration:

Run configuration

  • Source mode: Auto
  • Repository: Unavailable
  • Branch: Unavailable
  • File: No configuration file from this repository applied
  • Commit: Unavailable
  • Layers: Organization UI
  • Review profile: CHILL
  • Plan: Advanced
  • Run ID: 32c0bd11-81c2-48c7-9995-3f618578764d
Commits

Reviewing files that changed from the base of the PR and between 306e114 and 03c91dd.

Files ignored due to path filters: 1

  • Cargo.lock is excluded by !**/*.lock
Files selected for processing: 14

  • .github/workflows/nightly.yml
  • Cargo.toml
  • README.md
  • justfile
  • quest/m0/vcam/README.md
  • quest/m0/vcam/linux.md
  • rs/moq-studio/Cargo.toml
  • rs/moq-studio/src/engine.rs
  • rs/moq-studio/src/main.rs
  • rs/moq-studio/src/ui.rs
  • rs/moq-studio/src/vcam.rs
  • rs/moq-studio/src/vcam/pipewire.rs
  • rs/moq-studio/src/vcam/v4l2.rs
  • sh/pipewire-session.sh
  • Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

Share

@kixelated kixelated left a comment

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Automated review by review (OpenAI)

Reviewed 63720dc against 299a97a. No actionable introduced defects found.

Direction looks sound: the camera reuses the pinned upstream readback, keeps backend work outside the program-state lock, and orders canvas reopen and picture delivery on its worker. Reviewed the PipeWire and v4l2loopback paths, UI toggle/device filtering, and build/test wiring. The documented program-only scope leaves per-scene cameras for the planned follow-up.

Verification: GitHub-only static inspection, including full changed implementation files and relevant callees. No builds, tests, quest context, GPU measurements, or actual browser/PipeWire/v4l2loopback sessions were run. This does not independently validate the reported hardware checks or conferencing-app compatibility.

(Written by review (OpenAI))

@kixelated
kixelated enabled auto-merge (squash) November 7, 2026 04:13
@kixelated
kixelated merged commit 852a6ac into main Nov 7, 2026
3 checks passed
@kixelated
kixelated deleted the quest/m0/vcam/linux branch November 7, 2026 04:30
kixelated added a commit that referenced this pull request Nov 7, 2026
The virtual camera from #9 reads the program, which published scenes keep;
a camera per published scene stays quest/m0/vcam/linux.md, which no longer
waits on published scenes.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant