Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
7 changes: 3 additions & 4 deletions quest/m0/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -60,9 +60,8 @@ cameras, packaging, signing) that one monorepo no longer pays off.
- **Pin:** moq is a `moq` git submodule whose crates are path dependencies,
as in moq.pro, so the studio compiles against one moq checkout.
- **moq-scene** moved here with the app, its only consumer. The
[viewer-side composition](/quest/m3/compose/README.md) epic (now m3) moved whole:
browser composition ships as a package from this repository layered on
`@moq/watch`, and the server composite is a headless binary here rather
[viewer-side composition](/quest/m2/compose/README.md) epic (now m2) moved whole:
the server composite is a headless binary here rather
than a moq-cli subcommand, so moq never depends on the studio.
- **Egress library:** lives here first, beside its studio consumer, and moves
to moq only when a hosted service adopts it.
Expand All @@ -77,7 +76,7 @@ broadcast reaches viewers, not the fallback. Sources are not published to the
relay by default (decided 2026-11-03 on
[moq-dev/studio#21](https://github.com/moq-dev/studio/pull/21)), because
each encoding costs an encoder session and consumer GPUs have few, so the
[viewer-side composition](/quest/m3/compose/README.md) epic moved to m1.
[viewer-side composition](/quest/m2/compose/README.md) epic moved to m1.
Overlays and animation render into the program here, and travel to viewers
in the layout track only once that epic lands.

Expand Down
5 changes: 4 additions & 1 deletion quest/m2/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -10,8 +10,11 @@ it.
Opened 2026-11-05 (maintainer, /quest-plan). A quest moves up to m0 or m1
when a consumer appears. Publishing a source independently and viewer-side
composition moved on to [m3](/quest/m3/README.md) on 2026-11-07 (maintainer):
no customer asks for them.
no customer asks for them. They came back to m2 on 2026-11-08 (maintainer,
/quest-plan), replanned around players that composite independent sources
over the program.

## Required

- [Overlays drop their Blitz panic guard](/quest/m2/overlay-blitz-release.md) - the next Blitz release lets overlay checks parse without `catch_unwind`
- [Viewer-side composition](/quest/m2/compose/README.md) - players composite independently published sources over the program from a layer layout track
58 changes: 58 additions & 0 deletions quest/m2/compose/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,58 @@
# Viewer-side composition

## Goal

A studio broadcast carries each independently published source as its own
track, plus a layer layout track that says where and when each one shows.
Players composite those sources over the program and mix their audio in, so
a viewer sees the creator's picture while those sources travel, and can be
toggled, as their own tracks. The end goal is moq's own players, `moq play`
and `@moq/watch`, playing a studio broadcast this way from a layer layout
that hang specifies.

This README's own work is the end-to-end test: a studio broadcast with an
independently published camera and microphone plays in `moq play` and
`@moq/watch` with the camera placed and timed as the studio's preview shows
it, within a stated tolerance, and with its sound mixed in.

## Plan

Replanned 2026-11-08 (maintainer, /quest-plan). This supersedes the
2026-11-07 deferral on
[moq-dev/studio#97](https://github.com/moq-dev/studio/pull/97), which waited
for [Published independently](/quest/m2/compose/publish-independently.md) to
land before replanning:

- **Reference picture:** ✅ the program with each independently published
source layered over it (rejected: the full scene rebuilt from every
source's track, which reverses the 2026-11-04 encoder-cost decision; both,
chosen per scene).
- **Players:** ✅ native first (rejected: a browser player only; a browser
page layering sources without the studio's placement rules). The end goal
is moq's `moq play` and `@moq/watch`.
- **Compositor:** ✅ hang's draft specifies the layer layout, and each player
implements it with its own code. Parity comes from a shared fixture within a
tolerance, not shared code. Rejected: moving moq-scene into moq, which
reverses the 2026-11-02 rule that moq never depends on this repository.
This replaces the 2026-10-26 "one compositor, three hosts" rule.
- **Prototype:** ✅ a throwaway `moq-studio --watch` on moq-scene with a
provisional layer format, so the format settles without moq round trips
(rejected: straight to `moq play`). moq's spec quest starts from it.
- **Audio:** ✅ players mix independently published audio sources with the
program's, at the gain and mute the layout carries (rejected: video only).
- **Controls:** ✅ viewer toggles and priorities land in the native player
(rejected: browser only).
- **Browser composition** (moq-scene's browser build layered on
`@moq/watch`): ✅ deleted, since `@moq/watch` layering supersedes it.
Server composition stays, replanned to flatten program and layers for
destinations that cannot compose.
- **Milestone:** ✅ m2 (rejected: m1; staying in m3). The moq quests join
moq's m3.

Still settled from 2026-10-26: local sources are tracks in the studio's one
broadcast, and guests stay their own broadcasts, named through the hang
catalog's `contributions` references.

moq's side, in its own quest tree (m3): hang specifies a layer layout track,
and `moq play` and `@moq/watch` composite it.

26 changes: 26 additions & 0 deletions quest/m2/compose/controls.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,26 @@
# [M] Viewer controls

## Goal

A viewer watching a studio broadcast can switch off any layer the layout
marks toggleable (a facecam, a chat overlay), and stops fetching its track.
The layout's per-layer priorities steer delivery, so under congestion the
least important sources degrade first. The creator sets both per source in
the studio.

## Plan

Decided 2026-11-08 (maintainer, /quest-plan): the controls land in the native
player, the [watch prototype](/quest/m2/compose/watch.md) (rejected: browser
only). moq's players take them from hang's layer spec.

Map layer priority onto the subscription priorities moq-lite already has
(`doc/concept/moq-lite.md`), rather than a second mechanism. A toggled-off
layer is unsubscribed, so it costs the viewer and the relay nothing. The
studio keeps encoding it, so toggling it back on can reach its recent past.
Subscriptions are per track, not per layer: a source shown by several layers
stays subscribed while any of them is on, at the highest priority among them.

## Required

- [moq-studio watch prototype](/quest/m2/compose/watch.md) - the viewer that exposes the toggles
30 changes: 30 additions & 0 deletions quest/m2/compose/hang-layers.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,30 @@
# [S] The studio publishes hang's layer layout

## Goal

The `moq` submodule includes hang's layer layout track and `moq play`'s layer
compositing. The studio publishes hang's format instead of the provisional
`dev.moq.studio.layers`, and the `moq-studio --watch` prototype is gone, with
`README.md` pointing viewers at `moq play` instead.

## Plan

Decided 2026-11-08 (maintainer, /quest-plan): the layer layout becomes
official moq, so `moq play` and `@moq/watch` play a studio broadcast. Once
the wait clears, advance the gitlink with `just upgrade-moq` in this PR, move
the studio to hang's types, and delete the
[watch prototype](/quest/m2/compose/watch.md) and its flag.

## Waiting

- moq-dev/moq: hang's layer layout spec and `moq play`'s layer compositing
land, planned in moq's m3 from the
[watch prototype](/quest/m2/compose/watch.md)'s format. Check: both quests'
PRs have merged on moq's `main`, deleting their files, and hang exposes the
layer layout. A quest file that moved, or went away without its work
landing, does not count.

## Required

- [moq-studio watch prototype](/quest/m2/compose/watch.md) - the format moq specifies
- [Viewer controls](/quest/m2/compose/controls.md) - toggles and priorities the spec carries
48 changes: 48 additions & 0 deletions quest/m2/compose/layout.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,48 @@
# [M] Layer layout track

## Goal

A studio broadcast that publishes a source independently also carries a layer
layout track. For each such source, it says where and when that source shows
over the program: its transform, crop, z-order, opacity, and visibility, its
item's keyframes and when its scene went on program, all on the media clock,
and for audio its gain and mute. Until hang specifies it, the track is the
provisional `dev.moq.studio.layers` in the studio catalog.

## Plan

Replanned 2026-11-08 (maintainer, /quest-plan) around the program with
independent layers over it, rather than the full scene state; see the
[epic](/quest/m2/compose/README.md).

A JSON track in snapshot mode, so a late joiner gets the current state.
Version it, and have a player refuse an unknown version loudly and play the
program alone. Keep it free of local paths and credentials: layers name
tracks and contribution references, never files.

Layers draw over the program in the layout's order, so the studio's preview
of a scene with an independent source shows what viewers see. A layer carries
only what every player can reproduce: transform, crop, z-order, opacity,
visibility, and keyframes. A picture source's track is its capture before any
filter (decided 2026-11-02, maintainer). So an item that applies anything
more, such as a video filter, mask, LUT, blend mode, or background removal,
needs that effect baked into its track, or the option refused for it. Settle
which here, with
[Published independently](/quest/m2/compose/publish-independently.md)'s
background-removal note. An audio source's track already carries its sound
after its filters (decided 2026-11-02), so its layer carries only gain and
mute.

Items carry their keyframes (`moq_scene::Keyframe`), which play from when
their scene went on program. The layout says when that was, so a late joiner
lands at the same point.

Each layer also carries its priority and whether a viewer may toggle it, for
[Viewer controls](/quest/m2/compose/controls.md).

The [watch prototype](/quest/m2/compose/watch.md) is this track's first
reader. What the two settle becomes moq's hang spec draft.

## Required

- [Published independently](/quest/m2/compose/publish-independently.md) - the source tracks a layer names
Original file line number Diff line number Diff line change
Expand Up @@ -24,18 +24,18 @@ source among those on program (`Publish::set_on_program` in

Moved to m2 on 2026-11-05 (maintainer, /quest-plan): no service places
sources itself yet, so it waits behind m0 and m1. Moved on to m3 on
2026-11-07 (maintainer, /quest-spawn): it drives no customer demand.
2026-11-07 (maintainer, /quest-spawn): it drives no customer demand. Back to
m2 on 2026-11-08 (maintainer, /quest-plan) in the
[viewer-side composition](/quest/m2/compose/README.md) epic, whose players
composite independently published sources over the program and mix their
audio back in.

A source with its background removed (`background = "remove"`) keeps the
background in its track, since H.264 carries no transparency. Publishing one
independently needs its cut-out in the track, such as moq-video encoding
alpha, or the option refused for it.

Low priority: this is for services that move webcams and overlays themselves,
and none exists yet. The option lives on the source, in the scene file, so it
The option lives on the source, in the scene file, so it
travels with the scene. The option and its effect on the program change what a
user does, so `README.md` documents them.

## Related

- [Viewer-side composition](/quest/m3/compose/README.md) - the epic that composes from independently published sources
31 changes: 31 additions & 0 deletions quest/m2/compose/server.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,31 @@
# [M] Server composition

## Goal

A headless compositor binary in this repository subscribes to a studio
broadcast's program, its independently published sources, and its layer
layout, and publishes the flattened program, headless (a GPU, or lavapipe
without one). It can push that program to RTMP and WHIP through the studio's
egress library. So destinations that cannot compose, such as RTMP, WHIP, HLS,
and legacy players, still see the sources the creator published
independently.

## Plan

Replanned 2026-11-08 (maintainer, /quest-plan): it flattens the program and
its layers, since a source published independently leaves the program; see
the [epic](/quest/m2/compose/README.md).

Decided 2026-10-26 (maintainer): its own command, not a mode of
`moq transcode`. A hosted service embeds the same library. It publishes its
own broadcast, never the studio's names: each publisher produces its own
stream.

Changed 2026-11-02 (maintainer): it was planned as `moq compose` in
moq-cli, but moq-scene moved here with the studio, so the command lives here
too and moq never depends on this repository. Settle its name in this quest
and document it in `README.md`.

## Required

- [Layer layout track](/quest/m2/compose/layout.md) - what the server flattens
27 changes: 27 additions & 0 deletions quest/m2/compose/watch.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,27 @@
# [M] moq-studio watch prototype

## Goal

`moq-studio --watch <url>` plays a studio broadcast: the program, with each
source the layer layout names composited over it on the media clock, and
their audio mixed with the program's. A broadcast without a layer layout plays
its program alone.

## Plan

Decided 2026-11-08 (maintainer, /quest-plan): a throwaway prototype, so the
layer format settles without moq round trips before hang specifies it
(rejected: going straight to `moq play`). It lives in `moq-studio`, not its
own binary, since [hang-layers](/quest/m2/compose/hang-layers.md) deletes it
once `moq play` plays layers.

Build it on moq-scene, which already plays remote broadcasts as sources for
guests (`rs/moq-studio/src/guests.rs`): a scene that the layout drives rather
than a scene file. `README.md` documents the flag while it exists.

Its working format is the starting point for moq's hang layer spec, which
waits on this quest.

## Required

- [Layer layout track](/quest/m2/compose/layout.md) - what the prototype plays
7 changes: 2 additions & 5 deletions quest/m3/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -9,9 +9,6 @@ is offered for work until a customer need moves it up.

Opened 2026-11-07 (maintainer, /quest-spawn): publishing a source
independently drives no customer demand, so it and the viewer-side
composition epic that requires it moved here from m2.
composition epic that requires it moved here from m2. Both went back to
[m2](/quest/m2/README.md) on 2026-11-08 (maintainer, /quest-plan).

## Required

- [Published independently](/quest/m3/publish-independently.md) - a source opts out of the program and publishes its own track for services that place it themselves
- [Viewer-side composition](/quest/m3/compose/README.md) - independently published sources plus a layout track, composed by viewers and by a server
70 changes: 0 additions & 70 deletions quest/m3/compose/README.md

This file was deleted.

27 changes: 0 additions & 27 deletions quest/m3/compose/browser.md

This file was deleted.

Loading