diff --git a/CHANGELOG.md b/CHANGELOG.md index 2be475c..b059512 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -7,6 +7,35 @@ are tagged `vMAJOR.MINOR.PATCH` and published as ## [Unreleased] +## [0.1.47] - 2026-09-15 + +### Added +- ISO feed selection: choose routed outputs, remember choices, and create files + only when selected feeds deliver media. TCP accepts `source_uuids`; OSC uses + saved panel selections. Empty selections no longer record every source. + +### Fixed +- Each participant ISO is one continuous 1920x1080/30 H.264 MP4 with AAC audio. + Incoming resolution changes are scaled with aspect ratio preserved, without + splitting files. Duplicate output routes share the same participant recording. +- ISO audio uses dedicated isolated participant subscriptions, independently of + the OBS source's Mix/Isolated setting, without changing Audience audio routing. +- Participant workers hold video and fill silence through gaps. Startup timing, + full-range color, planar chroma, and batched audio timestamps are preserved. +- Tiles retains valid pending pixels when a shared-memory read is rejected, + preventing a path that could flash incorrect colors during resolution changes. + +### Validation +- Windows plugin/engine build and 72 native regressions pass. Real MP4 fixtures + cover eight concurrent NVENC tracks, changing resolutions/audio formats, gaps, + color preservation and flash/tone sync within 20 ms. +- Alternating-speaker tests through routing, shared memory and MP4 encoding + confirm silence in each participant's file while the other speaker talks. +- These are synthetic media checks; a live Zoom soak of the final isolated-audio + build has not been completed. Existing mixed recordings are not repaired. +- Windows installer and ZIP include both updated plugin and engine. The signed + macOS installer still requires a separate build on the maintainer's Mac. + ## [0.1.46] - 2026-09-14 ### Fixed diff --git a/CLAUDE.md b/CLAUDE.md index c2f85d7..0b3e4a1 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -227,21 +227,27 @@ Every one of these is documented at length where it lives; the list is the map. interval rather than zeroing on fire -- zeroing discards the remainder that pushed a tick over threshold, which at 60 fps lands every 7 frames (~117 ms, ~8.6 Hz) instead of the documented 10 Hz. -- **ISO recording timing** (`src/iso-video-pacer.h`, `src/iso-audio-gap-fill.h`): - raw video has no per-frame timestamps and ffmpeg cannot be trusted to - invent correct ones from a byte stream — `-use_wallclock_as_timestamps` - is confirmed (via `ffprobe -show_frames`, 2026-08-21) to have **no - effect** on this project's ffmpeg build's rawvideo demuxer, despite - looking like the textbook fix. `record_video_frame()` is called 1:1 with - Zoom's real, fluctuating (10-60fps) per-source delivery, so it must pace - itself to a fixed cadence (duplicate to backfill a stall, drop to shed a - burst) BEFORE the pipe — see `iso_video_frames_due()`. Audio has the - mirror-image problem for a different reason: Zoom only calls back audio - for someone currently talking, so `record_audio_frame()` must backfill - silence across every gap (`iso_audio_silence_frames()`) or the WAV - shrinks by every silent stretch. Both anchor to the same - `os_gettime_ns()` clock so video and audio stay in sync with each other, - not just individually correct. +- **ISO participant A/V recording** (`src/iso-track-writer.cpp`, + `src/iso-av-mux.h`): one writer per Zoom participant ID owns a continuous + 1920x1080/30 H.264 + 48 kHz stereo AAC MP4 until Stop. Source UUIDs and input + dimensions MUST NOT key writer lifetime. Workers scale/letterbox and resample, + hold video/fill silence against the same monotonic clock, and mux timestamped + raw media to one FFmpeg pipe. The input is full-range BT.709; preserve its + colour metadata. B-frames are disabled to avoid fragmented-MP4 startup A/V + offset. Keep startup video history while workers catch up: shrinking the + input queue on the first tick loses early frames. Validate decoded flash/tone + timing with `tests/verify-iso-av.py`, not only frame counts or process success. + ISO capture is explicitly selected by output source UUID (`source_uuids` in + TCP, saved panel choices for OSC). No default-all fallback. Do not pre-open + writers from configured participant IDs: selected routes open on first real + media delivery, so offline/unrouted sources produce no blank files. The feed + checklist updates rows by UUID and retains operator selections across refresh. + Audio MUST come from `IsoAudioTap`, not ZoomSource's embedded PCM (which may + be the meeting mix labelled with the route's participant ID). Audio-only IPC + subscriptions carry `recording_only=true`: receive one-way audio but never + claim a participant out of Audience routing. Install plugin AND engine for + this change. Each tap owns a fresh SHM reader, checks per-slot attribution, + preserves first buffers, and uses the recording epoch to reject stale callbacks. - **Colour range is normalised, never re-declared** (`src/i420-range-expand.h`, applied in `engine/src/engine-video.cpp`'s `onRawDataFrameReceived`): the engine requests `VideoRawdataColorspace_BT709_F` and the plugin declares @@ -1574,6 +1580,16 @@ Home, download, and plugin docs point to `/download/#macos`. ## Media failure presentation (2026-09-06 soak) +Tiles SHM reads must use `read_candidate`, never the pending `frame` directly. +`shm_read_i420_frame` copies before validating its final sequence and can return +Invalid with a modified destination. If `has_frame` was already true, reading +into that frame published rejected pixels with the previous size/generation, +causing a possible one-frame color flash during resolution changes. Commit only +successful even-sized reads by swapping buffers (`zoom-tile-frame-read.h`). +`CoreVideoTileFrameRead` injects rejected resized reads; it fails with the old +direct-write behavior. Live diagnostics log `Tiles kept last valid frame` on +the first rejected read and every 300 thereafter. + `MediaFailureState` tracks current source media failures, bounded by live source assignments. Eight tiles × three failed attempts retain all 24 raw error diagnostics but emit one nonmodal episode notice. Dock polling diff --git a/CMakeLists.txt b/CMakeLists.txt index d56d371..79eb96f 100644 --- a/CMakeLists.txt +++ b/CMakeLists.txt @@ -285,6 +285,9 @@ if(COREVIDEO_BUILD_PLUGIN) src/zoom-diagnostics-dialog.cpp src/zoom-output-profile.cpp src/zoom-iso-recorder.cpp + src/iso-track-writer.cpp + src/iso-audio-tap.cpp + src/iso-audio-reader.cpp src/iso-ffmpeg-pipe.cpp src/zoom-iso-panel.cpp src/zoom-dock.cpp @@ -317,7 +320,7 @@ if(COREVIDEO_BUILD_PLUGIN) if(WIN32) # winmm: talkback-cue.cpp's PlaySound(SND_MEMORY | SND_ASYNC) call -- # see that file's header comment for why it was chosen over WASAPI. - target_link_libraries(obs-zoom-plugin PRIVATE crypt32 winmm) + target_link_libraries(obs-zoom-plugin PRIVATE crypt32 winmm d3d11 d3dcompiler) add_executable(CoreVideoOAuthCallback src/oauth-callback-helper.cpp) target_link_libraries(CoreVideoOAuthCallback PRIVATE ws2_32 shell32 ole32) install(TARGETS CoreVideoOAuthCallback @@ -651,9 +654,37 @@ if(BUILD_TESTING) target_include_directories(CoreVideoIsoRecordingTest PRIVATE "${CMAKE_CURRENT_SOURCE_DIR}/src") + if(COREVIDEO_BUILD_PLUGIN AND WIN32) + add_executable(CoreVideoIsoTrackWriterTest tests/iso-track-writer-test.cpp + src/iso-track-writer.cpp src/iso-ffmpeg-pipe.cpp) + target_include_directories(CoreVideoIsoTrackWriterTest PRIVATE src) + target_link_libraries(CoreVideoIsoTrackWriterTest PRIVATE OBS::libobs) + add_executable(CoreVideoIsoIsolatedRecordingTest tests/iso-isolated-recording-test.cpp + src/iso-track-writer.cpp src/iso-ffmpeg-pipe.cpp src/iso-audio-reader.cpp) + target_include_directories(CoreVideoIsoIsolatedRecordingTest PRIVATE src) + target_link_libraries(CoreVideoIsoIsolatedRecordingTest PRIVATE OBS::libobs d3d11 d3dcompiler) + target_compile_definitions(CoreVideoIsoIsolatedRecordingTest PRIVATE NOMINMAX WIN32_LEAN_AND_MEAN) + if(WIN32) + target_compile_definitions(CoreVideoIsoTrackWriterTest PRIVATE NOMINMAX WIN32_LEAN_AND_MEAN) + target_link_libraries(CoreVideoIsoTrackWriterTest PRIVATE d3d11 d3dcompiler) + endif() + endif() + add_executable(CoreVideoIsoEncoderPlanTest tests/iso-encoder-plan-test.cpp ) + add_executable(CoreVideoIsoProviderPolicyTest tests/iso-provider-policy-test.cpp) + add_executable(CoreVideoIsoFeedSelectionTest tests/iso-feed-selection-test.cpp) + add_executable(CoreVideoIsoAudioRoutingTest tests/iso-audio-routing-test.cpp) + add_executable(CoreVideoIsoAudioReaderTest tests/iso-audio-reader-test.cpp src/iso-audio-reader.cpp) + target_include_directories(CoreVideoIsoAudioReaderTest PRIVATE src) + add_test(NAME CoreVideoIsoAudioReader COMMAND CoreVideoIsoAudioReaderTest) + target_include_directories(CoreVideoIsoAudioRoutingTest PRIVATE src) + add_test(NAME CoreVideoIsoAudioRouting COMMAND CoreVideoIsoAudioRoutingTest) + target_include_directories(CoreVideoIsoFeedSelectionTest PRIVATE src) + add_test(NAME CoreVideoIsoFeedSelection COMMAND CoreVideoIsoFeedSelectionTest) + target_include_directories(CoreVideoIsoProviderPolicyTest PRIVATE src) + add_test(NAME CoreVideoIsoProviderPolicy COMMAND CoreVideoIsoProviderPolicyTest) target_include_directories(CoreVideoIsoEncoderPlanTest PRIVATE "${CMAKE_CURRENT_SOURCE_DIR}/src" ) @@ -915,6 +946,9 @@ if(BUILD_TESTING) add_executable(CoreVideoTileTextureTest tests/tile-texture-test.cpp ) + add_executable(CoreVideoTileFrameReadTest tests/tile-frame-read-test.cpp) + target_include_directories(CoreVideoTileFrameReadTest PRIVATE src) + add_test(NAME CoreVideoTileFrameRead COMMAND CoreVideoTileFrameReadTest) target_include_directories(CoreVideoTileTextureTest PRIVATE "${CMAKE_CURRENT_SOURCE_DIR}/src" ) diff --git a/docs/CORE_PLUGIN_FUNCTIONALITY.md b/docs/CORE_PLUGIN_FUNCTIONALITY.md index 6444168..edf4b9b 100644 --- a/docs/CORE_PLUGIN_FUNCTIONALITY.md +++ b/docs/CORE_PLUGIN_FUNCTIONALITY.md @@ -580,15 +580,24 @@ There are no Companion actions for talkback yet. ## Auto ISO Recording -![CoreVideo ISO recording flow](images/iso-recording-flow.svg) - -ISO recording is controlled by the OBS plugin, not the engine. When enabled, +ISO recording is controlled by the OBS plugin. On macOS, configure and test a working FFmpeg executable in **Zoom ISO Recorder** first; the package does not bundle FFmpeg. -CoreVideo records one video file and one PCM WAV audio file per active source -segment. A new segment starts when the resolved participant or source resolution -changes. +CoreVideo records one continuous MP4 per Zoom participant ID during Record/Stop. +Each file contains H.264 video at 1920x1080, 30 fps and AAC stereo audio at 48 kHz. +ISO audio comes from a dedicated one-way subscription for that participant, +independent of the OBS source's Mix/Isolated setting. Recording subscriptions +do not remove participants from the Audience mix. A muted/silent participant +has silence in their MP4; other speakers are never substituted. +Choose feeds in the ISO Recorder before starting. Choices are remembered by +source UUID; no feeds are selected by default. Unrouted and unselected feeds do +not start recordings. A selected feed opens its file only when its first actual +audio or video arrives, so an offline route does not produce an empty recording. +Incoming video is scaled to fit with black bars when needed. Zoom resolution +changes, duplicate OBS sources, camera gaps, and source reassignment do not +restart that participant's encoder or create another file. A new Zoom ID after +a rejoin is treated as a new participant; display names are not identity keys. Requirements: @@ -612,11 +621,13 @@ dock. The panel provides: requested encoder, actual encoder, and fallback state. - **Also start/stop OBS program recording** toggle. - **Start ISO Recording** and **Stop ISO Recording** buttons. +- **Feeds to record** checkboxes, preserved through routing refreshes and OBS + restarts. Selection is fixed for the recording run; stop to change it. - Live status showing idle/recording and active session count. - Active session table with source, participant, resolution, video frame count, - audio chunk count, current video/audio file paths, and FFmpeg error details. + audio chunk count, combined file path, and FFmpeg error details. - Recently completed sessions remain in the table after stop so operators can - confirm completed MP4/WAV outputs before opening the folder. + confirm completed MP4 outputs before opening the folder. The panel uses the same `ZoomIsoRecorder` backend as the TCP and OSC APIs. It persists the output folder, FFmpeg path, and program-recording toggle in OBS @@ -626,9 +637,14 @@ less than 2 GB free and warns below 10 GB free. TCP start example: ```json -{"cmd":"iso_recording_start","output_dir":"C:/Recordings/CoreVideo","ffmpeg_path":"ffmpeg","record_program":true} +{"cmd":"iso_recording_start","output_dir":"C:/Recordings/CoreVideo","ffmpeg_path":"ffmpeg","record_program":true,"source_uuids":["OUTPUT_SOURCE_UUID"]} ``` +`source_uuids` selects routed CoreVideo output sources. Omitting it uses the +feed choices saved in the ISO panel; an empty selection is an error. OSC start +also uses those saved choices. Multiple selected feeds for one participant +share a single participant MP4. + TCP status example: ```json @@ -651,33 +667,25 @@ OSC equivalents: Output files are written as: -- `*.mp4` for encoded I420 video through FFmpeg using the selected H.264 - encoder -- `*.wav` for matching PCM audio -- `*.ffmpeg.log` beside them, holding FFmpeg's own account of the session. Read - this first when a file is missing or truncated. +- `*.mp4` containing H.264 video and AAC audio for one participant. +- `*.mp4.ffmpeg.log` containing encoder diagnostics. ### Timing -Both files are paced to real elapsed time against the same clock, so each is -individually accurate and the two stay in sync with each other. - -That is not free, and it is worth knowing why. Raw video carries no per-frame -timestamps, and Zoom's per-source delivery fluctuates between roughly 10 and -60 fps with conditions outside CoreVideo's control - so frames are paced to a -fixed cadence before they reach FFmpeg, duplicating the held frame to backfill a -stall and dropping excess from a burst. Audio has the mirror-image problem for a -different reason: Zoom only calls audio back for someone currently making sound, -so silence is backfilled across every gap or the WAV shrinks by the total silent -duration. Both were wrong before v0.1.42/v0.1.43 - a source averaging 15 fps -recorded under a declared 30 fps finished in about half the real duration, and an -over-eager first gap-fill briefly doubled every WAV. If you see either symptom, -check your version first. - -The hardware encoder fallback chain is NVENC -> QSV -> AMF -> libx264, and it -walks the whole chain: a source demoted off NVENC on a machine with no working -QSV or AMF runtime now reaches libx264, the tier with no hardware dependency to -fail on. +A worker per participant uses a monotonic clock to emit 30 video frames and +48,000 audio samples per second. It holds the last picture during video gaps +(or black before the first frame) and inserts silence during audio gaps. Input +resolution and audio format changes are conformed on the worker without +restarting the output. Files start on first media delivery. Participants first +encountered later have a `start_offset_ms` in status. + +Timestamped video and PCM audio travel over one internal Matroska pipe to +FFmpeg, which writes a single fragmented MP4. There are no separate temporary +media files and no merge step at Stop. Producer buffers and encoder queues are +bounded. An encoder failure is reported on that participant; it does not start +an automatic replacement file. Hardware availability is tested by encoding an +actual frame before recording starts. Automatic placement uses working NVENC, +QSV, AMF, then libx264, accounting for OBS's NVENC sessions. When `record_program` is true, CoreVideo also starts the normal OBS program recording and stops it when ISO recording stops, but only if CoreVideo started diff --git a/docs/iso-feed-selection-validation-2026-09-14.md b/docs/iso-feed-selection-validation-2026-09-14.md new file mode 100644 index 0000000..6b39baf --- /dev/null +++ b/docs/iso-feed-selection-validation-2026-09-14.md @@ -0,0 +1,30 @@ +# ISO feed selection + +Development build: `v0.1.47-dev.3`. Includes participant A/V recording and the +pending Tiles frame-handoff fix. Not installed while the operator is in a meeting. + +The ISO panel now lists output feeds with checkboxes, defaulting to none. +Selections are stored by source UUID, survive live routing refreshes, and are +locked for the recording run. Unrouted/unsupported entries cannot be newly +selected; a saved choice is retained if its route temporarily disappears. +Start requires at least one selected routed feed. Disk and encoder estimates +count only selected feeds, deduplicating fixed participant routes. + +The recorder validates selection independently of the UI and filters both +audio and video callbacks. Starting no longer pre-opens writers for configured +participant IDs. Files open only on media delivery from a selected route. +Once opened, normal gap handling and one-file-per-participant ownership remain. + +TCP `iso_recording_start` accepts `source_uuids`, an array of output UUIDs; +omission uses saved panel choices, and explicit empty selection is rejected. +OSC start uses saved choices. Neither API falls back to recording every source. +Status exposes `selected_source_uuids`; the panel shows an armed/waiting state +when recording is enabled but no selected feed has delivered media. + +Windows full build succeeded and all 70 CTest regressions passed. The new +selection-policy regression covers selected/unselected, unrouted, unsupported, +empty selection, and empty UUID cases. Live UI and recording acceptance remain +pending the operator's meeting ending and installation: select two routed +feeds, leave others unchecked, verify only selected participants create files, +and confirm offline/unrouted feeds create none. Check saved choices after OBS +restart and a routing refresh. diff --git a/docs/iso-isolated-audio-validation-2026-09-15.md b/docs/iso-isolated-audio-validation-2026-09-15.md new file mode 100644 index 0000000..b6af1cb --- /dev/null +++ b/docs/iso-isolated-audio-validation-2026-09-15.md @@ -0,0 +1,36 @@ +# Dedicated ISO audio validation + +Build `v0.1.47-dev.4` includes the feed picker, Tiles handoff fix, and dedicated +participant audio recording. Both plugin and ZoomObsEngine must be installed. + +Root cause: ZoomSource forwarded its embedded audio to the ISO writer. Sources +configured as Mix received mixed PCM labelled with the configured participant +ID by the engine. A participant ID alone therefore did not prove isolation. +Decoded audio envelopes from the last show's participant files were strongly +correlated (two pairs 0.9956 and 0.9467), consistent with this path. + +ISO now owns an audio-only subscription per recorded participant. Source PCM +is never passed to the writer. The engine forces these subscriptions to receive +only that participant's one-way audio and excludes them from the set of output +routes that claim participants away from Audience audio. Per-slot attribution +is validated again at the recorder's dedicated SHM reader. + +Validation completed: + +- Full Windows build and 72 CTest regressions passed. +- Routing regression rejects mix and other participants while retaining normal + isolated, meeting-mix, and Audience behavior. +- Real SHM reader regression retains first/coalesced buffers, does not duplicate + drained buffers, clears notifications, and rejects another participant ID. +- End-to-end synthetic alternating speakers through the production routing + policy, real shared-memory reader, resampler, and FFmpeg MP4 writers passed. + Decoded one-second RMS for participant 1: 8490.78, 0, 8485.57, 0. + Participant 2: 0, 8482.77, 0, 8482.81. Non-speaking intervals were digital silence. + Both files decoded completely without errors. + +Reproduce: run `CoreVideoIsoIsolatedRecordingTest ` in an empty directory, +then `python tests/verify-iso-isolated.py ` from the repo root. +These are synthetic SDK-input tests, not a live Zoom meeting. Live acceptance +still requires two people alternating speech and checking their respective MP4s. +Existing mixed recordings are unchanged; this fix cannot recover original +isolated stems that were never captured. diff --git a/docs/iso-participant-av-validation-2026-09-14.md b/docs/iso-participant-av-validation-2026-09-14.md new file mode 100644 index 0000000..790e7f2 --- /dev/null +++ b/docs/iso-participant-av-validation-2026-09-14.md @@ -0,0 +1,77 @@ +# Continuous participant ISO recording validation + +Development build: `v0.1.47-dev`. This change has not been installed into the +running OBS instance or published as a release. + +## Change + +The previous recorder opened separate video MP4 and audio WAV files and keyed +video lifetime to source/resolution changes. The replacement owns one writer +per Zoom participant ID for the Record/Stop interval. Duplicate source UUIDs +share that writer. Input size changes only reconfigure the scaler. + +Every participant MP4 contains 1920x1080/30 H.264 and 48 kHz stereo AAC. +Aspect ratio is preserved with black bars. A worker per participant performs +scaling, resampling and timestamped A/V transport over one FFmpeg stdin pipe. +Video gaps hold the picture; audio gaps contain silence. A participant with no +media callbacks still gets black video and silence. No merge step or automatic +replacement segment is created. Diagnostic text logs remain beside the MP4. + +The Windows GPU scaler and aspect-fit geometry were adapted from CoreVideo +Pro's `D3DIsoFrameConformer.h` and `IsoFrameConform.h`. This recorder uses planar +I420 rather than Pro's NV12 output, and preserves the engine's full-range +BT.709 pixels. Other platforms use the OBS video scaler. + +## Completed checks + +- Full Windows plugin/engine build succeeded; 68 CTest regressions passed. +- Real worker fixtures: two simultaneous libx264 participant encoders and eight + simultaneous NVENC participant encoders on this RTX 4090 host. Each run also + had a silent/no-video libx264 writer and a deliberately invalid encoder process. +- Every participant produced one MP4 with both tracks, 180 decoded video frames + over six seconds, fixed 1920x1080/30 video, stereo 48 kHz AAC, and no decode errors. +- Inputs changed between 640x360, 1280x720, 1920x1080, and 640x480, followed by a + camera-off second and recovery. Decoded pixels verified pillarboxing, recovery + to full canvas, and separate U/V colors through scaling and native-size input. +- Input audio changed from 32 kHz mono to 48 kHz stereo, with callbacks withheld + outside test tones. Clustered callback timestamps verified that PCM placement + never rewinds or overwrites a previously received batch. +- Decoded flashes/tones stayed within 20 ms on both encoders; all six 100 ms + audio tones retained their duration. No late audio samples were reported. +- No-input tracks decoded as continuous black and silent AAC. The failed encoder + reported an error without stopping the healthy recordings. +- Provider-policy regression covers duplicate source suppression, monotonic + timestamps, alternate takeover after a gap, and source removal. + +Validation found and corrected startup history loss, fragmented MP4 startup +offset with B-frames, input color metadata, planar chroma conversion, and audio +rewinding on bunched arrival timestamps. + +## Reproduce + +Build `CoreVideoIsoTrackWriterTest` with the Windows plugin dependencies. Run +from an empty directory, with OBS's runtime DLL directory on PATH: + +```powershell +& $writerTest $ffmpeg h264_nvenc 8 +python tests/verify-iso-av.py $ffmpegBin $fixtureDirectory +``` + +Run the verifier from the repository root. Use `libx264 2` for the CPU fixture. +The test refuses to overwrite existing MP4s. The verifier checks decoded media, +not only FFmpeg exit status. Local final results are in `build-iso/av-burst-cpu` +and `build-iso/av-burst-nvenc`; regression results are in `build-iso/av-ctest.log`. + +## Remaining release validation + +- Live Zoom soak, including repeated resolution changes, mute/camera toggles, + duplicate OBS sources, source reassignment/removal, Stop, and a new Record run. +- Live lip-sync check, sustained resource usage at the intended participant + count, and playback/import in the production editor. +- Hosted macOS/Linux builds and platform recording checks; local validation + above is Windows only. + +The exposed identity is Zoom's numeric participant ID. A rejoin that receives a +new ID creates a new participant file; matching by display name would risk +combining different people. Known fixed participants share the Record epoch; +late participants expose their offset through `start_offset_ms` status. diff --git a/docs/tiles-flash-validation-2026-09-14.md b/docs/tiles-flash-validation-2026-09-14.md new file mode 100644 index 0000000..34cc6ac --- /dev/null +++ b/docs/tiles-flash-validation-2026-09-14.md @@ -0,0 +1,34 @@ +# Tiles pending-frame handoff fix + +Build: `v0.1.47-dev.2`, including the previous participant A/V ISO changes. + +The reported symptom was brightness/color flashing inside faces, possibly at +speaker changes. The existing color-range normalization is present in the +installed engine and its logs show it operating. Speaker/roster updates do not +reassign a tile when its participant list is unchanged. + +Code inspection found a definite failure path in the Tiles frame handoff: + +1. A valid frame is waiting for the render thread (`has_frame=true`). +2. Another callback copies SHM pixels directly into that pending frame. +3. The shared-memory sequence check rejects the new copy, or its dimensions + are unsuitable for I420. The callback returns without updating metadata. +4. The render thread still sees the previous ready flag, dimensions and frame + generation, but the underlying pixels have been overwritten. On a resize, + even the Y/U/V plane offsets can now describe the wrong buffer. + +The fix reads into a separate reusable candidate buffer and commits pixels only +after a successful read with even dimensions. Failed reads leave the pending +valid picture untouched. It adds one retained input buffer per tile, no extra +full-frame copy, and no changes to subscriptions or color conversion. + +Validation: `CoreVideoTileFrameRead` failed with direct writes to the pending +buffer, then passed after transactional publication. It covers rejected resize, +successful resize recovery, and 100 subsequent rejected reads. The full Windows +build and all 69 CTest regressions passed. + +The exact connection to the observed live flashes still needs confirmation on +the new build. The previous build did not log rejected Tiles reads. The new +build logs `Tiles kept last valid frame` at the first rejection and each 300th +rejection so a live test can correlate these events. Verify repeated speaker +changes and resolution changes without visible brightness/color pops. diff --git a/engine/src/engine-audio.cpp b/engine/src/engine-audio.cpp index 6ba0c82..e5373e8 100644 --- a/engine/src/engine-audio.cpp +++ b/engine/src/engine-audio.cpp @@ -1,6 +1,7 @@ #include "engine-audio.h" #include "engine-writer.h" #include "tile-clock-log.h" +#include "../../src/iso-audio-routing.h" #if __has_include() #include #else @@ -15,9 +16,10 @@ bool EngineAudio::init(IpcFd e2p_fd, const std::string &source_uuid, uint32_t participant_id, bool isolate_audio, - bool audience_audio) + bool audience_audio, bool recording_only) { // isolate wins if both are set — defensive, the plugin UI prevents this. + if (recording_only) isolate_audio = true; if (isolate_audio) audience_audio = false; m_e2p_fd = e2p_fd; @@ -47,6 +49,7 @@ bool EngineAudio::init(IpcFd e2p_fd, it->second->isolate_audio = isolate_audio; it->second->audience_audio = audience_audio; } + m_targets.at(source_uuid)->recording_only = recording_only; } if (!m_raw_media_active) { @@ -380,7 +383,8 @@ void EngineAudio::onMixedAudioRawDataReceived(AudioRawData *data) for (auto &entry : m_targets) { if (!entry.second) continue; // Skip isolate AND audience targets — both receive only one-way audio. - if (entry.second->isolate_audio || entry.second->audience_audio) continue; + if (!audio_receives_mix(entry.second->isolate_audio, entry.second->audience_audio, + entry.second->recording_only)) continue; output_audio_frame(*entry.second, entry.first, data, "audio_frame_received"); } @@ -397,9 +401,10 @@ void EngineAudio::onOneWayAudioRawDataReceived(AudioRawData *data, uint32_t user // determine whether this user is "claimed" by any isolate target. bool claimed_by_isolate = false; for (auto &entry : m_targets) { - if (!entry.second || !entry.second->isolate_audio) continue; - if (entry.second->participant_id != user_id) continue; - claimed_by_isolate = true; + if (!entry.second || !audio_receives_participant(entry.second->participant_id,user_id, + entry.second->isolate_audio,entry.second->recording_only)) continue; + if (audio_claims_audience(entry.second->isolate_audio,entry.second->recording_only)) + claimed_by_isolate = true; output_audio_frame(*entry.second, entry.first, data, "audio_one_way_frame_received"); } diff --git a/engine/src/engine-audio.h b/engine/src/engine-audio.h index 92c8787..84292a5 100644 --- a/engine/src/engine-audio.h +++ b/engine/src/engine-audio.h @@ -34,7 +34,7 @@ class EngineAudio : public ZOOMSDK::IZoomSDKAudioRawDataDelegate { const std::string &source_uuid, uint32_t participant_id, bool isolate_audio, - bool audience_audio); + bool audience_audio, bool recording_only = false); bool retry_subscribe(const std::string &reason); void set_raw_media_active(bool active); void reset_subscription(const std::string &reason); @@ -60,6 +60,7 @@ class EngineAudio : public ZOOMSDK::IZoomSDKAudioRawDataDelegate { uint32_t participant_id = 0; bool isolate_audio = false; bool audience_audio = false; + bool recording_only = false; ShmRegion shm; // The generation the CURRENT region was created under; part of the name // from gen 2 and published with every audio event. A record of what was diff --git a/engine/src/main.cpp b/engine/src/main.cpp index 0dd3dfd..505dae8 100644 --- a/engine/src/main.cpp +++ b/engine/src/main.cpp @@ -1996,7 +1996,8 @@ int main() line.find(R"("audience_audio":true)") != std::string::npos; if (is_valid_source_uuid(uuid)) { EngineAudio::instance().init(e2p, uuid, pid, - isolate_audio, audience_audio); + isolate_audio, audience_audio, + line.find(R"("recording_only":true)") != std::string::npos); } } else if (command == IpcCommand::Subscribe) { diff --git a/package-lock.json b/package-lock.json index 9e117ba..2d5c5b1 100644 --- a/package-lock.json +++ b/package-lock.json @@ -41,9 +41,9 @@ } }, "node_modules/@cloudflare/workerd-darwin-64": { - "version": "1.20260903.1", - "resolved": "https://registry.npmjs.org/@cloudflare/workerd-darwin-64/-/workerd-darwin-64-1.20260903.1.tgz", - "integrity": "sha512-FG+4mGxAXhKiL/1temH42alevIkumtYXNidhTa//3yULpzux6APw5UNVocI/vCQ1yG1YOEZRfbVy8lyuipM9MQ==", + "version": "1.20260911.1", + "resolved": "https://registry.npmjs.org/@cloudflare/workerd-darwin-64/-/workerd-darwin-64-1.20260911.1.tgz", + "integrity": "sha512-785eaY1bkR1cm4Z/PCUeteZYmTMe6lre2zz63/GdGGimsoMsKxgl4brFPRukim8iv28EyD1XoCB/VPYF20BERA==", "cpu": [ "x64" ], @@ -58,9 +58,9 @@ } }, "node_modules/@cloudflare/workerd-darwin-arm64": { - "version": "1.20260903.1", - "resolved": "https://registry.npmjs.org/@cloudflare/workerd-darwin-arm64/-/workerd-darwin-arm64-1.20260903.1.tgz", - "integrity": "sha512-o241VefnjG8eG+kGap5CjgV4zOT5UaAmS3OR1VZCpNj7vkXGxvp9KftKvtQgcCsIqJKaKx+7Xd7xq0L1DjkfPQ==", + "version": "1.20260911.1", + "resolved": "https://registry.npmjs.org/@cloudflare/workerd-darwin-arm64/-/workerd-darwin-arm64-1.20260911.1.tgz", + "integrity": "sha512-WU4bFqEN0H7ndGWxoedegv95DmNVBtv0ncXcHG9nYFTUI78sxEb0qoT3U6Ga4hyBkzsJFBX/zvVBIGX3qKldGA==", "cpu": [ "arm64" ], @@ -75,9 +75,9 @@ } }, "node_modules/@cloudflare/workerd-linux-64": { - "version": "1.20260903.1", - "resolved": "https://registry.npmjs.org/@cloudflare/workerd-linux-64/-/workerd-linux-64-1.20260903.1.tgz", - "integrity": "sha512-/VEvvtQ/XKf6HlBbg6FbvpwcpfYUcx6Fv6RkASY8DyEmUyuJ8rc7Qxil83ClRFoBzz/GY0BV94UZ6F+wers/hg==", + "version": "1.20260911.1", + "resolved": "https://registry.npmjs.org/@cloudflare/workerd-linux-64/-/workerd-linux-64-1.20260911.1.tgz", + "integrity": "sha512-0Y2gy62oxQxWa38qinSPE6zNL5+JmumJtDY9AWW1HB8KHuATxN71o5MGzmVFfB8PwZsiHfUd2Sv7O22krCOrhw==", "cpu": [ "x64" ], @@ -92,9 +92,9 @@ } }, "node_modules/@cloudflare/workerd-linux-arm64": { - "version": "1.20260903.1", - "resolved": "https://registry.npmjs.org/@cloudflare/workerd-linux-arm64/-/workerd-linux-arm64-1.20260903.1.tgz", - "integrity": "sha512-OWhihGC6KoTXF4u2C1AonfpgXeM4/7p/1IXuALqXESmFUpLLP5gZhRzjSk/gWW+mrCZDfSrvnjifl+lRqselbA==", + "version": "1.20260911.1", + "resolved": "https://registry.npmjs.org/@cloudflare/workerd-linux-arm64/-/workerd-linux-arm64-1.20260911.1.tgz", + "integrity": "sha512-kttNPnx1r2lCqFUoMH62z7CqGV+j4QBbw5fdtaz4pzOrzBv0AWkNATt7onFUe+SwP8zhcepMtbm2F4kKzTf6VA==", "cpu": [ "arm64" ], @@ -109,9 +109,9 @@ } }, "node_modules/@cloudflare/workerd-windows-64": { - "version": "1.20260903.1", - "resolved": "https://registry.npmjs.org/@cloudflare/workerd-windows-64/-/workerd-windows-64-1.20260903.1.tgz", - "integrity": "sha512-soPMF9/aMHlHKK7M0vq5HrRRicPbnZO1F6ZZ7JWN8EltxWJU5L7CEq7CxjO878DzyPx7gYvqBFy3SVgdZKZjMQ==", + "version": "1.20260911.1", + "resolved": "https://registry.npmjs.org/@cloudflare/workerd-windows-64/-/workerd-windows-64-1.20260911.1.tgz", + "integrity": "sha512-5iO/YfoBDOgO3CrHdkiiVP8SL3O2jC+c6Ux3d378TSPKLhU5+CgHjtE/ZSodWQrzr4FzFRqdW8S7n5nbyD1MHQ==", "cpu": [ "x64" ], @@ -602,9 +602,9 @@ } }, "node_modules/@img/sharp-darwin-arm64": { - "version": "0.35.2", - "resolved": "https://registry.npmjs.org/@img/sharp-darwin-arm64/-/sharp-darwin-arm64-0.35.2.tgz", - "integrity": "sha512-eEieHsMksAW4IiO5NzauESRl2D2qz3J/kwUxUrSfV06A93eEaRfMpHXyUb1mAqrR7i8U9A0GRqE9pjn6u1Jjpg==", + "version": "0.35.4", + "resolved": "https://registry.npmjs.org/@img/sharp-darwin-arm64/-/sharp-darwin-arm64-0.35.4.tgz", + "integrity": "sha512-Uhfl4V4lhP2nbUVF9+hyH1+luj86f1gUFeo8ALYxFoULoU+G87D43BfeMP8XHsk9boxAnCY/bf2EHwhA7MuGsA==", "cpu": [ "arm64" ], @@ -621,13 +621,13 @@ "url": "https://opencollective.com/libvips" }, "optionalDependencies": { - "@img/sharp-libvips-darwin-arm64": "1.3.1" + "@img/sharp-libvips-darwin-arm64": "1.3.3" } }, "node_modules/@img/sharp-darwin-x64": { - "version": "0.35.2", - "resolved": "https://registry.npmjs.org/@img/sharp-darwin-x64/-/sharp-darwin-x64-0.35.2.tgz", - "integrity": "sha512-BaktuGPCeHJMARpodR8jK4uKiZrPAy9WrfQW0sdI37clracq8Bp01AYS3SZgi5FS/y5twa9t4+LIuuxQjqRrWw==", + "version": "0.35.4", + "resolved": "https://registry.npmjs.org/@img/sharp-darwin-x64/-/sharp-darwin-x64-0.35.4.tgz", + "integrity": "sha512-hWniXY3bG5qKpkKrAwPe4y+VTPmf086YQAnkxWh7uA1YrlRouWGa0M0Mxj3ZjnXFkv7/TD1bTy9lGUK26vRvWw==", "cpu": [ "x64" ], @@ -644,13 +644,13 @@ "url": "https://opencollective.com/libvips" }, "optionalDependencies": { - "@img/sharp-libvips-darwin-x64": "1.3.1" + "@img/sharp-libvips-darwin-x64": "1.3.3" } }, "node_modules/@img/sharp-freebsd-wasm32": { - "version": "0.35.2", - "resolved": "https://registry.npmjs.org/@img/sharp-freebsd-wasm32/-/sharp-freebsd-wasm32-0.35.2.tgz", - "integrity": "sha512-YoAxdnd8hPUkvLHd3bWY+YA8nw3xM/RyRopYucNsWHVSan8NLVM3X2volsfoRDcXdUJPg6tXahSd7HXPK7lRnw==", + "version": "0.35.4", + "resolved": "https://registry.npmjs.org/@img/sharp-freebsd-wasm32/-/sharp-freebsd-wasm32-0.35.4.tgz", + "integrity": "sha512-lIsKw/BU+kjB4eZjxrYrZmwOJYi3Ajrv66iAlBmUPyKc3HpnloevB1g3wxGD9P/5BbQ1brBGl65VRRrCvQDEqA==", "dev": true, "license": "Apache-2.0", "optional": true, @@ -658,7 +658,7 @@ "freebsd" ], "dependencies": { - "@img/sharp-wasm32": "0.35.2" + "@img/sharp-wasm32": "0.35.4" }, "engines": { "node": ">=20.9.0" @@ -668,9 +668,9 @@ } }, "node_modules/@img/sharp-libvips-darwin-arm64": { - "version": "1.3.1", - "resolved": "https://registry.npmjs.org/@img/sharp-libvips-darwin-arm64/-/sharp-libvips-darwin-arm64-1.3.1.tgz", - "integrity": "sha512-4V/M3roRMTYjiwZY9IOVQOE8OyeCxFAkYmyZDrZl51uOKjibm3oeEJ4WAmLxutAfzFbC9jqUiPs2gbnGflH+7g==", + "version": "1.3.3", + "resolved": "https://registry.npmjs.org/@img/sharp-libvips-darwin-arm64/-/sharp-libvips-darwin-arm64-1.3.3.tgz", + "integrity": "sha512-suTBPTDGrI9WodccaDdwZItTSaBYASlBk1NSfElSHrUfzu3szG6lvIF58+WiFvnfzuK8ZBFS5zE00PxqxnRiPg==", "cpu": [ "arm64" ], @@ -685,9 +685,9 @@ } }, "node_modules/@img/sharp-libvips-darwin-x64": { - "version": "1.3.1", - "resolved": "https://registry.npmjs.org/@img/sharp-libvips-darwin-x64/-/sharp-libvips-darwin-x64-1.3.1.tgz", - "integrity": "sha512-c0/DxItpJv2+dGhgycJBBgotdqruGYDvA79drdh0MD1dFpy7JzJ/PlXwi1H4rFf0eTy8tgbI91aHDnZIceY3jQ==", + "version": "1.3.3", + "resolved": "https://registry.npmjs.org/@img/sharp-libvips-darwin-x64/-/sharp-libvips-darwin-x64-1.3.3.tgz", + "integrity": "sha512-FVJZ5mITMobmXIz/hPDTw0EintTW5H3WfrxwLqEqjiIihlu+hVRyGrFQ60xl0Lxn7Bt3zdpevPaQi0HEzqz9fw==", "cpu": [ "x64" ], @@ -702,16 +702,13 @@ } }, "node_modules/@img/sharp-libvips-linux-arm": { - "version": "1.3.1", - "resolved": "https://registry.npmjs.org/@img/sharp-libvips-linux-arm/-/sharp-libvips-linux-arm-1.3.1.tgz", - "integrity": "sha512-aGGy9aWzXgHBG7HNyQPWorZthlp7+x6fDRoPAQbGO3ThcttuTyKIx3NuSHb6zb4gBNq6/yNn9f1cy9nFKS/Vmg==", + "version": "1.3.3", + "resolved": "https://registry.npmjs.org/@img/sharp-libvips-linux-arm/-/sharp-libvips-linux-arm-1.3.3.tgz", + "integrity": "sha512-3rbU4vqXXc3hY/OiXdl52xZvT0F1yEngWfvqudtPJg/KkyiaQw2DRsFrNzpmLvfavbwOq3qXn36GP8obHRULQA==", "cpu": [ "arm" ], "dev": true, - "libc": [ - "glibc" - ], "license": "LGPL-3.0-or-later", "optional": true, "os": [ @@ -722,16 +719,13 @@ } }, "node_modules/@img/sharp-libvips-linux-arm64": { - "version": "1.3.1", - "resolved": "https://registry.npmjs.org/@img/sharp-libvips-linux-arm64/-/sharp-libvips-linux-arm64-1.3.1.tgz", - "integrity": "sha512-JznefmcK9j1JKPz8AkQDh89kjojubyfOasWBPKfzMIhPwsgDy9evpE/naJTXXXmghS1iFwR8u/kTwh/I2/+GCw==", + "version": "1.3.3", + "resolved": "https://registry.npmjs.org/@img/sharp-libvips-linux-arm64/-/sharp-libvips-linux-arm64-1.3.3.tgz", + "integrity": "sha512-0DaL0A6Xu6sQSQFwe4iVCrKWU2cCTItnRsYsCdxAMm9NF6twAA9BKnoqy4hqz4+azQ0JHuA26qiUKsf1XJ/v5A==", "cpu": [ "arm64" ], "dev": true, - "libc": [ - "glibc" - ], "license": "LGPL-3.0-or-later", "optional": true, "os": [ @@ -742,16 +736,13 @@ } }, "node_modules/@img/sharp-libvips-linux-ppc64": { - "version": "1.3.1", - "resolved": "https://registry.npmjs.org/@img/sharp-libvips-linux-ppc64/-/sharp-libvips-linux-ppc64-1.3.1.tgz", - "integrity": "sha512-1EkwGNCZk6iWNCMWqrvdJ+r1j0PT1zIz60CNPhYnJlK/zyeWqlsPZIe+ocBVqPF8k/Ssee/NCk+tE9Ryrko6ng==", + "version": "1.3.3", + "resolved": "https://registry.npmjs.org/@img/sharp-libvips-linux-ppc64/-/sharp-libvips-linux-ppc64-1.3.3.tgz", + "integrity": "sha512-cdn1OvUBwsXhbC0zSzJnNzf5MZ/mTrobawDvNXBTxe8VtqKAm0sRuEY2Evzovb/w9JMk4TvRxqt1mekSuJz64w==", "cpu": [ "ppc64" ], "dev": true, - "libc": [ - "glibc" - ], "license": "LGPL-3.0-or-later", "optional": true, "os": [ @@ -762,16 +753,13 @@ } }, "node_modules/@img/sharp-libvips-linux-riscv64": { - "version": "1.3.1", - "resolved": "https://registry.npmjs.org/@img/sharp-libvips-linux-riscv64/-/sharp-libvips-linux-riscv64-1.3.1.tgz", - "integrity": "sha512-Ilays+w2bXdnxzxtQdmXR62u8o8GYa3eL4+Gr+1KiE4xperMZUslRaVPJwwPkzlHEjGfXAfRVAa/7CYCtSqsBw==", + "version": "1.3.3", + "resolved": "https://registry.npmjs.org/@img/sharp-libvips-linux-riscv64/-/sharp-libvips-linux-riscv64-1.3.3.tgz", + "integrity": "sha512-HjPVx7yKz+0lqdhDlTw1tt90wamBoxhiXpvl1XZpJLiHH4RCJ5yDTqH+VlYPv2fwFs89JFw4c1IexYOcQUi4IQ==", "cpu": [ "riscv64" ], "dev": true, - "libc": [ - "glibc" - ], "license": "LGPL-3.0-or-later", "optional": true, "os": [ @@ -782,16 +770,13 @@ } }, "node_modules/@img/sharp-libvips-linux-s390x": { - "version": "1.3.1", - "resolved": "https://registry.npmjs.org/@img/sharp-libvips-linux-s390x/-/sharp-libvips-linux-s390x-1.3.1.tgz", - "integrity": "sha512-VfBwVHQTbRoj4XlpA/KLZ7ltgMpz+4WSejFzQ+GnoImjo1PtEJ59QB2qR1xQEeRPYIkNrPIm2L4cICMvz4C2ew==", + "version": "1.3.3", + "resolved": "https://registry.npmjs.org/@img/sharp-libvips-linux-s390x/-/sharp-libvips-linux-s390x-1.3.3.tgz", + "integrity": "sha512-neWLh+3yCNThxnfy3c4BbVBeGgt9aftno+XbT56iK28RgeDs3UOFWviLWlUu0bArYVYJaFDK+RRohbicUNCm8Q==", "cpu": [ "s390x" ], "dev": true, - "libc": [ - "glibc" - ], "license": "LGPL-3.0-or-later", "optional": true, "os": [ @@ -802,16 +787,13 @@ } }, "node_modules/@img/sharp-libvips-linux-x64": { - "version": "1.3.1", - "resolved": "https://registry.npmjs.org/@img/sharp-libvips-linux-x64/-/sharp-libvips-linux-x64-1.3.1.tgz", - "integrity": "sha512-+c8ukgwU62DS54nCAjw7keOfHUkmr0B5QHEdcOqRnodF/MNXJbVI8Eopoj4B/0H8Asr65I+A4Amrn7a85/md6A==", + "version": "1.3.3", + "resolved": "https://registry.npmjs.org/@img/sharp-libvips-linux-x64/-/sharp-libvips-linux-x64-1.3.3.tgz", + "integrity": "sha512-4vKmvAst9nrowcqquKFAyZJUDolUaIp8uRiN0mWFguJ1IplC9/pitXtlnnlU4aa/eJw3J7i67V+pwUL+wZGdsA==", "cpu": [ "x64" ], "dev": true, - "libc": [ - "glibc" - ], "license": "LGPL-3.0-or-later", "optional": true, "os": [ @@ -822,16 +804,13 @@ } }, "node_modules/@img/sharp-libvips-linuxmusl-arm64": { - "version": "1.3.1", - "resolved": "https://registry.npmjs.org/@img/sharp-libvips-linuxmusl-arm64/-/sharp-libvips-linuxmusl-arm64-1.3.1.tgz", - "integrity": "sha512-qlKb/pwbkAi1WMsJrYHk7CuDrd12s27U2QnRhFYUoJNrRCmkosMTttuRFat/DDB3IlDm5qE1TJgZ4JDnHX8Ldw==", + "version": "1.3.3", + "resolved": "https://registry.npmjs.org/@img/sharp-libvips-linuxmusl-arm64/-/sharp-libvips-linuxmusl-arm64-1.3.3.tgz", + "integrity": "sha512-Y9kQaLMuNoB0bPYOOdcZMaseNrFpPodIWWMrx+CZyydf2xn68j9WYc6sWWRrDwNkzCQjKYfc68L7jKjGlHMibw==", "cpu": [ "arm64" ], "dev": true, - "libc": [ - "musl" - ], "license": "LGPL-3.0-or-later", "optional": true, "os": [ @@ -842,16 +821,13 @@ } }, "node_modules/@img/sharp-libvips-linuxmusl-x64": { - "version": "1.3.1", - "resolved": "https://registry.npmjs.org/@img/sharp-libvips-linuxmusl-x64/-/sharp-libvips-linuxmusl-x64-1.3.1.tgz", - "integrity": "sha512-yO21HwoUVLN8Qa+/SBjQLMYwBWAVJjeGPNe+hc0OUeMeifEtJqu5a1c4HayE1nNpDih9y3/KkoltfkDodmKAlg==", + "version": "1.3.3", + "resolved": "https://registry.npmjs.org/@img/sharp-libvips-linuxmusl-x64/-/sharp-libvips-linuxmusl-x64-1.3.3.tgz", + "integrity": "sha512-fj8Mv0HHfD1Rr+4I68+3agJynxDWtBFgicTbSOb9Bke6pIwzGcJ+RX/yHjmiEGFMCavY/dxvem7MyNaJF+wDiw==", "cpu": [ "x64" ], "dev": true, - "libc": [ - "musl" - ], "license": "LGPL-3.0-or-later", "optional": true, "os": [ @@ -862,16 +838,13 @@ } }, "node_modules/@img/sharp-linux-arm": { - "version": "0.35.2", - "resolved": "https://registry.npmjs.org/@img/sharp-linux-arm/-/sharp-linux-arm-0.35.2.tgz", - "integrity": "sha512-SE4kzF2mepn6z+6E7L6lsV8FzuLL6IPQdyX8ZiwROAG/G8td+hP/m7FsFPwidtrF19gvajuC9l6TxAVcsA4S7A==", + "version": "0.35.4", + "resolved": "https://registry.npmjs.org/@img/sharp-linux-arm/-/sharp-linux-arm-0.35.4.tgz", + "integrity": "sha512-7OAS8gI0EReKGVN2HssHlM6umJgxF5VI3xN0p9FA91p/YO+ou5hiNghLdZ5BEHztwaaK5+bLKRf8x/o2L2nk9A==", "cpu": [ "arm" ], "dev": true, - "libc": [ - "glibc" - ], "license": "Apache-2.0", "optional": true, "os": [ @@ -884,20 +857,17 @@ "url": "https://opencollective.com/libvips" }, "optionalDependencies": { - "@img/sharp-libvips-linux-arm": "1.3.1" + "@img/sharp-libvips-linux-arm": "1.3.3" } }, "node_modules/@img/sharp-linux-arm64": { - "version": "0.35.2", - "resolved": "https://registry.npmjs.org/@img/sharp-linux-arm64/-/sharp-linux-arm64-0.35.2.tgz", - "integrity": "sha512-af12Pnd0ZGu2HfP8NayB0kk6eC/lrfbQE6HlR4jD+34wdJ1Vw9TF6TMn6ZvffT+WgqVsl0hRbmNvz2u/23VmwA==", + "version": "0.35.4", + "resolved": "https://registry.npmjs.org/@img/sharp-linux-arm64/-/sharp-linux-arm64-0.35.4.tgz", + "integrity": "sha512-De4jpEnAU8Hd5oT0j1G3uL4ZvTuipVMn7YC6vPaJhy6/7EwEae0SVAoBrUMYQbkLGDm85taVWwuPc1a44LTzCQ==", "cpu": [ "arm64" ], "dev": true, - "libc": [ - "glibc" - ], "license": "Apache-2.0", "optional": true, "os": [ @@ -910,20 +880,17 @@ "url": "https://opencollective.com/libvips" }, "optionalDependencies": { - "@img/sharp-libvips-linux-arm64": "1.3.1" + "@img/sharp-libvips-linux-arm64": "1.3.3" } }, "node_modules/@img/sharp-linux-ppc64": { - "version": "0.35.2", - "resolved": "https://registry.npmjs.org/@img/sharp-linux-ppc64/-/sharp-linux-ppc64-0.35.2.tgz", - "integrity": "sha512-hYSBm7zcNtDCozCxQHYZJiu63b/bXsgRZuOxCIBZsStMM9Vap47iFHdbX4kCvQsblPB/k+clhELpdQJHQLSHvg==", + "version": "0.35.4", + "resolved": "https://registry.npmjs.org/@img/sharp-linux-ppc64/-/sharp-linux-ppc64-0.35.4.tgz", + "integrity": "sha512-2oYZJeIl4kCcMGk4ouZVjnkCtFrpQFlNEtJ6GbxzhHQchwH0NH/qEb9ykmOl29dqwMq+JhFdZn+1ak2FKhI9fQ==", "cpu": [ "ppc64" ], "dev": true, - "libc": [ - "glibc" - ], "license": "Apache-2.0", "optional": true, "os": [ @@ -936,20 +903,17 @@ "url": "https://opencollective.com/libvips" }, "optionalDependencies": { - "@img/sharp-libvips-linux-ppc64": "1.3.1" + "@img/sharp-libvips-linux-ppc64": "1.3.3" } }, "node_modules/@img/sharp-linux-riscv64": { - "version": "0.35.2", - "resolved": "https://registry.npmjs.org/@img/sharp-linux-riscv64/-/sharp-linux-riscv64-0.35.2.tgz", - "integrity": "sha512-qQt0Kc13+Hoan/Awq/qMSQw3L+RI1NCRPgD5cUJ/1WSSmIoysLOc72jlRM3E0OHN9Yr313jgeQ2T+zW+F03QFA==", + "version": "0.35.4", + "resolved": "https://registry.npmjs.org/@img/sharp-linux-riscv64/-/sharp-linux-riscv64-0.35.4.tgz", + "integrity": "sha512-cPbNChoRURAWdebDIHSenxRpgEdy7JkPydSnUxRm9VvKD7m0/xVaR/8Fzlu81pk5nHEvHH87UZUA7cTtwnbJSA==", "cpu": [ "riscv64" ], "dev": true, - "libc": [ - "glibc" - ], "license": "Apache-2.0", "optional": true, "os": [ @@ -962,20 +926,17 @@ "url": "https://opencollective.com/libvips" }, "optionalDependencies": { - "@img/sharp-libvips-linux-riscv64": "1.3.1" + "@img/sharp-libvips-linux-riscv64": "1.3.3" } }, "node_modules/@img/sharp-linux-s390x": { - "version": "0.35.2", - "resolved": "https://registry.npmjs.org/@img/sharp-linux-s390x/-/sharp-linux-s390x-0.35.2.tgz", - "integrity": "sha512-E4fLLfRPzDLlEeDaTzI98OFLcv++WL5ChLLMwPoVd0CIoZQqupBSNbOisPL5am9XsbQ9T84+iiMpUvbFtkunbA==", + "version": "0.35.4", + "resolved": "https://registry.npmjs.org/@img/sharp-linux-s390x/-/sharp-linux-s390x-0.35.4.tgz", + "integrity": "sha512-RY0JFY8Fd6RonCBtHz+DvadaPkXDSI1AUn6yWL9TipqkZ1vY8w8evqdgyDFnkm4/K1ve1TvZiaePP5oSd4+WVQ==", "cpu": [ "s390x" ], "dev": true, - "libc": [ - "glibc" - ], "license": "Apache-2.0", "optional": true, "os": [ @@ -988,20 +949,17 @@ "url": "https://opencollective.com/libvips" }, "optionalDependencies": { - "@img/sharp-libvips-linux-s390x": "1.3.1" + "@img/sharp-libvips-linux-s390x": "1.3.3" } }, "node_modules/@img/sharp-linux-x64": { - "version": "0.35.2", - "resolved": "https://registry.npmjs.org/@img/sharp-linux-x64/-/sharp-linux-x64-0.35.2.tgz", - "integrity": "sha512-gi0zFJJRLswfCZmHtJdikXPOc5u7qamSOS3NHedLqLd4W8Q0NqjdBr6TTRIgsfFjqfTsHFgdfvJ9LwqSgcHiAA==", + "version": "0.35.4", + "resolved": "https://registry.npmjs.org/@img/sharp-linux-x64/-/sharp-linux-x64-0.35.4.tgz", + "integrity": "sha512-9qvvEAuk8k89TfWUoX2htWjbAMX8p+NxCppjpcg5k6xMsjhBQPTsoIh36h9Qde4WRuGpJeYnOjdosDn/cnv+OA==", "cpu": [ "x64" ], "dev": true, - "libc": [ - "glibc" - ], "license": "Apache-2.0", "optional": true, "os": [ @@ -1014,20 +972,17 @@ "url": "https://opencollective.com/libvips" }, "optionalDependencies": { - "@img/sharp-libvips-linux-x64": "1.3.1" + "@img/sharp-libvips-linux-x64": "1.3.3" } }, "node_modules/@img/sharp-linuxmusl-arm64": { - "version": "0.35.2", - "resolved": "https://registry.npmjs.org/@img/sharp-linuxmusl-arm64/-/sharp-linuxmusl-arm64-0.35.2.tgz", - "integrity": "sha512-siWbOW1u6HFnFLrp0waKyW7VEf7jYvcDWdrXEFa8AkdAQgEvuu5Fz8/Y70w9EeqAdwDtfU012BhEHHaDqvQNzg==", + "version": "0.35.4", + "resolved": "https://registry.npmjs.org/@img/sharp-linuxmusl-arm64/-/sharp-linuxmusl-arm64-0.35.4.tgz", + "integrity": "sha512-KB5jxpfWQTr0nc3xdHtWChdbifHrBGsd2SM62Eyxrl8afikm+f5qGBU75SJIZBT/S1MC8XyacdlXBMSWq6OURA==", "cpu": [ "arm64" ], "dev": true, - "libc": [ - "musl" - ], "license": "Apache-2.0", "optional": true, "os": [ @@ -1040,20 +995,17 @@ "url": "https://opencollective.com/libvips" }, "optionalDependencies": { - "@img/sharp-libvips-linuxmusl-arm64": "1.3.1" + "@img/sharp-libvips-linuxmusl-arm64": "1.3.3" } }, "node_modules/@img/sharp-linuxmusl-x64": { - "version": "0.35.2", - "resolved": "https://registry.npmjs.org/@img/sharp-linuxmusl-x64/-/sharp-linuxmusl-x64-0.35.2.tgz", - "integrity": "sha512-YBqMMcjDi4QGYiSn4vNOYBhmlC4z5AXqkOUUqI2e0AFA4urNv4ESgOgwNl3K+4etQhha0twXlzeF20bbULm9Yg==", + "version": "0.35.4", + "resolved": "https://registry.npmjs.org/@img/sharp-linuxmusl-x64/-/sharp-linuxmusl-x64-0.35.4.tgz", + "integrity": "sha512-f+eZJZIQNEEd26RPSW+76chwOf1XtA2Y/O+5ocVyLliHkeih3e+jhLVBdNTd2rS3IbNXK8+ug93Vf5ZXtF5Lxg==", "cpu": [ "x64" ], "dev": true, - "libc": [ - "musl" - ], "license": "Apache-2.0", "optional": true, "os": [ @@ -1066,18 +1018,18 @@ "url": "https://opencollective.com/libvips" }, "optionalDependencies": { - "@img/sharp-libvips-linuxmusl-x64": "1.3.1" + "@img/sharp-libvips-linuxmusl-x64": "1.3.3" } }, "node_modules/@img/sharp-wasm32": { - "version": "0.35.2", - "resolved": "https://registry.npmjs.org/@img/sharp-wasm32/-/sharp-wasm32-0.35.2.tgz", - "integrity": "sha512-Mrv4JQNYVQ94xH+jzZ9r+gowleN8mv2FTgKT+PI6bx5C0G8TdNYndu161pg2i7uoBwxy2ImPMHrJOM2LZef7Bw==", + "version": "0.35.4", + "resolved": "https://registry.npmjs.org/@img/sharp-wasm32/-/sharp-wasm32-0.35.4.tgz", + "integrity": "sha512-zQnl4Kwp7Q6NHsENtU2T/00Zi+w3AQNwz3+UaTyVBy2FpXrzXzGjndpK61onhZjRtRpQXxCTeqw19bVyXOh7jA==", "dev": true, "license": "Apache-2.0 AND LGPL-3.0-or-later AND MIT", "optional": true, "dependencies": { - "@emnapi/runtime": "^1.11.1" + "@emnapi/runtime": "^1.11.3" }, "engines": { "node": ">=20.9.0" @@ -1087,9 +1039,9 @@ } }, "node_modules/@img/sharp-webcontainers-wasm32": { - "version": "0.35.2", - "resolved": "https://registry.npmjs.org/@img/sharp-webcontainers-wasm32/-/sharp-webcontainers-wasm32-0.35.2.tgz", - "integrity": "sha512-QNV27pxs9wpApEiCfvHM1RDoP1w1+2KrUWWDPEhEwg+latvOrfuhWrHWZKwdSFwU6jh3myjw/yOCRsUIuOft3g==", + "version": "0.35.4", + "resolved": "https://registry.npmjs.org/@img/sharp-webcontainers-wasm32/-/sharp-webcontainers-wasm32-0.35.4.tgz", + "integrity": "sha512-ESfNkywmCfPNyaZjxooddJQiQ+l/nTpGEOGthxiLnIHXC/CmcBixnfwUleX9mCz9ovrUUvKMap/pm8RYbzfwaA==", "cpu": [ "wasm32" ], @@ -1097,7 +1049,7 @@ "license": "Apache-2.0", "optional": true, "dependencies": { - "@img/sharp-wasm32": "0.35.2" + "@img/sharp-wasm32": "0.35.4" }, "engines": { "node": ">=20.9.0" @@ -1107,9 +1059,9 @@ } }, "node_modules/@img/sharp-win32-arm64": { - "version": "0.35.2", - "resolved": "https://registry.npmjs.org/@img/sharp-win32-arm64/-/sharp-win32-arm64-0.35.2.tgz", - "integrity": "sha512-BiVRYc/t6/Vl3e1hBx0hugG4oN9Pydf4fgMSpxTQJmwGUg/YoXTWHiFeRymHfCZzifxu4F4rpk/I67D0LQ20wQ==", + "version": "0.35.4", + "resolved": "https://registry.npmjs.org/@img/sharp-win32-arm64/-/sharp-win32-arm64-0.35.4.tgz", + "integrity": "sha512-iNdlBX9gLVvqe2I3uIJSIKTq6wckP/DYxZtcqxm09x5Gi24DnFBmPAWZmr60ZyYMG0xlzo6goG3670ar+RXvRw==", "cpu": [ "arm64" ], @@ -1127,9 +1079,9 @@ } }, "node_modules/@img/sharp-win32-ia32": { - "version": "0.35.2", - "resolved": "https://registry.npmjs.org/@img/sharp-win32-ia32/-/sharp-win32-ia32-0.35.2.tgz", - "integrity": "sha512-YYEhx9PImCC7T0tI8JDMi4DB9LwLCXCU5OWNYEXAxh5Q1ShKkyC6byxzoBJ3gEFDnH2lQckWuDe70G7mB2XJog==", + "version": "0.35.4", + "resolved": "https://registry.npmjs.org/@img/sharp-win32-ia32/-/sharp-win32-ia32-0.35.4.tgz", + "integrity": "sha512-kqRsbaa5CS6KHlpxnN7WhE6vAAugXyZButpRdvDWetlv6Qv4N9WTcrWzF7tXfB9T7MsoadqdI8hmwLq6UlLvtw==", "cpu": [ "ia32" ], @@ -1147,9 +1099,9 @@ } }, "node_modules/@img/sharp-win32-x64": { - "version": "0.35.2", - "resolved": "https://registry.npmjs.org/@img/sharp-win32-x64/-/sharp-win32-x64-0.35.2.tgz", - "integrity": "sha512-imoOyBcoM/iiUr4J6VPpCNjPnjvP/Gks95898yB8YqoGGYmHYbOyCuNv9FMhFgtaiHFGbHW8bxKqRV6VjtXThQ==", + "version": "0.35.4", + "resolved": "https://registry.npmjs.org/@img/sharp-win32-x64/-/sharp-win32-x64-0.35.4.tgz", + "integrity": "sha512-XtmnYhBcrORsJ4XJngyzr/EWP0hRZLAZRFaApdKuviyqF78+ylxh2y06ZmtULAMOnObJ3ucpN0AcwSWnMowTRg==", "cpu": [ "x64" ], @@ -1352,16 +1304,16 @@ } }, "node_modules/miniflare": { - "version": "5.20260903.0-alpha", - "resolved": "https://registry.npmjs.org/miniflare/-/miniflare-5.20260903.0-alpha.tgz", - "integrity": "sha512-VCZIFxOqFXeibRBJWwTlppqbv2lkeO10IX3QBRGK9j1QiMAfH/OSsCvbjp1MslBMhVZmy2VTRlVaVnsKnbDp6A==", + "version": "5.20260911.1-alpha", + "resolved": "https://registry.npmjs.org/miniflare/-/miniflare-5.20260911.1-alpha.tgz", + "integrity": "sha512-7IDj9monoYcCPrS8HfcTt90T3pDwKGvNEAR1Y061KbJhBd5JkStOwOpHMUDFBo9PEbjWVDxPICAnXGNNV2LNfQ==", "dev": true, "license": "MIT", "dependencies": { "@cspotcode/source-map-support": "0.8.1", - "sharp": "0.35.2", + "sharp": "0.35.4", "undici": "7.29.0", - "workerd": "1.20260903.1", + "workerd": "1.20260911.1", "ws": "8.21.0", "youch": "4.1.0-beta.10" }, @@ -1397,15 +1349,15 @@ } }, "node_modules/sharp": { - "version": "0.35.2", - "resolved": "https://registry.npmjs.org/sharp/-/sharp-0.35.2.tgz", - "integrity": "sha512-FVtFjtBCMiJS6yb5CX7Sop45WFMpeGw6oRKuJnXYgf/f1ms/D7LE/ZUSNxnW7rZ/dbslQWYkoqFHGPaDBtaK4w==", + "version": "0.35.4", + "resolved": "https://registry.npmjs.org/sharp/-/sharp-0.35.4.tgz", + "integrity": "sha512-n++8XWcj+jCOr2IOl7h8LbKnGBDY4aPbmprMONBNFdn0ImXqpGVv5zliDs0V9HbmbCQLpbuo2ej9rAoOQTvMDA==", "dev": true, "license": "Apache-2.0", "dependencies": { "@img/colour": "^1.1.0", "detect-libc": "^2.1.2", - "semver": "^7.8.4" + "semver": "^7.8.5" }, "engines": { "node": ">=20.9.0" @@ -1414,31 +1366,36 @@ "url": "https://opencollective.com/libvips" }, "optionalDependencies": { - "@img/sharp-darwin-arm64": "0.35.2", - "@img/sharp-darwin-x64": "0.35.2", - "@img/sharp-freebsd-wasm32": "0.35.2", - "@img/sharp-libvips-darwin-arm64": "1.3.1", - "@img/sharp-libvips-darwin-x64": "1.3.1", - "@img/sharp-libvips-linux-arm": "1.3.1", - "@img/sharp-libvips-linux-arm64": "1.3.1", - "@img/sharp-libvips-linux-ppc64": "1.3.1", - "@img/sharp-libvips-linux-riscv64": "1.3.1", - "@img/sharp-libvips-linux-s390x": "1.3.1", - "@img/sharp-libvips-linux-x64": "1.3.1", - "@img/sharp-libvips-linuxmusl-arm64": "1.3.1", - "@img/sharp-libvips-linuxmusl-x64": "1.3.1", - "@img/sharp-linux-arm": "0.35.2", - "@img/sharp-linux-arm64": "0.35.2", - "@img/sharp-linux-ppc64": "0.35.2", - "@img/sharp-linux-riscv64": "0.35.2", - "@img/sharp-linux-s390x": "0.35.2", - "@img/sharp-linux-x64": "0.35.2", - "@img/sharp-linuxmusl-arm64": "0.35.2", - "@img/sharp-linuxmusl-x64": "0.35.2", - "@img/sharp-webcontainers-wasm32": "0.35.2", - "@img/sharp-win32-arm64": "0.35.2", - "@img/sharp-win32-ia32": "0.35.2", - "@img/sharp-win32-x64": "0.35.2" + "@img/sharp-darwin-arm64": "0.35.4", + "@img/sharp-darwin-x64": "0.35.4", + "@img/sharp-freebsd-wasm32": "0.35.4", + "@img/sharp-libvips-darwin-arm64": "1.3.3", + "@img/sharp-libvips-darwin-x64": "1.3.3", + "@img/sharp-libvips-linux-arm": "1.3.3", + "@img/sharp-libvips-linux-arm64": "1.3.3", + "@img/sharp-libvips-linux-ppc64": "1.3.3", + "@img/sharp-libvips-linux-riscv64": "1.3.3", + "@img/sharp-libvips-linux-s390x": "1.3.3", + "@img/sharp-libvips-linux-x64": "1.3.3", + "@img/sharp-libvips-linuxmusl-arm64": "1.3.3", + "@img/sharp-libvips-linuxmusl-x64": "1.3.3", + "@img/sharp-linux-arm": "0.35.4", + "@img/sharp-linux-arm64": "0.35.4", + "@img/sharp-linux-ppc64": "0.35.4", + "@img/sharp-linux-riscv64": "0.35.4", + "@img/sharp-linux-s390x": "0.35.4", + "@img/sharp-linux-x64": "0.35.4", + "@img/sharp-linuxmusl-arm64": "0.35.4", + "@img/sharp-linuxmusl-x64": "0.35.4", + "@img/sharp-webcontainers-wasm32": "0.35.4", + "@img/sharp-win32-arm64": "0.35.4", + "@img/sharp-win32-ia32": "0.35.4", + "@img/sharp-win32-x64": "0.35.4" + }, + "peerDependenciesMeta": { + "@types/node": { + "optional": true + } } }, "node_modules/supports-color": { @@ -1478,14 +1435,15 @@ "integrity": "sha512-i7qRCmY42zmCwnYlh9H2SvLEypEFGye5iRmEMKjcGi7zk9UquigRjFtTLz0TYqr0ZGLZhaMHl/foy1bZR+Cwlw==", "dev": true, "license": "MIT", + "peer": true, "dependencies": { "pathe": "^2.0.3" } }, "node_modules/workerd": { - "version": "1.20260903.1", - "resolved": "https://registry.npmjs.org/workerd/-/workerd-1.20260903.1.tgz", - "integrity": "sha512-xJzt2RnCy7ulOULmZy/4JLbEPg1uisp9lVoOUsEz+UVhDsTmrSQ0rBXZMGcXuhr2HCGKs89bg8nnbbzvBSX1Ig==", + "version": "1.20260911.1", + "resolved": "https://registry.npmjs.org/workerd/-/workerd-1.20260911.1.tgz", + "integrity": "sha512-vRr8QdBxueQOZJO1hRCI73EZlix87IAyBAcSyI3rA1VB+6oxjw3oaqzYnIV8C4IOPtUgihbdMAgzkb5GM4V7DQ==", "dev": true, "hasInstallScript": true, "license": "Apache-2.0", @@ -1496,17 +1454,17 @@ "node": ">=16" }, "optionalDependencies": { - "@cloudflare/workerd-darwin-64": "1.20260903.1", - "@cloudflare/workerd-darwin-arm64": "1.20260903.1", - "@cloudflare/workerd-linux-64": "1.20260903.1", - "@cloudflare/workerd-linux-arm64": "1.20260903.1", - "@cloudflare/workerd-windows-64": "1.20260903.1" + "@cloudflare/workerd-darwin-64": "1.20260911.1", + "@cloudflare/workerd-darwin-arm64": "1.20260911.1", + "@cloudflare/workerd-linux-64": "1.20260911.1", + "@cloudflare/workerd-linux-arm64": "1.20260911.1", + "@cloudflare/workerd-windows-64": "1.20260911.1" } }, "node_modules/wrangler": { - "version": "4.129.0", - "resolved": "https://registry.npmjs.org/wrangler/-/wrangler-4.129.0.tgz", - "integrity": "sha512-PGPvs9UPoFrwxT0VogpESSZGvZIctAuTK3wGsLLPHtHsSgS85kNdvtpa2d14UzG8gwLWD64XUGFPGG9tOXG9VQ==", + "version": "4.131.2", + "resolved": "https://registry.npmjs.org/wrangler/-/wrangler-4.131.2.tgz", + "integrity": "sha512-jmkGE7monbPKyYQr1FPQN+SARVhddqw2fhXOmTKCw4lroqlFGSS6rit/RTvPi/qzNLKrXxkS8DhWXasJnStplg==", "dev": true, "license": "MIT OR Apache-2.0", "dependencies": { @@ -1514,10 +1472,10 @@ "@cloudflare/unenv-preset": "2.16.1", "blake3-wasm": "2.1.5", "esbuild": "0.28.1", - "miniflare": "5.20260903.0-alpha", + "miniflare": "5.20260911.1-alpha", "path-to-regexp": "6.3.0", "unenv": "2.0.0-rc.24", - "workerd": "1.20260903.1" + "workerd": "1.20260911.1" }, "bin": { "cf-wrangler": "bin/cf-wrangler.js", @@ -1531,7 +1489,7 @@ "fsevents": "2.3.3" }, "peerDependencies": { - "@cloudflare/workers-types": "^5.20260903.1" + "@cloudflare/workers-types": "^5.20260911.1" }, "peerDependenciesMeta": { "@cloudflare/workers-types": { diff --git a/scripts/build-site.mjs b/scripts/build-site.mjs index a23371e..998849e 100644 --- a/scripts/build-site.mjs +++ b/scripts/build-site.mjs @@ -784,7 +784,7 @@ function downloadPageContent() { ${fileRows} -

v0.1.45 is the current Windows release and adds the Zoom Talkback dock and the panelist loudness readiness board over v0.1.44. Existing installs are offered it by the in-plugin update check.

+

See the Windows release notes for the latest changes. Existing installs are offered new stable releases by the in-plugin update check.

macOS — Apple Silicon beta v${MAC_VERSION}

Developer ID signed, notarized by Apple, and stapled. The macOS installer is built and notarized separately from the Windows release, so its version can trail Windows; it carries the same raw-media recovery, waiting-room and video-quality fixes and the panelist loudness readiness board.

diff --git a/src/iso-audio-reader.cpp b/src/iso-audio-reader.cpp new file mode 100644 index 0000000..18b6f77 --- /dev/null +++ b/src/iso-audio-reader.cpp @@ -0,0 +1,72 @@ +#include "iso-audio-reader.h" +#include +void IsoAudioReader::read(uint32_t bytes, uint32_t generation) +{ + // Copy and release the tap lock before calling the recorder: Stop acquires + // the recorder lock first, and a callback must never reverse that order. + struct Packet { std::vector pcm; uint32_t rate; uint16_t channels; uint64_t ns; }; + std::vector packets; + { + std::lock_guard lock(m_mutex); + if (m_closed || !m_error.empty() || !bytes || bytes > 1024*1024) return; + const auto base = IPC_SHM_PREFIX + m_uuid + "_audio"; + const auto name = shm_region_name(base, generation); + if (m_generation != generation || m_region.size < shm_audio_region_bytes(bytes)) { + shm_region_destroy(m_region); m_started = false; + } + if (!m_region.ptr) { + if (!shm_region_open_readwrite(m_region, name, shm_audio_region_bytes(bytes))) return; + m_generation = generation; + } + auto *ring = static_cast(m_region.ptr); + if (ring->slot_count != kAudioRingSlots || !ring->slot_bytes || ring->slot_bytes > 1024*1024) { + m_error = "ISO audio ring format is invalid"; return; + } + const auto needed = shm_audio_region_bytes(ring->slot_bytes); + if (needed > m_region.size) { + shm_region_destroy(m_region); + if (!shm_region_open_readwrite(m_region, name, needed)) return; + ring = static_cast(m_region.ptr); + } + if (ring->slot_count != kAudioRingSlots || ring->slot_bytes > 1024*1024 || + shm_audio_region_bytes(ring->slot_bytes) > m_region.size) { + m_error = "ISO audio ring resized beyond mapped bounds"; return; + } + if (!m_started) { + // This UUID is unique to the recording. Preserve its first buffers. + m_index = ring->write_index > kAudioRingSlots ? ring->write_index-kAudioRingSlots : 0; + m_started = true; + } + for (int pass=0; pass<16; ++pass) { + const auto end = ring->write_index; + if (end-m_index > kAudioRingSlots) { + m_error = "ISO isolated audio ring overrun"; audio_ring_reader_abandon(ring); break; + } + while (m_index != end) { + const auto index = m_index++; + const auto *slot = reinterpret_cast( + static_cast(m_region.ptr)+shm_audio_slot_offset(*ring,index%kAudioRingSlots)); + for (int attempt=0; attempt<3; ++attempt) { + const auto seq = slot->sequence; + std::atomic_thread_fence(std::memory_order_acquire); + if (seq&1) continue; + const auto len=slot->byte_len, participant=slot->participant_id; + const auto rate=ring->sample_rate; const auto channels=ring->channels; + if (!len || len>ring->slot_bytes) break; + m_pcm.resize(len); std::memcpy(m_pcm.data(),slot+1,len); + std::atomic_thread_fence(std::memory_order_acquire); + if (slot->sequence!=seq || ring->write_index-index>kAudioRingSlots) continue; + if (participant != m_participant) { + m_error="ISO audio participant attribution mismatch"; break; + } + packets.push_back({m_pcm,rate,channels,m_now()}); + break; + } + } + if (audio_ring_reader_done(ring,m_index)) break; + if (pass==15) audio_ring_reader_abandon(ring); + } + } + for (const auto &packet:packets) + m_deliver(packet.pcm.data(),uint32_t(packet.pcm.size()),packet.rate,packet.channels,packet.ns); +} diff --git a/src/iso-audio-reader.h b/src/iso-audio-reader.h new file mode 100644 index 0000000..c70a17a --- /dev/null +++ b/src/iso-audio-reader.h @@ -0,0 +1,27 @@ +#pragma once +#include "engine-ipc.h" +#include +#include +#include + +class IsoAudioReader { +public: + using Deliver = std::function; + using Now = std::function; + IsoAudioReader(std::string uuid, uint32_t participant, Deliver deliver, Now now) + : m_uuid(std::move(uuid)), m_participant(participant), m_deliver(std::move(deliver)), m_now(std::move(now)) {} + ~IsoAudioReader() { shm_region_destroy(m_region); } + void read(uint32_t bytes, uint32_t generation); + std::string error() const { std::lock_guard lock(m_mutex); return m_error; } +protected: + std::string m_uuid; + uint32_t m_participant; + Deliver m_deliver; + mutable std::mutex m_mutex; + ShmRegion m_region{}; + uint32_t m_generation = 0, m_index = 0; + bool m_started = false, m_closed = false; + std::vector m_pcm; + std::string m_error; + Now m_now; +}; diff --git a/src/iso-audio-routing.h b/src/iso-audio-routing.h new file mode 100644 index 0000000..288c47c --- /dev/null +++ b/src/iso-audio-routing.h @@ -0,0 +1,15 @@ +#pragma once +#include + +inline bool audio_receives_mix(bool isolated, bool audience, bool recording) +{ + return !isolated && !audience && !recording; +} +inline bool audio_receives_participant(uint32_t target, uint32_t speaker, bool isolated, bool recording) +{ + return target != 0 && target == speaker && (isolated || recording); +} +inline bool audio_claims_audience(bool isolated, bool recording) +{ + return isolated && !recording; +} diff --git a/src/iso-audio-tap.cpp b/src/iso-audio-tap.cpp new file mode 100644 index 0000000..42e7c3a --- /dev/null +++ b/src/iso-audio-tap.cpp @@ -0,0 +1,37 @@ +#include "iso-audio-tap.h" +#include "zoom-engine-client.h" +#include +#include + +IsoAudioTap::~IsoAudioTap() { stop(); } +bool IsoAudioTap::start() +{ + const std::weak_ptr weak = shared_from_this(); + ZoomEngineClient::instance().register_source(m_uuid, { + {}, [weak](uint32_t bytes, uint32_t, uint32_t gen) { + if (auto tap = weak.lock()) tap->read(bytes, gen); + }, [weak] { + if (auto tap = weak.lock()) { + std::lock_guard lock(tap->m_mutex); + shm_region_destroy(tap->m_region); + tap->m_started = false; + tap->m_error = "ISO audio engine restarted; stop and restart ISO recording"; + } + }}); + if (ZoomEngineClient::instance().subscribe_audio(m_uuid, m_participant, true, false, true)) + return true; + std::lock_guard lock(m_mutex); + m_error = "ISO isolated audio subscription could not be sent"; + return false; +} +void IsoAudioTap::stop() +{ + { + std::lock_guard lock(m_mutex); + if (m_closed) return; + m_closed = true; + shm_region_destroy(m_region); + } + ZoomEngineClient::instance().unregister_source(m_uuid); + ZoomEngineClient::instance().unsubscribe(m_uuid); +} diff --git a/src/iso-audio-tap.h b/src/iso-audio-tap.h new file mode 100644 index 0000000..7b1efe0 --- /dev/null +++ b/src/iso-audio-tap.h @@ -0,0 +1,11 @@ +#pragma once +#include "iso-audio-reader.h" +#include + +class IsoAudioTap : public IsoAudioReader, public std::enable_shared_from_this { +public: + using IsoAudioReader::IsoAudioReader; + ~IsoAudioTap(); + bool start(); + void stop(); +}; diff --git a/src/iso-av-mux.h b/src/iso-av-mux.h new file mode 100644 index 0000000..fdc34da --- /dev/null +++ b/src/iso-av-mux.h @@ -0,0 +1,114 @@ +#pragma once + +// Streaming Matroska transport to FFmpeg over one pipe. One fixed I420 track +// and one PCM S16LE stereo track, both on a microsecond clock. This is an +// internal transport; FFmpeg writes the user's single H.264/AAC MP4. +// Codec mapping: https://www.matroska.org/technical/codec_specs.html +#include +#include +#include + +namespace iso_av +{ +using Bytes = std::vector; +inline void be(Bytes &out, uint64_t value, int count) +{ + for (int i = count - 1; i >= 0; --i) + out.push_back(uint8_t(value >> (i * 8))); +} +inline void element(Bytes &out, uint32_t id, const Bytes &value) +{ + int n = id > 0xffffff ? 4 : id > 0xffff ? 3 : id > 0xff ? 2 : 1; + be(out, id, n); + // Fixed eight-byte EBML size, valid for every element in this transport. + be(out, (uint64_t(1) << 56) | value.size(), 8); + out.insert(out.end(), value.begin(), value.end()); +} +inline void number(Bytes &out, uint32_t id, uint64_t value) +{ + Bytes b; + be(b, value, 8); + element(out, id, b); +} +inline void string(Bytes &out, uint32_t id, const char *value) +{ + element(out, id, Bytes(value, value + std::strlen(value))); +} +inline Bytes header() +{ + Bytes out, ebml, info, tracks, video, audio, v, a; + number(ebml, 0x4286, 1); + number(ebml, 0x42f7, 1); + number(ebml, 0x42f2, 4); + number(ebml, 0x42f3, 8); + string(ebml, 0x4282, "matroska"); + number(ebml, 0x4287, 4); + number(ebml, 0x4285, 2); + element(out, 0x1a45dfa3, ebml); + be(out, 0x18538067, 4); + be(out, 0x01ffffffffffffffULL, 8); // unknown segment size + number(info, 0x2ad7b1, 1000); // one tick = one microsecond + string(info, 0x4d80, "CoreVideo"); + string(info, 0x5741, "CoreVideo"); + element(out, 0x1549a966, info); + number(video, 0xd7, 1); + number(video, 0x73c5, 1); + number(video, 0x83, 1); + string(video, 0x86, "V_MS/VFW/FOURCC"); + Bytes bitmap(40, 0); + const auto le = [&](size_t at, uint32_t value) { + for (int i = 0; i < 4; ++i) + bitmap[at + i] = uint8_t(value >> (8 * i)); + }; + le(0, 40); + le(4, 1920); + le(8, 1080); + bitmap[12] = 1; + bitmap[14] = 12; + le(16, 0x30323449); + le(20, 1920 * 1080 * 3 / 2); // I420 BITMAPINFOHEADER + element(video, 0x63a2, bitmap); + number(video, 0x23e383, 33333333); + number(v, 0xb0, 1920); + number(v, 0xba, 1080); + // The engine normalizes Zoom pixels to full-range BT.709 before delivery. + Bytes colour; + number(colour, 0x55b1, 1); + number(colour, 0x55b9, 2); + number(colour, 0x55ba, 1); + number(colour, 0x55bb, 1); + element(v, 0x55b0, colour); + element(video, 0xe0, v); + element(tracks, 0xae, video); + number(audio, 0xd7, 2); + number(audio, 0x73c5, 2); + number(audio, 0x83, 2); + string(audio, 0x86, "A_PCM/INT/LIT"); + double rate = 48000; + uint64_t bits; + std::memcpy(&bits, &rate, 8); + Bytes frequency; + be(frequency, bits, 8); + element(a, 0xb5, frequency); + number(a, 0x9f, 2); + number(a, 0x6264, 16); + element(audio, 0xe1, a); + element(tracks, 0xae, audio); + element(out, 0x1654ae6b, tracks); + return out; +} +inline Bytes frame(uint64_t index, const Bytes &video, const Bytes &audio) +{ + Bytes cluster, block; + number(cluster, 0xe7, index * 1000000ULL / 30); + block = {0x81, 0, 0, 0x80}; + block.insert(block.end(), video.begin(), video.end()); + element(cluster, 0xa3, block); + block = {0x82, 0, 0, 0x80}; + block.insert(block.end(), audio.begin(), audio.end()); + element(cluster, 0xa3, block); + Bytes out; + element(out, 0x1f43b675, cluster); + return out; +} +} // namespace iso_av diff --git a/src/iso-d3d-conformer.h b/src/iso-d3d-conformer.h new file mode 100644 index 0000000..018232d --- /dev/null +++ b/src/iso-d3d-conformer.h @@ -0,0 +1,204 @@ +#pragma once + +#include "iso-frame-conform.h" +#include +#include +#include +#include + +namespace corevideo::modules +{ + +// Adapted from CoreVideo Pro's D3DIsoFrameConformer. This transport needs planar +// I420 (Pro's writer takes NV12), with full-range black. Owned by one ISO worker; +// no immediate context is shared with the live compositor. +class D3DIsoFrameConformer +{ + public: + explicit D3DIsoFrameConformer(bool warpForTest = false) : warpForTest_(warpForTest) {} + bool prepare(std::string &error) { return device_ || initialize(error); } + + bool convert(const std::vector &input, int sw, int sh, int dw, int dh, + std::vector &output, std::string &error) + { + if (sw < 2 || sh < 2 || dw < 2 || dh < 2 || sw > 8192 || sh > 8192 || dw > 8192 || + dh > 8192 || ((sw | sh | dw | dh) & 1) || input.size() < size_t(sw) * sh * 3 / 2) { + error = "invalid I420 conformance dimensions or payload"; + return false; + } + if (!device_ && !initialize(error)) + return false; + const UINT inputBytes = UINT(size_t(sw) * sh * 3 / 2); + const UINT outputBytes = UINT(size_t(dw) * dh * 3 / 2); + if ((inputBytes + 3u) / 4u * 4u != inputCapacity_ || + (outputBytes + 3u) / 4u * 4u != outputCapacity_) { + if (!allocate(inputBytes, outputBytes, error)) + return false; + } + D3D11_MAPPED_SUBRESOURCE mapped{}; + if (FAILED(context_->Map(input_.Get(), 0, D3D11_MAP_WRITE_DISCARD, 0, &mapped))) { + error = "map ISO GPU upload"; + return false; + } + std::memcpy(mapped.pData, input.data(), inputBytes); + if (inputCapacity_ > inputBytes) + std::memset(static_cast(mapped.pData) + inputBytes, 0, + inputCapacity_ - inputBytes); + context_->Unmap(input_.Get(), 0); + const auto fit = isoFitRect(sw, sh, dw, dh); + const UINT groups = (outputCapacity_ / 4u + 255u) / 256u; + const UINT groupsX = std::min(groups, 65535u); + const UINT constants[12] = {UINT(sw), UINT(sh), UINT(dw), UINT(dh), + UINT(fit.x), UINT(fit.y), UINT(fit.width), UINT(fit.height), + inputBytes, outputBytes, groupsX * 256u, 0}; + context_->UpdateSubresource(constants_.Get(), 0, nullptr, constants, 0, 0); + ID3D11ShaderResourceView *srv = inputView_.Get(); + ID3D11UnorderedAccessView *uav = outputView_.Get(); + ID3D11Buffer *cb = constants_.Get(); + context_->CSSetShader(shader_.Get(), nullptr, 0); + context_->CSSetShaderResources(0, 1, &srv); + context_->CSSetUnorderedAccessViews(0, 1, &uav, nullptr); + context_->CSSetConstantBuffers(0, 1, &cb); + context_->Dispatch(groupsX, (groups + groupsX - 1) / groupsX, 1); + srv = nullptr; + uav = nullptr; + context_->CSSetShaderResources(0, 1, &srv); + context_->CSSetUnorderedAccessViews(0, 1, &uav, nullptr); + context_->CopyResource(readback_.Get(), output_.Get()); + if (FAILED(context_->Map(readback_.Get(), 0, D3D11_MAP_READ, 0, &mapped))) { + error = "map ISO GPU readback"; + return false; + } + output.resize(outputBytes); + std::memcpy(output.data(), mapped.pData, outputBytes); + context_->Unmap(readback_.Get(), 0); + return true; + } + + private: + template using Ptr = Microsoft::WRL::ComPtr; + bool initialize(std::string &error) + { + D3D_FEATURE_LEVEL level{}; + const D3D_FEATURE_LEVEL requested[] = {D3D_FEATURE_LEVEL_11_0}; + if (FAILED(D3D11CreateDevice( + nullptr, warpForTest_ ? D3D_DRIVER_TYPE_WARP : D3D_DRIVER_TYPE_HARDWARE, nullptr, 0, + requested, 1, D3D11_SDK_VERSION, &device_, &level, &context_))) { + error = "create ISO GPU conformance device"; + return false; + } + // Four output bytes per invocation avoid sub-word UAV write races. All + // box bounds, rounding, chroma alignment, and bar levels match the oracle. + static constexpr char source[] = R"( +ByteAddressBuffer src : register(t0); +RWByteAddressBuffer dst : register(u0); +cbuffer Parameters : register(b0) { uint4 dims; uint4 fit; uint4 sizes; }; +uint byteAt(uint i) { return (src.Load(i & ~3u) >> ((i & 3u) * 8u)) & 255u; } +uint box(uint offset, uint pw, uint ph, uint x, uint y, uint rw, uint rh) { + uint x0=x*pw/rw, x1=min(pw,max(x0+1,(x+1)*pw/rw)); + uint y0=y*ph/rh, y1=min(ph,max(y0+1,(y+1)*ph/rh)); + uint total=0, count=(x1-x0)*(y1-y0); + for(uint sy=y0;sy=sizes.y) return 0; + uint luma=dims.z*dims.w; + if(i=fit.x+fit.z || y=fit.y+fit.w) return 0; + return box(0,dims.x,dims.y,x-fit.x,y-fit.y,fit.z,fit.w); + } + uint cl=luma/4,j=i-luma,plane=j/cl,p=j%cl,x=p%(dims.z/2),y=p/(dims.z/2); + if(x=(fit.x+fit.z)/2 || y=(fit.y+fit.w)/2) return 128; + uint sl=dims.x*dims.y; + return box(sl+plane*(sl/4),dims.x/2,dims.y/2,x-fit.x/2,y-fit.y/2,fit.z/2,fit.w/2); +} +[numthreads(256,1,1)] void main(uint3 tid : SV_DispatchThreadID) { + uint at=(tid.x+tid.y*sizes.z)*4; + if(at>=sizes.y) return; + dst.Store(at,pixel(at)|(pixel(at+1)<<8)|(pixel(at+2)<<16)|(pixel(at+3)<<24)); +})"; + Ptr code, diagnostics; + const HRESULT compiled = D3DCompile( + source, sizeof(source) - 1, "IsoBoxConform", nullptr, nullptr, "main", "cs_5_0", + D3DCOMPILE_ENABLE_STRICTNESS | D3DCOMPILE_OPTIMIZATION_LEVEL3, 0, &code, &diagnostics); + if (FAILED(compiled) || + FAILED(device_->CreateComputeShader(code->GetBufferPointer(), code->GetBufferSize(), + nullptr, &shader_))) { + error = "compile ISO GPU box filter"; + device_.Reset(); + context_.Reset(); + return false; + } + D3D11_BUFFER_DESC desc{}; + desc.ByteWidth = 48; + desc.Usage = D3D11_USAGE_DEFAULT; + desc.BindFlags = D3D11_BIND_CONSTANT_BUFFER; + if (FAILED(device_->CreateBuffer(&desc, nullptr, &constants_))) { + error = "create ISO GPU constants"; + device_.Reset(); + context_.Reset(); + return false; + } + return true; + } + bool allocate(UINT inBytes, UINT outBytes, std::string &error) + { + inputView_.Reset(); + outputView_.Reset(); + input_.Reset(); + output_.Reset(); + readback_.Reset(); + inputCapacity_ = outputCapacity_ = 0; + D3D11_BUFFER_DESC desc{}; + desc.ByteWidth = (inBytes + 3u) / 4u * 4u; + desc.Usage = D3D11_USAGE_DYNAMIC; + desc.BindFlags = D3D11_BIND_SHADER_RESOURCE; + desc.CPUAccessFlags = D3D11_CPU_ACCESS_WRITE; + desc.MiscFlags = D3D11_RESOURCE_MISC_BUFFER_ALLOW_RAW_VIEWS; + HRESULT hr = device_->CreateBuffer(&desc, nullptr, &input_); + D3D11_SHADER_RESOURCE_VIEW_DESC srv{}; + srv.Format = DXGI_FORMAT_R32_TYPELESS; + srv.ViewDimension = D3D11_SRV_DIMENSION_BUFFEREX; + srv.BufferEx.NumElements = desc.ByteWidth / 4; + srv.BufferEx.Flags = D3D11_BUFFEREX_SRV_FLAG_RAW; + if (SUCCEEDED(hr)) + hr = device_->CreateShaderResourceView(input_.Get(), &srv, &inputView_); + desc.ByteWidth = (outBytes + 3u) / 4u * 4u; + desc.Usage = D3D11_USAGE_DEFAULT; + desc.BindFlags = D3D11_BIND_UNORDERED_ACCESS; + desc.CPUAccessFlags = 0; + if (SUCCEEDED(hr)) + hr = device_->CreateBuffer(&desc, nullptr, &output_); + D3D11_UNORDERED_ACCESS_VIEW_DESC uav{}; + uav.Format = DXGI_FORMAT_R32_TYPELESS; + uav.ViewDimension = D3D11_UAV_DIMENSION_BUFFER; + uav.Buffer.NumElements = desc.ByteWidth / 4; + uav.Buffer.Flags = D3D11_BUFFER_UAV_FLAG_RAW; + if (SUCCEEDED(hr)) + hr = device_->CreateUnorderedAccessView(output_.Get(), &uav, &outputView_); + desc.Usage = D3D11_USAGE_STAGING; + desc.BindFlags = 0; + desc.CPUAccessFlags = D3D11_CPU_ACCESS_READ; + desc.MiscFlags = 0; + if (SUCCEEDED(hr)) + hr = device_->CreateBuffer(&desc, nullptr, &readback_); + if (FAILED(hr)) { + error = "allocate ISO GPU conformance buffers"; + return false; + } + inputCapacity_ = (inBytes + 3u) / 4u * 4u; + outputCapacity_ = (outBytes + 3u) / 4u * 4u; + return true; + } + bool warpForTest_ = false; + UINT inputCapacity_ = 0, outputCapacity_ = 0; + Ptr device_; + Ptr context_; + Ptr shader_; + Ptr input_, output_, readback_, constants_; + Ptr inputView_; + Ptr outputView_; +}; +} // namespace corevideo::modules diff --git a/src/iso-feed-selection.h b/src/iso-feed-selection.h new file mode 100644 index 0000000..0601539 --- /dev/null +++ b/src/iso-feed-selection.h @@ -0,0 +1,11 @@ +#pragma once +#include +#include +#include + +inline bool iso_feed_selected(const std::vector &selected, + const std::string &uuid, bool routed, bool excluded) +{ + return routed && !excluded && !uuid.empty() && + std::find(selected.begin(), selected.end(), uuid) != selected.end(); +} diff --git a/src/iso-frame-conform.h b/src/iso-frame-conform.h new file mode 100644 index 0000000..52e2ec8 --- /dev/null +++ b/src/iso-frame-conform.h @@ -0,0 +1,47 @@ +#pragma once + +// Aspect-fit geometry adapted from CoreVideo Pro IsoFrameConform.h. +// ISO output is always 1920x1080; chroma-aligned bars preserve aspect ratio. +#include +#include +#include +#include + +namespace corevideo::modules +{ + +struct IsoFitRect { + int x = 0; + int y = 0; + int width = 0; + int height = 0; +}; + +// Largest centred rect of the source's aspect that fits the destination, with +// every edge EVEN: I420 and NV12 carry chroma at half resolution, so an odd +// origin or extent splits a chroma sample across the letterbox edge and fringes +// it. A degenerate input returns an empty rect rather than dividing by zero. +[[nodiscard]] inline IsoFitRect isoFitRect(int srcW, int srcH, int dstW, int dstH) +{ + if (srcW <= 0 || srcH <= 0 || dstW <= 0 || dstH <= 0) { + return {}; + } + const auto even = [](int value) { return value & ~1; }; + // Compare aspects in integer cross-products: no float, no rounding drift. + const long long srcAspect = static_cast(srcW) * dstH; + const long long dstAspect = static_cast(dstW) * srcH; + int width = dstW; + int height = dstH; + if (srcAspect > dstAspect) { + // Source is wider: full width, bars top and bottom. + height = static_cast((static_cast(dstW) * srcH + srcW / 2) / srcW); + } else if (srcAspect < dstAspect) { + // Source is taller: full height, bars left and right. + width = static_cast((static_cast(dstH) * srcW + srcH / 2) / srcH); + } + width = std::max(2, std::min(even(width), even(dstW))); + height = std::max(2, std::min(even(height), even(dstH))); + return {even((dstW - width) / 2), even((dstH - height) / 2), width, height}; +} + +} // namespace corevideo::modules diff --git a/src/iso-provider-policy.h b/src/iso-provider-policy.h new file mode 100644 index 0000000..8c4a324 --- /dev/null +++ b/src/iso-provider-policy.h @@ -0,0 +1,17 @@ +#pragma once +#include +#include + +// One provider per participant stream prevents duplicate OBS sources from +// doubling audio. An alternate may take over after a 250 ms delivery gap. +inline bool iso_take_provider(std::string &owner, uint64_t &last, const std::string &source, + uint64_t now) +{ + if (now <= last) + return false; + if (!owner.empty() && owner != source && now - last < 250000000ULL) + return false; + owner = source; + last = now; + return true; +} diff --git a/src/iso-track-writer.cpp b/src/iso-track-writer.cpp new file mode 100644 index 0000000..40a62ed --- /dev/null +++ b/src/iso-track-writer.cpp @@ -0,0 +1,329 @@ +#include "iso-track-writer.h" +#include "iso-av-mux.h" +#include "iso-frame-conform.h" +#ifdef _WIN32 +#include "iso-d3d-conformer.h" +#else +#include +#endif +#include +#include +#include +#include +#include + +IsoTrackWriter::IsoTrackWriter(Config config) + : m_config(std::move(config)), m_thread([this] { run(); }) +{ +} +IsoTrackWriter::~IsoTrackWriter() +{ + { + std::lock_guard lock(m_mutex); + m_abort = true; + } + m_cv.notify_all(); + if (m_thread.joinable()) + m_thread.join(); +} +void IsoTrackWriter::fail(const std::string &error) +{ + std::lock_guard lock(m_mutex); + if (m_status.error.empty()) + m_status.error = error; +} +IsoTrackWriter::Status IsoTrackWriter::status() const +{ + std::lock_guard lock(m_mutex); + return m_status; +} +void IsoTrackWriter::close(uint64_t end_ns) +{ + { + std::lock_guard lock(m_mutex); + m_closing = true; + m_end_ns = end_ns; + } + m_cv.notify_all(); +} +bool IsoTrackWriter::wait_finished(int timeout_ms) +{ + std::unique_lock lock(m_mutex); + return m_cv.wait_for(lock, std::chrono::milliseconds(timeout_ms), + [&] { return m_status.done; }); +} +bool IsoTrackWriter::video(uint32_t w, uint32_t h, const uint8_t *y, const uint8_t *u, + const uint8_t *v, uint32_t sy, uint32_t suv, uint64_t ns) +{ + if (!y || !u || !v || w < 2 || h < 2 || w > 4096 || h > 2160 || ((w | h) & 1) || sy < w || + suv < w / 2) + return false; + Video frame{w, h, ns, std::vector(size_t(w) * h * 3 / 2)}; + auto *dst = frame.pixels.data(); + for (uint32_t row = 0; row < h; ++row) { + std::memcpy(dst, y + size_t(row) * sy, w); + dst += w; + } + for (auto *plane : {u, v}) + for (uint32_t row = 0; row < h / 2; ++row) { + std::memcpy(dst, plane + size_t(row) * suv, w / 2); + dst += w / 2; + } + std::lock_guard lock(m_mutex); + if (m_closing || m_status.done || !m_status.error.empty()) + return false; + // Coalesce delivery bursts; the independent output clock never loses time. + // Keep one second through GPU/encoder startup and delivery bursts. Shrinking + // this after the first output tick discards still-pending startup frames. + const size_t limit = 32; + while (m_video.size() >= limit) { + m_video.pop_front(); + ++m_status.coalesced_video; + } + m_video.push_back(std::move(frame)); + m_cv.notify_all(); + return true; +} +bool IsoTrackWriter::audio(const uint8_t *pcm, uint32_t bytes, uint32_t rate, uint16_t channels, + uint64_t ns) +{ + if (!pcm || !bytes || (channels != 1 && channels != 2) || rate < 8000 || rate > 192000 || + bytes % (channels * 2) || bytes > rate * channels * 2 / 2) + return false; + std::lock_guard lock(m_mutex); + if (m_closing || m_status.done || !m_status.error.empty()) + return false; + if (m_audio.size() >= 256) { + m_status.error = "ISO audio input backlog exceeded its bound"; + return false; + } + m_audio.push_back({rate, channels, ns, {pcm, pcm + bytes}}); + m_cv.notify_all(); + return true; +} +void IsoTrackWriter::run() +{ + IsoFfmpegPipe pipe; + audio_resampler_t *resampler = nullptr; +#ifndef _WIN32 + video_scaler_t *scaler = nullptr; +#endif + try { + // One encoder process owns both streams for the participant's lifetime. + // Matroska carries explicit timestamps across the single stdin pipe. + std::vector args = { + "-hide_banner", "-loglevel", "warning", "-n", "-f", "matroska", "-i", + "pipe:0", "-map", "0:v:0", "-map", "0:a:0", "-c:v", m_config.encoder}; + if (m_config.encoder == "libx264") + args.insert(args.end(), {"-preset", "veryfast", "-crf", "18", "-threads", "2"}); + else + args.insert(args.end(), {"-b:v", "12M", "-maxrate", "20M", "-bufsize", "24M"}); + args.insert(args.end(), {"-r", "30", + "-fps_mode", "cfr", + "-g", "60", + "-bf", "0", + "-color_range", "pc", + "-colorspace", "bt709", + "-c:a", "aac", + "-b:a", "192k", + "-ar", "48000", + "-ac", "2", + "-movflags", "+frag_keyframe+empty_moov", + m_config.path}); + std::string error; + if (!pipe.start(m_config.ffmpeg, args, m_config.log_path, IsoFfmpegPipe::kMaximumPixelBytes, + &error)) + throw std::runtime_error(error); + if (!pipe.try_queue(iso_av::header())) + throw std::runtime_error("ISO transport header rejected"); + constexpr uint64_t latency = 150000000ULL; + constexpr size_t ringFrames = 48000 * 4; + std::vector ring(ringFrames * 2, 0); + uint64_t index = 0, audioEnd = 0; + uint32_t rate = 0; + uint16_t channels = 0; + std::vector picture(1920 * 1080 * 3 / 2, 128); + std::fill_n(picture.data(), 1920 * 1080, 0); +#ifdef _WIN32 + corevideo::modules::D3DIsoFrameConformer conformer; +#else + uint32_t oldW = 0, oldH = 0; +#endif + for (;;) { + // Advance a fixed 30 Hz output clock, allowing input a short arrival window. + // A missing video frame retains the previous conformed picture. + Video newest{}; + std::deque