Fix participant ISO recording, isolated audio, and Tiles frame handoff

CHANGELOG.md

@@ -7,6 +7,35 @@ are tagged `vMAJOR.MINOR.PATCH` and published as
 
 ## [Unreleased]
 
+## [0.1.47] - 2026-09-15
+
+### Added
+- ISO feed selection: choose routed outputs, remember choices, and create files
+  only when selected feeds deliver media. TCP accepts `source_uuids`; OSC uses
+  saved panel selections. Empty selections no longer record every source.
+
+### Fixed
+- Each participant ISO is one continuous 1920x1080/30 H.264 MP4 with AAC audio.
+  Incoming resolution changes are scaled with aspect ratio preserved, without
+  splitting files. Duplicate output routes share the same participant recording.
+- ISO audio uses dedicated isolated participant subscriptions, independently of
+  the OBS source's Mix/Isolated setting, without changing Audience audio routing.
+- Participant workers hold video and fill silence through gaps. Startup timing,
+  full-range color, planar chroma, and batched audio timestamps are preserved.
+- Tiles retains valid pending pixels when a shared-memory read is rejected,
+  preventing a path that could flash incorrect colors during resolution changes.
+
+### Validation
+- Windows plugin/engine build and 72 native regressions pass. Real MP4 fixtures
+  cover eight concurrent NVENC tracks, changing resolutions/audio formats, gaps,
+  color preservation and flash/tone sync within 20 ms.
+- Alternating-speaker tests through routing, shared memory and MP4 encoding
+  confirm silence in each participant's file while the other speaker talks.
+- These are synthetic media checks; a live Zoom soak of the final isolated-audio
+  build has not been completed. Existing mixed recordings are not repaired.
+- Windows installer and ZIP include both updated plugin and engine. The signed
+  macOS installer still requires a separate build on the maintainer's Mac.
+
 ## [0.1.46] - 2026-09-14
 
 ### Fixed

CLAUDE.md

@@ -227,21 +227,27 @@ Every one of these is documented at length where it lives; the list is the map.
   interval rather than zeroing on fire -- zeroing discards the remainder
   that pushed a tick over threshold, which at 60 fps lands every 7 frames
   (~117 ms, ~8.6 Hz) instead of the documented 10 Hz.
-- **ISO recording timing** (`src/iso-video-pacer.h`, `src/iso-audio-gap-fill.h`):
-  raw video has no per-frame timestamps and ffmpeg cannot be trusted to
-  invent correct ones from a byte stream — `-use_wallclock_as_timestamps`
-  is confirmed (via `ffprobe -show_frames`, 2026-08-21) to have **no
-  effect** on this project's ffmpeg build's rawvideo demuxer, despite
-  looking like the textbook fix. `record_video_frame()` is called 1:1 with
-  Zoom's real, fluctuating (10-60fps) per-source delivery, so it must pace
-  itself to a fixed cadence (duplicate to backfill a stall, drop to shed a
-  burst) BEFORE the pipe — see `iso_video_frames_due()`. Audio has the
-  mirror-image problem for a different reason: Zoom only calls back audio
-  for someone currently talking, so `record_audio_frame()` must backfill
-  silence across every gap (`iso_audio_silence_frames()`) or the WAV
-  shrinks by every silent stretch. Both anchor to the same
-  `os_gettime_ns()` clock so video and audio stay in sync with each other,
-  not just individually correct.
+- **ISO participant A/V recording** (`src/iso-track-writer.cpp`,
+  `src/iso-av-mux.h`): one writer per Zoom participant ID owns a continuous
+  1920x1080/30 H.264 + 48 kHz stereo AAC MP4 until Stop. Source UUIDs and input
+  dimensions MUST NOT key writer lifetime. Workers scale/letterbox and resample,
+  hold video/fill silence against the same monotonic clock, and mux timestamped
+  raw media to one FFmpeg pipe. The input is full-range BT.709; preserve its
+  colour metadata. B-frames are disabled to avoid fragmented-MP4 startup A/V
+  offset. Keep startup video history while workers catch up: shrinking the
+  input queue on the first tick loses early frames. Validate decoded flash/tone
+  timing with `tests/verify-iso-av.py`, not only frame counts or process success.
+  ISO capture is explicitly selected by output source UUID (`source_uuids` in
+  TCP, saved panel choices for OSC). No default-all fallback. Do not pre-open
+  writers from configured participant IDs: selected routes open on first real
+  media delivery, so offline/unrouted sources produce no blank files. The feed
+  checklist updates rows by UUID and retains operator selections across refresh.
+  Audio MUST come from `IsoAudioTap`, not ZoomSource's embedded PCM (which may
+  be the meeting mix labelled with the route's participant ID). Audio-only IPC
+  subscriptions carry `recording_only=true`: receive one-way audio but never
+  claim a participant out of Audience routing. Install plugin AND engine for
+  this change. Each tap owns a fresh SHM reader, checks per-slot attribution,
+  preserves first buffers, and uses the recording epoch to reject stale callbacks.
 - **Colour range is normalised, never re-declared** (`src/i420-range-expand.h`,
   applied in `engine/src/engine-video.cpp`'s `onRawDataFrameReceived`): the
   engine requests `VideoRawdataColorspace_BT709_F` and the plugin declares
@@ -1574,6 +1580,16 @@ Home, download, and plugin docs point to `/download/#macos`.
 
 ## Media failure presentation (2026-09-06 soak)
 
+Tiles SHM reads must use `read_candidate`, never the pending `frame` directly.
+`shm_read_i420_frame` copies before validating its final sequence and can return
+Invalid with a modified destination. If `has_frame` was already true, reading
+into that frame published rejected pixels with the previous size/generation,
+causing a possible one-frame color flash during resolution changes. Commit only
+successful even-sized reads by swapping buffers (`zoom-tile-frame-read.h`).
+`CoreVideoTileFrameRead` injects rejected resized reads; it fails with the old
+direct-write behavior. Live diagnostics log `Tiles kept last valid frame` on
+the first rejected read and every 300 thereafter.
+
 `MediaFailureState` tracks current source media failures, bounded
 by live source assignments. Eight tiles × three failed attempts retain all
 24 raw error diagnostics but emit one nonmodal episode notice. Dock polling

CMakeLists.txt

@@ -285,6 +285,9 @@ if(COREVIDEO_BUILD_PLUGIN)
         src/zoom-diagnostics-dialog.cpp
         src/zoom-output-profile.cpp
         src/zoom-iso-recorder.cpp
+        src/iso-track-writer.cpp
+        src/iso-audio-tap.cpp
+        src/iso-audio-reader.cpp
         src/iso-ffmpeg-pipe.cpp
         src/zoom-iso-panel.cpp
         src/zoom-dock.cpp
@@ -317,7 +320,7 @@ if(COREVIDEO_BUILD_PLUGIN)
     if(WIN32)
         # winmm: talkback-cue.cpp's PlaySound(SND_MEMORY | SND_ASYNC) call --
         # see that file's header comment for why it was chosen over WASAPI.
-        target_link_libraries(obs-zoom-plugin PRIVATE crypt32 winmm)
+        target_link_libraries(obs-zoom-plugin PRIVATE crypt32 winmm d3d11 d3dcompiler)
         add_executable(CoreVideoOAuthCallback src/oauth-callback-helper.cpp)
         target_link_libraries(CoreVideoOAuthCallback PRIVATE ws2_32 shell32 ole32)
         install(TARGETS CoreVideoOAuthCallback
@@ -651,9 +654,37 @@ if(BUILD_TESTING)
     target_include_directories(CoreVideoIsoRecordingTest PRIVATE
         "${CMAKE_CURRENT_SOURCE_DIR}/src")
 
+    if(COREVIDEO_BUILD_PLUGIN AND WIN32)
+        add_executable(CoreVideoIsoTrackWriterTest tests/iso-track-writer-test.cpp
+            src/iso-track-writer.cpp src/iso-ffmpeg-pipe.cpp)
+        target_include_directories(CoreVideoIsoTrackWriterTest PRIVATE src)
+        target_link_libraries(CoreVideoIsoTrackWriterTest PRIVATE OBS::libobs)
+        add_executable(CoreVideoIsoIsolatedRecordingTest tests/iso-isolated-recording-test.cpp
+            src/iso-track-writer.cpp src/iso-ffmpeg-pipe.cpp src/iso-audio-reader.cpp)
+        target_include_directories(CoreVideoIsoIsolatedRecordingTest PRIVATE src)
+        target_link_libraries(CoreVideoIsoIsolatedRecordingTest PRIVATE OBS::libobs d3d11 d3dcompiler)
+        target_compile_definitions(CoreVideoIsoIsolatedRecordingTest PRIVATE NOMINMAX WIN32_LEAN_AND_MEAN)
+        if(WIN32)
+            target_compile_definitions(CoreVideoIsoTrackWriterTest PRIVATE NOMINMAX WIN32_LEAN_AND_MEAN)
+            target_link_libraries(CoreVideoIsoTrackWriterTest PRIVATE d3d11 d3dcompiler)
+        endif()
+    endif()
+
     add_executable(CoreVideoIsoEncoderPlanTest
         tests/iso-encoder-plan-test.cpp
     )
+    add_executable(CoreVideoIsoProviderPolicyTest tests/iso-provider-policy-test.cpp)
+    add_executable(CoreVideoIsoFeedSelectionTest tests/iso-feed-selection-test.cpp)
+    add_executable(CoreVideoIsoAudioRoutingTest tests/iso-audio-routing-test.cpp)
+    add_executable(CoreVideoIsoAudioReaderTest tests/iso-audio-reader-test.cpp src/iso-audio-reader.cpp)
+    target_include_directories(CoreVideoIsoAudioReaderTest PRIVATE src)
+    add_test(NAME CoreVideoIsoAudioReader COMMAND CoreVideoIsoAudioReaderTest)
+    target_include_directories(CoreVideoIsoAudioRoutingTest PRIVATE src)
+    add_test(NAME CoreVideoIsoAudioRouting COMMAND CoreVideoIsoAudioRoutingTest)
+    target_include_directories(CoreVideoIsoFeedSelectionTest PRIVATE src)
+    add_test(NAME CoreVideoIsoFeedSelection COMMAND CoreVideoIsoFeedSelectionTest)
+    target_include_directories(CoreVideoIsoProviderPolicyTest PRIVATE src)
+    add_test(NAME CoreVideoIsoProviderPolicy COMMAND CoreVideoIsoProviderPolicyTest)
     target_include_directories(CoreVideoIsoEncoderPlanTest PRIVATE
         "${CMAKE_CURRENT_SOURCE_DIR}/src"
     )
@@ -915,6 +946,9 @@ if(BUILD_TESTING)
     add_executable(CoreVideoTileTextureTest
         tests/tile-texture-test.cpp
     )
+    add_executable(CoreVideoTileFrameReadTest tests/tile-frame-read-test.cpp)
+    target_include_directories(CoreVideoTileFrameReadTest PRIVATE src)
+    add_test(NAME CoreVideoTileFrameRead COMMAND CoreVideoTileFrameReadTest)
     target_include_directories(CoreVideoTileTextureTest PRIVATE
         "${CMAKE_CURRENT_SOURCE_DIR}/src"
     )

docs/CORE_PLUGIN_FUNCTIONALITY.md

@@ -580,15 +580,24 @@ There are no Companion actions for talkback yet.
 
 ## Auto ISO Recording
 
-![CoreVideo ISO recording flow](images/iso-recording-flow.svg)
-
-ISO recording is controlled by the OBS plugin, not the engine. When enabled,
+ISO recording is controlled by the OBS plugin.
 On macOS, configure and test a working FFmpeg executable in **Zoom ISO Recorder**
 first; the package does not bundle FFmpeg.
 
-CoreVideo records one video file and one PCM WAV audio file per active source
-segment. A new segment starts when the resolved participant or source resolution
-changes.
+CoreVideo records one continuous MP4 per Zoom participant ID during Record/Stop.
+Each file contains H.264 video at 1920x1080, 30 fps and AAC stereo audio at 48 kHz.
+ISO audio comes from a dedicated one-way subscription for that participant,
+independent of the OBS source's Mix/Isolated setting. Recording subscriptions
+do not remove participants from the Audience mix. A muted/silent participant
+has silence in their MP4; other speakers are never substituted.
+Choose feeds in the ISO Recorder before starting. Choices are remembered by
+source UUID; no feeds are selected by default. Unrouted and unselected feeds do
+not start recordings. A selected feed opens its file only when its first actual
+audio or video arrives, so an offline route does not produce an empty recording.
+Incoming video is scaled to fit with black bars when needed. Zoom resolution
+changes, duplicate OBS sources, camera gaps, and source reassignment do not
+restart that participant's encoder or create another file. A new Zoom ID after
+a rejoin is treated as a new participant; display names are not identity keys.
 
 Requirements:
 
@@ -612,11 +621,13 @@ dock. The panel provides:
   requested encoder, actual encoder, and fallback state.
 - **Also start/stop OBS program recording** toggle.
 - **Start ISO Recording** and **Stop ISO Recording** buttons.
+- **Feeds to record** checkboxes, preserved through routing refreshes and OBS
+  restarts. Selection is fixed for the recording run; stop to change it.
 - Live status showing idle/recording and active session count.
 - Active session table with source, participant, resolution, video frame count,
-  audio chunk count, current video/audio file paths, and FFmpeg error details.
+  audio chunk count, combined file path, and FFmpeg error details.
 - Recently completed sessions remain in the table after stop so operators can
-  confirm completed MP4/WAV outputs before opening the folder.
+  confirm completed MP4 outputs before opening the folder.
 
 The panel uses the same `ZoomIsoRecorder` backend as the TCP and OSC APIs. It
 persists the output folder, FFmpeg path, and program-recording toggle in OBS
@@ -626,9 +637,14 @@ less than 2 GB free and warns below 10 GB free.
 TCP start example:
 
 ```json
-{"cmd":"iso_recording_start","output_dir":"C:/Recordings/CoreVideo","ffmpeg_path":"ffmpeg","record_program":true}
+{"cmd":"iso_recording_start","output_dir":"C:/Recordings/CoreVideo","ffmpeg_path":"ffmpeg","record_program":true,"source_uuids":["OUTPUT_SOURCE_UUID"]}
 ```
 
+`source_uuids` selects routed CoreVideo output sources. Omitting it uses the
+feed choices saved in the ISO panel; an empty selection is an error. OSC start
+also uses those saved choices. Multiple selected feeds for one participant
+share a single participant MP4.
+
 TCP status example:
 
 ```json
@@ -651,33 +667,25 @@ OSC equivalents:
 
 Output files are written as:
 
-- `*.mp4` for encoded I420 video through FFmpeg using the selected H.264
-  encoder
-- `*.wav` for matching PCM audio
-- `*.ffmpeg.log` beside them, holding FFmpeg's own account of the session. Read
-  this first when a file is missing or truncated.
+- `*.mp4` containing H.264 video and AAC audio for one participant.
+- `*.mp4.ffmpeg.log` containing encoder diagnostics.
 
 ### Timing
 
-Both files are paced to real elapsed time against the same clock, so each is
-individually accurate and the two stay in sync with each other.
-
-That is not free, and it is worth knowing why. Raw video carries no per-frame
-timestamps, and Zoom's per-source delivery fluctuates between roughly 10 and
-60 fps with conditions outside CoreVideo's control - so frames are paced to a
-fixed cadence before they reach FFmpeg, duplicating the held frame to backfill a
-stall and dropping excess from a burst. Audio has the mirror-image problem for a
-different reason: Zoom only calls audio back for someone currently making sound,
-so silence is backfilled across every gap or the WAV shrinks by the total silent
-duration. Both were wrong before v0.1.42/v0.1.43 - a source averaging 15 fps
-recorded under a declared 30 fps finished in about half the real duration, and an
-over-eager first gap-fill briefly doubled every WAV. If you see either symptom,
-check your version first.
-
-The hardware encoder fallback chain is NVENC -> QSV -> AMF -> libx264, and it
-walks the whole chain: a source demoted off NVENC on a machine with no working
-QSV or AMF runtime now reaches libx264, the tier with no hardware dependency to
-fail on.
+A worker per participant uses a monotonic clock to emit 30 video frames and
+48,000 audio samples per second. It holds the last picture during video gaps
+(or black before the first frame) and inserts silence during audio gaps. Input
+resolution and audio format changes are conformed on the worker without
+restarting the output. Files start on first media delivery. Participants first
+encountered later have a `start_offset_ms` in status.
+
+Timestamped video and PCM audio travel over one internal Matroska pipe to
+FFmpeg, which writes a single fragmented MP4. There are no separate temporary
+media files and no merge step at Stop. Producer buffers and encoder queues are
+bounded. An encoder failure is reported on that participant; it does not start
+an automatic replacement file. Hardware availability is tested by encoding an
+actual frame before recording starts. Automatic placement uses working NVENC,
+QSV, AMF, then libx264, accounting for OBS's NVENC sessions.
 
 When `record_program` is true, CoreVideo also starts the normal OBS program
 recording and stops it when ISO recording stops, but only if CoreVideo started

docs/iso-feed-selection-validation-2026-09-14.md

@@ -0,0 +1,30 @@
+# ISO feed selection
+
+Development build: `v0.1.47-dev.3`. Includes participant A/V recording and the
+pending Tiles frame-handoff fix. Not installed while the operator is in a meeting.
+
+The ISO panel now lists output feeds with checkboxes, defaulting to none.
+Selections are stored by source UUID, survive live routing refreshes, and are
+locked for the recording run. Unrouted/unsupported entries cannot be newly
+selected; a saved choice is retained if its route temporarily disappears.
+Start requires at least one selected routed feed. Disk and encoder estimates
+count only selected feeds, deduplicating fixed participant routes.
+
+The recorder validates selection independently of the UI and filters both
+audio and video callbacks. Starting no longer pre-opens writers for configured
+participant IDs. Files open only on media delivery from a selected route.
+Once opened, normal gap handling and one-file-per-participant ownership remain.
+
+TCP `iso_recording_start` accepts `source_uuids`, an array of output UUIDs;
+omission uses saved panel choices, and explicit empty selection is rejected.
+OSC start uses saved choices. Neither API falls back to recording every source.
+Status exposes `selected_source_uuids`; the panel shows an armed/waiting state
+when recording is enabled but no selected feed has delivered media.
+
+Windows full build succeeded and all 70 CTest regressions passed. The new
+selection-policy regression covers selected/unselected, unrouted, unsupported,
+empty selection, and empty UUID cases. Live UI and recording acceptance remain
+pending the operator's meeting ending and installation: select two routed
+feeds, leave others unchecked, verify only selected participants create files,
+and confirm offline/unrouted feeds create none. Check saved choices after OBS
+restart and a routing refresh.

docs/iso-isolated-audio-validation-2026-09-15.md

@@ -0,0 +1,36 @@
+# Dedicated ISO audio validation
+
+Build `v0.1.47-dev.4` includes the feed picker, Tiles handoff fix, and dedicated
+participant audio recording. Both plugin and ZoomObsEngine must be installed.
+
+Root cause: ZoomSource forwarded its embedded audio to the ISO writer. Sources
+configured as Mix received mixed PCM labelled with the configured participant
+ID by the engine. A participant ID alone therefore did not prove isolation.
+Decoded audio envelopes from the last show's participant files were strongly
+correlated (two pairs 0.9956 and 0.9467), consistent with this path.
+
+ISO now owns an audio-only subscription per recorded participant. Source PCM
+is never passed to the writer. The engine forces these subscriptions to receive
+only that participant's one-way audio and excludes them from the set of output
+routes that claim participants away from Audience audio. Per-slot attribution
+is validated again at the recorder's dedicated SHM reader.
+
+Validation completed:
+
+- Full Windows build and 72 CTest regressions passed.
+- Routing regression rejects mix and other participants while retaining normal
+  isolated, meeting-mix, and Audience behavior.
+- Real SHM reader regression retains first/coalesced buffers, does not duplicate
+  drained buffers, clears notifications, and rejects another participant ID.
+- End-to-end synthetic alternating speakers through the production routing
+  policy, real shared-memory reader, resampler, and FFmpeg MP4 writers passed.
+  Decoded one-second RMS for participant 1: 8490.78, 0, 8485.57, 0.
+  Participant 2: 0, 8482.77, 0, 8482.81. Non-speaking intervals were digital silence.
+  Both files decoded completely without errors.
+
+Reproduce: run `CoreVideoIsoIsolatedRecordingTest <ffmpeg>` in an empty directory,
+then `python tests/verify-iso-isolated.py <ffmpeg> <directory>` from the repo root.
+These are synthetic SDK-input tests, not a live Zoom meeting. Live acceptance
+still requires two people alternating speech and checking their respective MP4s.
+Existing mixed recordings are unchanged; this fix cannot recover original
+isolated stems that were never captured.