Aligned multicam timeline for video, time offsets, and multicam fixes by romleiaj · Pull Request #1996 · Kitware/dive

romleiaj · 2026-09-28T15:09:48Z

Makes the aligned multicam timeline work for video as well as image sequences, and fixes several multicam pipeline bugs. Each camera can now carry a time offset, set from a slider in Multi Camera Tools and
saved with the dataset. The offset lines the cameras up for review and playback, can be applied to that camera's annotations on the server, and pairs 2-cam/3-cam pipeline inputs at matching instants. While an
offset hasn't been applied yet, annotation editing is paused, because edits would land on a different frame than the one shown.

Under an aligned timeline, the timeline chart and the track list show timeline slots, so their numbers match the playhead. Zoom is now kept when panels open, when the offset changes, and when Align View's
delayed re-fit runs. Apart from the notes below, datasets with no offset behave as before.

Also fixed: VIAME no longer crashes when VIRTUAL_ENV is set, warp pipelines request the renamed homography_json reader, multicam pipeline inputs are capped at the shortest camera, warped boxes are clipped to
the target camera's frame, annotation undo is reset after an offset is applied, and multicam datasets load faster (camera configs load in parallel, annotations still one camera at a time, and track insertion
no longer sleeps).

Things to watch:

A zoomed pane now keeps its view when the layout resizes (window, sidebar, panels, controls height) on every dataset, including single-camera. Untouched panes still refit.
Timestamp-aligned multicam datasets show timeline slots in the timeline chart and track list even with no offset, and the list's seek buttons go to those slots.
A nonzero offset takes precedence over filename timestamps on timestamp-aligned datasets.
Multicam pipeline inputs are capped at the shortest camera even with no offset set; video frame counts for that come from the container header.
Web warp results clip boxes to the target frame and drop boxes less than 25% inside it.
Transcodes put the MP4 index at the start of the file.
Registration JSON may now include frameOffset.
Applying an offset creates a new annotation revision across every annotation set. On desktop, it rewrites the local track file.

The worker image sets VIRTUAL_ENV to DIVE's own Python 3.11 venv, and `uv run` re-exports it to the Celery process, so every pipeline subprocess inherited it. The VIAME image's kwiver ships a second Python plugin loader (plugins_from_python) that reads VIRTUAL_ENV and edits sys.path via PySys_GetObject without holding the GIL after modules_python has already started the interpreter. The result is a SIGSEGV at plugin load for every `viame runner` invocation, before any pipeline process runs, with nothing useful on stderr. get_gpu_environment already strips the venv from PATH for the same reason; now it also drops VIRTUAL_ENV and the UV_PYTHON* variables. The variable has to be absent rather than empty because kwiver only checks getenv for null. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01EKg5t3obzg2v7yrjCR47VZ

VIAME renamed its DIVE registration reader on 2026-08-25 (b540a933b, "Rename dive_transform_io to homography_json_io"): the transform_2d_io implementation is registered as "homography_json" now, not "dive". Every VIAME built since then, including the web worker image, refuses the warp pipelines with Could not find implementation "dive" for "kwiver::vital::algo::transform_2d_io" from key "transform_reader:type" viame: Caught unhandled std::exception: Unable to create transform_reader Send the new name from both the web task and the Desktop backend. kwiver has no fallback for an algorithm type, so a VIAME older than the rename needs rebuilding before its 2-cam/3-cam warp pipes run again. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01EKg5t3obzg2v7yrjCR47VZ

DIVE lets the cameras of a rig differ in frame count, but the 2-cam and 3-cam pipes run in lockstep and kwiver's detected_object_output grabs its optional image_file_name port unconditionally. When one camera ran out first the writer for the other tried to read the "complete" marker as a string: RuntimeError: Failed to cast datum of type 'complete' into std::string: bad any_cast. Before launching, take the native frame count of each video camera from its folder's ffprobe metadata and the list length of each image camera, and when they disagree cap every camera at the minimum: image lists are truncated, video readers get vidl_ffmpeg:stop_after_frame (1-based, so exactly that many frames). A registration frame subset already pairs row for row and is left alone. The job log says when the cap is applied. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01EKg5t3obzg2v7yrjCR47VZ

buildAlignedTimeline needs a timestamp on every frame, which only image sequences carry (parsed from filenames). Video panes never qualify, so they scrub in raw-index lockstep and a recording start offset between two cameras is baked into the review -- a fixed EO/IR rig whose encoders start a fraction of a second apart has no way to express that today. On a fixed rig the offset is one constant per camera, so buildOffsetTimeline emits the same slot structure buildAlignedTimeline does from just that number. Everything downstream (pane seek, gap indication, cross-camera frame translation) then behaves exactly as it does for a timestamp-aligned dataset. Slots span the union of coverage, so the non-overlapping ends blank through the existing gap handling rather than being trimmed away. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01K9imLktmzCkAR3wmUybaEs (cherry picked from commit 821a3469561e4180d612a136ef22e1d16d48885f)

Holds the offsets on CameraRegistrationStore, beside the spatial transform -- they are the other half of how two cameras line up, and persist with the same dataset save. The viewer now falls back to buildOffsetTimeline when no frame carries a timestamp, which is always the case for video panes, so a fixed rig whose encoders started a fraction of a second apart can finally express that. An all-zero offset still yields { aligned: false }, keeping uncorrected datasets on the existing positional path. Also extracts the per-camera frame count the auto-register service was computing inline, since the timeline needs the same video-aware lookup, and moves the registration store's construction above the timeline watch that now reads it. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01K9imLktmzCkAR3wmUybaEs (cherry picked from commit 149942d72d69044789d7ace263e2d7f33a574cba)

Sits under Overlay Warp because the warp is how the value gets judged: scrub with the overlay on and only the correct offset makes moving subjects line up. The fit is no guide -- a static background matches at every offset, so nothing in the registration stats can confirm it. Slider plus single-frame nudge buttons, bounded at +/-3s of the dataset's frame rate, with a frames-and-seconds readout. Only the right camera moves; the left stays the time reference. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01K9imLktmzCkAR3wmUybaEs (cherry picked from commit e7de7c1d888d1cfd1e482ce85faef0f16f8c6391)

Adds cameraFrameOffsets to the mutable dataset config (client type, allowlisted mutable keys, and the server metadata model) and threads it through the two save paths and all three hydrate paths, so an offset dialed in during review survives a reload and travels with the dataset the same way the registration does. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01K9imLktmzCkAR3wmUybaEs (cherry picked from commit ee4d66b8b36e81012d9649a5d8949bf97d2f41c2)

Adds an optional per-pair frameOffset to the portable registration JSON: the right camera's start offset in its own frames, relative to the left. A producer that measured it (the batch registration driver) now hands it over with the transform instead of leaving the reviewer to rediscover it, and a correction made in the panel travels back out on export. Read on file load and through the multicam import seed, merged per camera so a later file wins one camera's offset without clearing another's, and omitted from written files when a camera is in step. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01K9imLktmzCkAR3wmUybaEs (cherry picked from commit 8b8191ed0ee3c3dd3b6c719234989a22e632ea0f)

Pipeline inputs pair positionally -- row i of one camera's list with row i of every other's -- so a rig whose recorders started at different times fed a 2-cam/3-cam detector mismatched instants, and every cross-camera association it made was quietly wrong. Only the auto-register path was safe, because it builds its imagePairs subset from aligned-timeline slots; every ordinary run took the raw-index path. Image-sequence cameras now drop the frames before their first paired instant and stop where the shortest camera does. Video cameras get the same correction as a seek (video_input's start_at_frame) rather than being extracted. Datasets with no offset set are untouched, taking exactly the previous path, and offsets that leave no overlapping span raise a clear error instead of writing empty input lists. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01K9imLktmzCkAR3wmUybaEs (cherry picked from commit 5b544c2568ed7f73599e23b3b2cc73643b489f7d)

The offset correction seeked each video to its first paired frame but left the tail open, so unequal-length recordings still ran on past each other -- which for video is not merely wasted work: one input completing while another still has frames desynchronizes the pipeline, and a writer takes a 'complete' datum where it expects data. Each video input now also carries stop_after_frame. Bounding the tail needs a frame count per camera, which a video only knows after a probe, so pairedStartFrames now takes counts rather than digging them out of the image list. That also fixes an all-video rig computing a zero-length span from image counts that were never there -- the case the previous commit would have wrongly refused to run. The probe reads the container header rather than decoding: counting frames for real would add tens of seconds per camera to every launch, and it only runs when an offset is actually set. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01K9imLktmzCkAR3wmUybaEs (cherry picked from commit fb9600090241b4809df1d386caa914ea55e4a260)

The time-offset branch corrected Desktop pipeline inputs but never the web task, so a web run of a 2-cam/3-cam pipe still paired frame i of each camera regardless of the stored cameraFrameOffsets. Port the same arithmetic: each camera skips to its first paired frame and every camera stops where the shortest does, sliced for image lists and set as start_at_frame/stop_after_frame on the video reader. Registration frame subsets are left alone since they already pair row for row. A dataset with no offset keeps the plain shortest-camera cap. The Desktop side set start_at_frame and stop_after_frame on the video_input process, which has no such keys, so the seek was silently ignored. Address the reader (video_reader:vidl_ffmpeg) instead. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01EKg5t3obzg2v7yrjCR47VZ

Installing an aligned-timeline resolver always seeked to slot 0. With a camera time offset the timeline starts with the earlier camera's lead, where the other camera has no frame yet, and every Time Offset nudge rebuilds the timeline -- so each nudge jumped the viewer to the start and blanked a pane with "No frame at this instant", which made the slider unusable for the very judgement it exists for. When a resolver replaces another, take the first camera's displayed local frame and seek to its slot in the new timeline instead. A fresh install still starts at slot 0. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01EKg5t3obzg2v7yrjCR47VZ (cherry picked from commit 38e5a499554beefca1c959d21825199cdeae5efe)

The offset slider lived in the Camera Registration panel, tied to that panel's left/right pair selection. It now sits in Multi Camera Tools with one control per non-reference camera (the first camera in display order is the reference, as in registration), so it no longer depends on which pair is picked. A "Save annotations" button moves every annotation on a shifted camera by its offset so the boxes line up with that camera's own video, then saves. The part of each offset already applied is recorded as cameraFrameOffsetsApplied, so a later nudge only shifts by the difference and the button disables when nothing is pending. Features that would land before frame 0 are dropped. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01EKg5t3obzg2v7yrjCR47VZ

Nudging the Time Offset moved a camera's video and its annotations together, so the boxes never disagreed with the frame under them and there was nothing to line up. Each camera pane now reads its annotations at `frame` minus that camera's unapplied offset, so while the reviewer dials the offset the boxes stay at the instant the reference camera shows and only the video underneath moves; Save annotations then bakes the offset in and the two coincide again. The first nudge on an uncorrected dataset also still jumped to frame 0: the earlier keep-the-instant fix only covered replacing one timeline with another, and a dataset with no offset has none, so a fresh install started at slot 0. Take the frame positional playback was showing in that case. Widen the slider to +/-10 s. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01EKg5t3obzg2v7yrjCR47VZ

Two playback problems with a camera time offset on a video rig. Playing fought itself. play() started every <video> element and, with an aligned timeline installed, the aggregate tick also re-seeked both videos every frame. Two free-running decoders yanked around by async seeks at 30 Hz cannot stay in step, which is the "not in lockstep" scrubbing and playback. When every camera is a video, play now aligns each pane to the slot once and lets the elements free-run; the global slot follows the reference camera's own clock, and pause snaps every pane back onto the slot so any drift is gone. Mixed image/video rigs keep the centralized tick. Scrubbing froze the picture while the boxes moved. geojs skips rendering a video quad while any of its videos is seeking (delayRenderWhenSeeking, default true), so under a continuous scrub two decoding streams never got a render until the scrub stopped, and a zoom was the next thing to force one. Both the native video quad and the aligned-view warp now render whatever frame is decoded. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01EKg5t3obzg2v7yrjCR47VZ

onResize refit every pane to its native bounds whenever its container changed size, so any change in the strip under the panes -- the controls growing a row, a gap indicator appearing on an offset timeline -- threw away the reviewer's zoom on the camera they were inspecting. Only a pane still at its fitted view is refit now; a pane the user has zoomed or panned keeps that view across the resize. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01EKg5t3obzg2v7yrjCR47VZ

This reverts commit 93debc9.

Turning delayRenderWhenSeeking off was a mistake: geojs's canvas renderer then paints immediately, and its video quad draw skips a video that is still seeking, so a seek cleared the pane and nothing retried -- a black or stale frame until a pan or zoom forced a draw. With the default back on, the renderer retries every animation frame until the seek lands and paints the new frame. That retry is also why a continuous scrub of two videos never painted: every slider event issued a new seek before the previous one finished, so a quad was always seeking. The video annotator now holds the newest request while a seek is in flight and issues it on 'seeked', so each landed seek paints before the next goes out. The annotation frame and shared time still follow the slider immediately. The resize handler also ignores sub-pixel differences between the pane's box and the geojs map size, since a "resize" refits the pane and discards the reviewer's zoom. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01EKg5t3obzg2v7yrjCR47VZ

warp_detections maps each box's corners through the homography and keeps the bound, with no notion of the target frame, so whatever EO sees outside IR's narrower field of view came back at negative or oversized coordinates and hung off the IR pane. After a warp pipeline finishes, the web task now clamps every box in each camera's output CSV to that camera's frame and drops rows with less than a quarter of their area inside (a zero-area box must lie inside). Frame size comes from the video's ffprobe metadata or the first image of a sequence; when neither is known the CSV is uploaded untouched with a warning in the job log. Only runs that handed the pipe a registration are affected. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01EKg5t3obzg2v7yrjCR47VZ

Applying a time offset used to rewrite every track on the shifted camera in the browser: a synchronous remove/rebuild/insert loop that is quadratic in track count (each store removal scans and splices the reactive id array), followed by an upload of the entire dataset as pending upserts and a full pydantic re-validation on the server. On detection-per-track datasets that blocked the page long enough for the browser's "wait or exit" dialog. The rewrite now happens where the data lives. A new PATCH /dive_dataset/{id}/camera_frame_offset takes a camera and its offset, shifts only the not-yet-applied part of that camera's tracks and groups across every annotation set as a new revision, and writes the offset and its applied record in the same update so a failed shift never leaves the record ahead of the data. The desktop Express server gains the same route over its track file, and desktop config save now persists the offset keys, which it had been dropping. The client sends one number per camera, saves pending edits first so the shift sees current data, and reloads just that camera through the same yielding bulk insert the initial load uses. Track shifting rebounds begin/end to the surviving features instead of clamping at zero, since the server's Track validator requires begin and end to match the first and last feature. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>

A multicamera dataset loaded one camera after another: config, then annotations, then track insertion with a hard 500 ms sleep every 4000 tracks, and only once every camera was through did the annotator panes mount and the videos begin to download. Two large cameras spent seconds sleeping and seconds waiting on serialized requests before either video had fetched a byte. Every camera's config and annotations are now requested at once, the media is applied for all cameras first, and a new progress.mediaLoaded flag mounts the panes at that point so the videos fetch their metadata while the annotations are still being inserted. The insertion loop yields with a zero-delay timeout instead of sleeping; the yield was only ever there to let the page paint. A progress ring overlays the panes until the annotations are in, and the pane keybindings stay off until then. Both transcodes also write the MP4 index up front (faststart), so a browser can read the video's metadata from the first bytes instead of reaching to the end of the file first. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>

The aligned timeline tried filename timestamps first and consulted the reviewer's per-camera offsets only when no timestamps were usable. On an image sequence with a timestamp on every frame that meant the Time Offset slider moved the annotations but never the images: the panes stayed lined up by capture time while the boxes drifted, which reads as the annotations being wrong. Pipelines pair cameras by index, not by capture time, so annotations warped across a rig are index-aligned regardless of what the filenames say; the offset exists to correct exactly that. Prefer the offset timeline whenever any camera's offset is nonzero, and keep the timestamp timeline, with its gap handling, for datasets that have no offset set. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>

Applying a camera time offset shifts that camera's tracks on the server and then reloads only that camera through reloadCameraAnnotations. The undo history kept its snapshots from before the shift, so Ctrl+Z put a track back at its unshifted frames, and the next save wrote it to the server that way while the dataset still recorded the offset as applied. The first edit after the reload had the same problem, because its "before" snapshot came from the stale baseline. Re-baseline the history once the reloaded tracks and groups are in, the same way the full reload path already does. MultiCamTools saves pending edits before applying an offset, so no unsaved work is lost with the history. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

Reformat two multi-context `with` statements the way black 26.5.1 now wants them, sort the multicam_pipeline test imports, and drop an unused MagicMock import from the camera frame offset tests. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

Until "Apply to annotations" runs, a shifted camera's pane draws its annotations at the video frame minus the offset, but the mode manager keys edits by the video frame (selectedCameraFrame). A box drawn there landed on a different frame than the one on screen, seemed to vanish, and was then shifted a second time when the offset was applied. Some edits went through LayerManager with the displayed frame and others through the video frame, so one session could write to both. Treat the dataset as read-only for editing while any camera has an unapplied offset. The lock covers every camera, not just the shifted one, because cross-camera writes (stereo copies, extending a detection to another camera) reach the shifted camera from the others. Saving is left on the plain readonlyState, since applying an offset first saves pending edits. A new offsetEditLock provide lets Multi Camera Tools keep "Apply to annotations" enabled under this lock while still disabling it in a truly read-only view, and it tells the user why editing is paused. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

Stepping a video forward updated `frame` right away, and LayerManager drew that frame's annotations over the picture still showing the old frame until the seek landed. The boxes visibly jumped ahead of the video on every step. Batching seeks behind the in-flight one can make that window longer. Annotations now draw at `syncedFrame`, which a video annotator only advances on 'seeked'. The annotator records which frame each seek is for rather than recomputing it from currentTime, because kwiverSeek times do not always round back to the requested frame when the dataset's frame rate differs from the video's. Image annotators already set syncedFrame together with frame, so their behavior is unchanged. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

The timeline's track bars, keyframe markers and count chart took each track's begin/end and feature frames straight from the cameras' stored frame numbers, while the playhead moves through aligned-timeline slots. Once a time offset shifts the timeline, those numberings disagree: on an EO/IR/UV sequence with one camera nudged by a frame, a detection that appeared at slot 1 was charted on frames 2-3. Under an aligned timeline, the Viewer now maps each camera's replica of a track through cameraFrameToSlot and hands the charts the resulting range and features; without one, the charts read the stored frames as before. The group chart is unchanged. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

The track list showed each track's begin/end as the minimum/maximum of its stored frames across cameras, in the cameras' own numberings, while the playhead and timeline count aligned slots. With a time offset set, the list disagreed with the timeline, and seeking to a track's start translated that mixed number through the selected camera, landing off by the offset. A trackTimeline provide gives each list row the track's slot range and a seek straight to a slot. The bottom list shows those begin/end frames and timestamps, both lists seek to them from the buttons and home/end keys, and start/end sorting follows the same range. Without an aligned timeline the range is null and rows use the stored frames and the existing seek, as before. Keyframe navigation and editing still use the selected camera's own frame, which is what useTime reports. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

Annotations draw at syncedFrame, but the image annotator advanced it the moment a frame was requested, while an uncached image loads asynchronously. Stepping mostly hits cached frames, so the gap was invisible; scrubbing requests uncached ones, and the next frame's box (often interpolated to a different size) sat over the previous image until the new one loaded, then snapped. syncedFrame now advances when the frame's image is drawn: at once for a cached image, after onloadPromise otherwise. onloadPromise also resolves on a load error, so a failed image cannot leave annotations stuck. LargeImageAnnotator is unchanged. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

A pane draws its annotations at syncedFrame minus the camera's unapplied shift. Moving the slider raised the shift at once, but the video only reaches the matching frame when the seek the rebuilt timeline issues lands. In between, the new shift was subtracted from the old on-screen frame, so the box was drawn from the wrong frame, a different size on an interpolated track, until the seek landed and it snapped back. The +/- buttons seek once and land quickly; dragging queues seeks, so the mismatch lasted long enough to see. Each camera now adopts a changed shift only once its on-screen frame equals its requested frame, i.e. once the seek paired with that shift has landed. The watch runs after the timeline rebuild in the same tick has updated the requested frame, so it cannot adopt the new shift early. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

'seeked' only means the seek finished; the browser can fire it before the decoded frame is presented, so on a fast scrub the boxes moved to the new frame while the picture still showed the old one. The landed frame now becomes syncedFrame from requestVideoFrameCallback, which fires when that frame is presented. A seek onto the frame already shown presents nothing new, so a 250 ms timeout applies it instead, and a landing on the frame already synced applies at once. Only the newest landing applies, and playback cancels a pending one since it drives syncedFrame itself. The aligned-view warp's imageRevision bump moves with it, so the warp also snapshots the painted frame. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

…ainted" This reverts commit 22cd23a. Waiting for the painted frame still left the boxes a paint behind GeoJS's own video redraw and made scrubbing stutter; video goes back to drawing at the requested frame instead. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

Holding a video's boxes until the seek landed kept them in step with the picture in principle, but GeoJS repaints the video quad on its own animation loop, so the boxes still trailed or jumped a paint apart and scrubbing felt awkward. Main draws a video's annotations at the requested frame the moment a seek is issued: the boxes lead the picture by the seek latency but move smoothly with the scrubber. A video pane's annotationFrame is now its requested frame minus the camera's unapplied shift, as on main. A Time Offset nudge changes the requested frame and the shift in the same tick, so the slider stays steady without waiting for the seek. Image panes keep drawing at syncedFrame with the settled shift, so their boxes still wait for their image. Seek coalescing stays, since it is what lets two offset videos paint during a continuous scrub. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

Video annotations draw at the requested frame, leading the picture while a seek lands. The new User Settings switch "Experimental: sync video annotations to the painted frame" (annotatorPreferences.videoPaintSync, on by default) draws them with the frame the browser has actually painted instead; switching it off restores main's behavior. With it on, every seek the video annotator issues watches for the next painted frame through requestVideoFrameCallback. The callback maps the frame's mediaTime back to a DIVE frame (videoTimeToFrame, the inverse of kwiverSeek, preferring the requested frame when several DIVE frames show the same video frame), sets syncedFrame, and redraws the video quad, so the annotation layers and the video paint in the same GeoJS animation frame. It keeps watching while seeks are in flight, so intermediate frames shown mid-scrub get their own boxes. A landing that paints nothing new settles after 250 ms, and playback ignores the callback since it drives syncedFrame itself. Browsers without requestVideoFrameCallback fall back to syncing when the seek lands. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

This reverts commit de54564. Syncing video annotations to the painted frame did not look right in testing; video goes back to drawing at the requested frame, as main does. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

onResize reset every pane whose container changed size, so opening a context panel such as Multi Camera Tools, or a Time Offset nudge that changes the timeline's gap row and with it the controls' height, threw away the zoom the reviewer had set, including under Align View. Each pane now remembers the view its last reset produced: the per-pane native fit, or the Align View reference fit when the aggregate reset goes through the aligned override. On a resize, a pane still at that view refits to the new size as before, and a pane the user has zoomed or panned keeps its zoom and center. Under Align View the reference pane's kept view is then copied to the others by the existing resizeTrigger re-link. An earlier version (59e8612, reverted in 7423a86 without a recorded reason) judged "fitted" against the pane's native bounds only, which is not where Align View leaves a pane. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

Turning Align View on fits the reference frame and arms a one-shot watch on each camera's imageRevision, so imagery that lands late gets a second fit. A video pane only bumps imageRevision when a seek lands, so after turning Align on and zooming without seeking, the watch stayed armed: the first seek afterward, such as the first Time Offset nudge, fired it and threw the zoom away. Later nudges kept the zoom because the watch had already been used up. snapFromReference now records the reference view it produced, and the late re-fits (the nextTick/animation-frame follow-ups and the imageRevision watch) run only while the reference pane still shows that view. A late image still re-fits an untouched Align-on view. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

Puts three changes that reach beyond multicam back to main's behavior, to judge the branch without them: - VideoAnnotator sets currentTime on every seek again instead of holding a new seek behind the one in flight, and syncedFrame is read back from currentTime on 'seeked'. - ImageAnnotator sets syncedFrame as soon as a frame is requested again, so an image pane draws the new frame's annotations while its image is still loading. - The camera panes mount once annotations are loaded (progress.loaded), with the full-page loading indicator until then, rather than at progress.mediaLoaded with a ring over the panes. Cameras still load in parallel. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

Opening a multicam dataset fetched every camera's config and annotations all at once. The total server work is unchanged, but the annotation request is the heavy one (every track read from Mongo and serialized), so a large rig put several of those in flight per open, sharpening peak load when many users open datasets together. Configs are small and still load in parallel. Annotations now load one camera at a time, alongside the configs, so each open holds at most one annotation request in flight, as on main. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

romleiaj force-pushed the dev/multicam-time-offset branch from 5e036dc to d80e8e8 Compare September 28, 2026 15:12

romleiaj and others added 29 commits September 28, 2026 16:32

Revert "Keep a zoomed pane's view when the layout resizes it"

7423a86

This reverts commit 93debc9.

Fix server lint failures

ca49687

Reformat two multi-context `with` statements the way black 26.5.1 now wants them, sort the multicam_pipeline test imports, and drop an unused MagicMock import from the camera frame offset tests. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

romleiaj and others added 6 commits September 28, 2026 16:32

romleiaj force-pushed the dev/multicam-time-offset branch from de54564 to 6692169 Compare September 28, 2026 20:37

romleiaj and others added 3 commits September 29, 2026 09:35

romleiaj changed the title ~~Multicam Fixes~~ Aligned multicam timeline for video, time offsets, and multicam fixes Sep 29, 2026

romleiaj requested review from BryonLewis and removed request for BryonLewis September 29, 2026 16:57

Navigation Menu

Aligned multicam timeline for video, time offsets, and multicam fixes - #1996

Aligned multicam timeline for video, time offsets, and multicam fixes#1996
romleiaj wants to merge 39 commits into
mainfrom
dev/multicam-time-offset

romleiaj commented Sep 28, 2026 •

edited

Loading

Uh oh!

This branch has not been deployed

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

Conversation

romleiaj commented Sep 28, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

This branch has not been deployed

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

romleiaj commented Sep 28, 2026 •

edited

Loading