capture: add open capture protocol and viewer runtime - #713
Conversation
# Conflicts: # bun.lock
…scan-node # Conflicts: # bun.lock
![cursor[bot]](https://avatars.githubusercontent.com/in/1210556?s=60&v=4){kind=small}
There was a problem hiding this comment.
Cursor Bugbot has reviewed your changes using high effort and found 3 potential issues.
❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.
Want reviews to match your repository better? Bugbot Learning can learn team-specific rules from PR activity. A team admin can enable Learning in the Cursor dashboard.
Reviewed by Cursor Bugbot for commit 8dda820. Configure here.
| ? EMPTY_STREAM_EPOCHS | ||
| : retainLiveCaptureStreamValues(current.streamEpochs, event.descriptor), | ||
| } | ||
| }) |
There was a problem hiding this comment.
Stale live stream subscriptions
High Severity
captureSubscriptionStreamIds runs only against the initial descriptor before subscribe(). When a later descriptor event adds streams or flips them from pending to live, the existing subscription keeps the old streamIds filter, so new live streams never receive packets until something remounts the source.
Reviewed by Cursor Bugbot for commit 8dda820. Configure here.
| return inline.success | ||
| ? parseDeviceTrajectoryPayload(inline.data) | ||
| : parseDeviceTrajectoryPackets(packets.map((packet) => packet.payload)) | ||
| }, [layerKey, packets, stream.inline]) |
There was a problem hiding this comment.
Inline motion blocks live updates
High Severity
For deviceMotion, a successful parse of stream.inline always wins over live packets. Point-cloud layers correctly prefer live data first, so device-motion streams that seed an inline trajectory and then stream samples stay frozen on the snapshot.
Reviewed by Cursor Bugbot for commit 8dda820. Configure here.
| const sourceState = useCaptureSource(scan.captureSession, resolveSource, { | ||
| maxPacketsPerStream, | ||
| streamFilter, | ||
| }) |
There was a problem hiding this comment.
Layer toggle remounts session
Medium Severity
streamFilter is recreated whenever scan.layers changes and is listed in the useCaptureSource effect deps, so toggling one layer tears down the whole source: re-resolve, re-describe, clear packets, and remount every other visible layer.
Additional Locations (1)
Reviewed by Cursor Bugbot for commit 8dda820. Configure here.
1 participant
What does this PR do?
Adds the public capture-session foundation used by Pascal Capture and Community:
@pascal-app/capture-protocolfor extensible capture manifests, artifacts, coordinate frames, and live streams@pascal-app/capture-viewerfor room models, point clouds, surface meshes, and animated device motionHow to test
bun run check.bun run check-types && bun run test.bun run buildand confirm the editor and package builds complete.Screenshots / screen recording
Validated through the Community/Capture integration on the companion private branch.
Checklist
bun devbun checkto verify)mainbranchNote
Medium Risk
Touches published package APIs, ScanNode schema, and the npm release workflow. Additive and well-tested, but a bad schema or publish-order change can break consumers and uninstallable releases.
Overview
Adds
@pascal-app/capture-protocoland@pascal-app/capture-viewerso capture sessions are a host-owned extension of the viewer, not a private renderer or extra scene graph.Protocol owns versioned manifests (v1 RoomPlan-style normalized to v2 streams), locators, packet headers, and
CaptureSource(createHttpCaptureSource,PushCaptureSource). No React, Three, auth, or prescribed transport. Viewer runtime mountsCaptureRuntimeas aViewerchild, portals layers into the scan node, unmounts hidden sessions/layers, and ships reference model, device-motion, point-cloud, and surface-mesh renderers plus custom stream renderers.ScanNodenow stores an optionalcaptureSessionlocator, nullable meshurl, and an extensiblelayersvisibility map. Legacy GLB scans still parse. Scan nodes become movable/rotatable/scalable; the editor can host extra tree children under scans, move/hide captures, and copy a project ID.Release workflow can bump/publish the new packages, sync
dependenciesas well as peers, refresh the lockfile, and refuse a single-package publish if a required@pascal-app/*dep is not on npm.Reviewed by Cursor Bugbot for commit 8dda820. Bugbot is set up for automated code reviews on this repo. Configure here.