[hougantc-nvda]

Description

Review stack · 2/5: #8787 → #8788 → #8789 → #8790 → #8791

Review only this PR’s changes. The PRs target main; the comparison isolates this layer from its predecessors.

Connect live Capture tracking and recorded MCAP input to kitless simulation through the OVXR adapter from #8787. SO101 gains CPU OvPhysX selection and the current Capture home-transform, clutch and gripper integration.

• Public entrypoints: workflows live in isaaclab_teleop.entrypoints; existing script paths delegate to those package entrypoints. Task discovery, callable CLI argument restoration and exit status follow upstream dispatch.
• Teleop integration: isaaclab_teleop owns control events, target-frame rebasing, input policy and session cleanup; task-specific action preparation remains with the task.
• Runtime selection: use upstream non-Kit keyboard/gamepad/SpaceMouse support and Capture 1.7 APIs. Kit keeps its separate --xr path; kitless uses --ovxr.
• Replay: MCAP supplies tracker/control input while the connected headset supplies the view. Reset resets the environment without rewinding the recording. HDF5 recording/replay remains on the Kit path.

Shared fixes address empty CPU Warp arrays, Newton graph capture overlapping legacy-stream uploads, and simulation-context ownership after a failed asynchronous drain. Kit fixes defer native USD imports until runtime launch and keep headless XR updating without changing ordinary headless on-demand capture.

See the live/replay runtime guide.

Dependencies: compatible Isaac Capture OVXR packages and the provider upgrade in #8858. Native-provider defaults are unchanged.

Type of change

• New feature
• Bug fix
• Documentation update

The public source-installation guide defines a separate optional XR selection without changing project-wide provider defaults. If input retirement fails, the entry points request worker shutdown without acknowledging quiescence or destroying borrowed resources.

Testing

The complete stack is based on upstream main 0ca25c56eec6fe7860fdcba44f8848f19693d6f6, including #8845. These are targeted checks, not a full-repository or per-PR runtime qualification. Kitless runs use Python 3.12 and Isaac Capture ae48621f1cc13054c2246210d8f92d30e27fe354 built from source.

• 187 affected tests passed across six files. Regressions reproduce the original missing lease method, unresolved-write cleanup, reset admission ordering, render-anchor height/rotation, camera metadata/layout, and waist-task safeguards before their fixes. The first-scene regression uses the actual lease class.
• All five PR boundaries independently pass full pre-commit checks and affected API/entrypoint tests. Fixes are placed in their owning predecessor and propagated through the stack.
• Omen IWER: G1 locomanipulation CPU passed stereo/task-object, PiP, mirror, meaningful controller/base motion, Reset, reconnect/resumed input, and normal mirror-close shutdown. GR1T2 waist-enabled split-GPU Newton passed stereo, mirror, hand-driven motion, Reset/resumed motion and normal mirror-close shutdown.
• G1’s isolated warm decoded-video median was 45.98 frames/s, compared with 47.60 at the preceding tested checkpoint, using the same scene, seed, input and PiP/mirror settings (three 10-second windows). No dropped frames or freezes occurred in the isolated windows. An earlier 44.83 measurement overlapped the documentation build and is retained as a confounded result. These rates are not physics-control or physical-headset FPS; this is a bounded prior-run comparison.
• The locked XR environment passes its dependency check. Sphinx generated the documentation but its warning-as-error check failed on six external intersphinx DNS failures; no other Sphinx warnings/errors were reported.

The preceding baseline also passed 205 distinct selected Lab/Kit tests, five kitless IWER scene variants, and ordinary headless Kit Franka XR motion/Reset. That broader matrix was not rerun for these focused corrections and is not transferred as fresh acceptance of the new heads.

Limits: synthetic IWER does not qualify physical-headset use or full task completion. Cold renderer attach, intermittent GR1T2 first-Play recovery and SO101 cube retention remain open; the concrete reset-order fix does not establish the cause of the historical first-Play symptom. Kit's upstream dependency selection has nine previously recorded metadata inconsistencies. Full-repository, hosted-CI and arm64 coverage are not claimed; upstream changes after the frozen base are not included.

Checklist

• I have read and understood the contribution guidelines
• I have run the pre-commit checks with uv run isaaclab --format
• I have made corresponding changes to the documentation
• My changes generate no new warnings
• I have validated the changed behavior with focused coverage and applied the test-audit criteria to any test changes
• I have added a changelog fragment under source/<pkg>/changelog.d/ for every touched package (do not edit CHANGELOG.rst or bump package versions — CI handles that)
• I have added my name to the CONTRIBUTORS.md or my name already exists there

The warnings checkbox remains open: dependency warnings and the separate Kit metadata audit are unresolved.

[hougantc-nvda]

@greptile Please review the latest changes in this PR.

[greptile-apps]

Confidence Score: 3/5

[High impact] The PR adds an optional kitless teleoperation path and changes shared renderer and runtime code.

Fix replay reset ordering and Kit’s separate render-anchor updates before merging.

Findings

1.  Replay stops after Reset ▶
2.  Kit view and controls diverge ▶

Summary

Adds kitless OVXR teleoperation and MCAP replay, with SO101 CPU OvPhysX selection.

• Live teleoperation and MCAP replay can use OVXR without Kit XR.
• The renderer and stage can be shared with a worker until its writes finish.
• The SO101 absolute-IK task can run with OVPhysX and a full clutch pose.
• Packing-table tasks prepare collision settings for standalone physics backends.
• Shared runtime paths handle kitless CPU arrays, Newton capture, and headless XR updates.

Diagram

%%{init: {'theme': 'neutral'}}%%
flowchart LR
    Input["Live tracking or MCAP"] --> Device["IsaacTeleopDevice"]
    Device --> Env["Environment step"]
    Env --> Publisher["Scene publisher"]
    Publisher --> Stage["Borrowed OVStage"]
    Stage --> Worker["Capture OVXR worker"]
    Worker --> View["Headset view"]

Loading

_{Reviews (2) · Last reviewed commit: "Add live and recorded OVXR teleoperation..." · Reviewed by Greptile}

[hougantc-nvda]

@greptile Please review the current head (62a1200) again. Recheck the fixes and remaining issues against the published code, including the actual interfaces and callers at this PR boundary.

[greptile-apps]

Replay stops after Reset

With a dynamic anchor, such as G1’s pelvis anchor, reset clears the pose used by input. If request_render() returns None, _ReplayPresentation.publish() keeps the reset pending, but the next loop iteration calls _advance_replay_input() before retrying it. Reading world_T_anchor then raises and ends the replay with a failure.

Retry the forced reset publication before advancing recorded input, as the live loop already does.

[greptile-apps]

Kit view and controls diverge

With anchor_prim_path and either render override set, Kit selects the separate render anchor, but each frame still updates only _xr_anchor_headset_path. The view therefore follows its parent’s raw motion instead of the tracking anchor’s fixed-height and rotation rules. Even the first update can put the two frames at different heights.

Update both anchors from the same filtered pose and preserve the render offset, as the OVXR path does.