Enable OVXR teleoperation with an application-owned renderer and scene

README.md

@@ -69,6 +69,7 @@ Our [documentation page](https://nvidia.github.io/IsaacCapture) provides everyth
 - [Architecture](https://nvidia.github.io/IsaacCapture/main/overview/architecture.html)
 - [Quick installation steps](https://nvidia.github.io/IsaacCapture/main/getting_started/quick_start.html)
 - [How to build from source](https://nvidia.github.io/IsaacCapture/main/getting_started/build_from_source.html)
+- [OVXR installation](docs/source/getting_started/ovxr.rst)
 
 ### Check a Device Integration
 
@@ -86,3 +87,17 @@ To get started, please refer to Isaac Lab's [Installation](https://isaac-sim.git
 The default license is [Apache-2.0](LICENSE.md).
 EgoPost is licensed under [OpenMDW-1.1](LICENSES/LicenseRef-OpenMDW-1.1.txt).
 Third-party materials may carry their own licenses.
+
+### Separately retrieved optional materials
+
+The optional `isaaccapture[ovxr]` extra retrieves OVXR and its OVRTX/OVStage
+dependencies.
+These packages are downloaded separately, rather than bundled in the Isaac Capture
+wheel. Their licenses apply independently of Isaac Capture's Apache-2.0 license.
+OVXR, OVRTX and OVStage are NVIDIA proprietary software.
+See [package sources and license details](docs/source/getting_started/ovxr.rst#optional-packages-and-licenses).
+Review the applicable package licenses and terms before installing or using these
+optional dependencies.
+
+Existing CloudXR native libraries are bundled in Isaac Capture wheels and remain subject
+to their separate NVIDIA license and EULA acceptance.

docs/source/getting_started/ovxr.rst

@@ -0,0 +1,127 @@
+.. SPDX-FileCopyrightText: Copyright (c) 2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved.
+.. SPDX-License-Identifier: Apache-2.0
+
+OVXR installation
+=================
+
+OVXR provides optional XR presentation for applications that use Isaac Capture
+for teleoperation input. Select the ``ovxr`` extra when installing Isaac Capture
+to include OVXR and its rendering dependencies.
+
+Install XR support
+------------------
+
+Follow :doc:`/development/build_from_source/index` to build Isaac Capture and
+select a Python environment matching the generated wheel. Include ``ovxr`` in
+the installation requirement:
+
+.. code-block:: bash
+
+   uv pip install "isaaccapture[ovxr]==X.Y+local" --find-links=./install/wheels/
+   uv pip check
+
+Replace ``X.Y`` with the version of your source build, as described in the build
+guide. Base installations and imports do not require OVXR.
+
+OVXR declares its compatible OVRTX and OVStage versions in its package metadata.
+The consuming application's dependency constraints must accept those versions.
+See the package pages below for supported Python versions and platforms.
+
+Optional packages and licenses
+------------------------------
+
+The ``ovxr`` extra installs OVXR and its OVRTX/OVStage dependencies separately
+from the Isaac Capture wheel. These packages retain their own licenses:
+
+* `OVXR <https://pypi.org/project/ovxr/>`__,
+  `OVRTX <https://pypi.org/project/ovrtx/>`__ and
+  `OVStage <https://pypi.org/project/ovstage/>`__ are NVIDIA proprietary software.
+  Review the license included with each distribution before use.
+
+CloudXR native libraries are bundled in Isaac Capture wheels and remain subject
+to their separate NVIDIA license and CloudXR EULA acceptance.
+
+Use from another application
+----------------------------
+
+``isaaccapture.ovxr_session_manager.ovxr_session`` provides ``OvxrSession``
+and ``OvxrSessionConfig``. Implement ``OvxrApplicationInterface`` to supply the
+application's renderer, Stage and ordinal ledger in an ``OvxrSessionBinding``.
+``camera_paths`` is required and must contain both stereo cameras. Set
+``up_axis="Y"`` for a Y-up Stage; the default is ``"Z"``.
+
+For a uv-managed application, declare Isaac Capture with its ``ovxr`` extra and
+select the source-built wheel in the application's project configuration:
+
+.. code-block:: toml
+
+   [project.optional-dependencies]
+   ovxr = ["isaaccapture[ovxr]"]
+
+   [tool.uv.sources]
+   isaaccapture = { path = "path/to/your/local-wheel.whl", extra = "ovxr" }
+
+Replace the wheel path with the file produced by your build. Refresh the
+application's lock file after changing the wheel or dependency selection, then
+run with ``uv run --extra ovxr``. Installing packages with ``uv pip install``
+alone does not update the application's lock file.
+
+Application-specific OVXR versions
+----------------------------------
+
+Isaac Capture pins the OVXR version used for compatibility testing. uv respects
+that pin and reports a conflict if the application requires incompatible OVRTX
+or OVStage versions.
+
+To validate another OVXR build with the same Isaac Capture release, select an
+OVXR release whose declared dependencies match the application's OVRTX/OVStage
+requirements. Add an override in the application's root ``pyproject.toml``:
+
+.. code-block:: toml
+
+   [tool.uv]
+   override-dependencies = ["ovxr==<compatible-version>"]
+
+Replace ``<compatible-version>`` with the selected release. If the application
+already has an ``override-dependencies`` list, append this entry to that list.
+Then refresh the lock file and synchronize the environment using the ``ovxr``
+extra from the example above:
+
+.. code-block:: bash
+
+   uv lock --upgrade-package ovxr
+   uv sync --extra ovxr
+
+An `override <https://docs.astral.sh/uv/concepts/resolution/#dependency-overrides>`__
+changes dependency resolution; it does not establish compatibility with Isaac
+Capture's OVXR integration. Validate the application's XR workflow before
+relying on the selected build.
+
+Applications using different provider versions need separate project
+environments and lock files. Each can install the same Isaac Capture wheel
+with its own OVXR/OVRTX/OVStage combination.
+
+
+Session lifetime
+----------------
+
+Use ``OvxrSession`` as a context manager, or call ``close()`` in a ``finally``
+block. Stop application steps and input borrowers before closing. Startup failure
+before handles are borrowed retires its resources on the owner thread; successful
+startup requires explicit closure. Do not rely on interpreter exit or object
+finalizers to stop native rendering.
+
+The session schedules work; its frame pump owns native frame processing and
+retirement, and its runtime owner stops only the CloudXR process it launched.
+Applications may supply ``runtime_launcher_factory`` for a compatible launcher;
+it receives the environment path and ``accept_eula`` and returns an object with
+``stop()``. Externally managed runtimes remain externally owned.
+
+``OvxrScenePublication`` and ``OvxrResourceOwner`` describe the two parts of the
+application contract. ``OvxrApplicationInterface`` combines them, so existing
+adapters can continue implementing the combined interface.
+
+If native context destruction fails without proving the context was consumed,
+closure reports the failure and retains its borrowed resources and runtime.
+Do not treat this as successful retirement or start a replacement session over
+those resources. There is no automatic retry of uncertain native destruction.

docs/source/index.rst

@@ -54,6 +54,7 @@ Table of Contents
    :caption: Isaac Teleop
 
    teleop/quick_start
+   getting_started/ovxr
    teleop/teleop_session
    teleop/control_state_machine
 

docs/source/references/build.rst

@@ -34,13 +34,17 @@ The layout below reflects the actual ``CMakeLists.txt`` hierarchy (root ``CMakeL
    │   ├── teleop_session_manager/ # Teleop session manager (Python)
    │   ├── cloudxr/                # CloudXR runtime helper (Python)
    │   └── python/                 # Python wheel packaging (when BUILD_PYTHON_BINDINGS=ON)
+   ├── src/python/isaaccapture/
+   │   └── ovxr_session_manager/   # OVXR session ownership and presentation
    ├── src/plugins/                # Built when BUILD_PLUGINS=ON
    │   ├── plugin_utils/
    │   ├── controller_synthetic_hands/
    │   ├── generic_3axis_pedal/
    │   ├── manus/
    │   ├── oak/                    # When BUILD_PLUGIN_OAK_CAMERA=ON
    |   └── haptikos/
+   ├── tests/python/core/
+   │   └── ovxr_session_manager/   # OVXR session and presentation tests
    └── examples/
        ├── oxr/                    # OpenXR examples (C++ and Python)
        ├── retargeting/

docs/source/teleop/teleop_session.rst

@@ -24,6 +24,15 @@ manage them directly:
 #. Runs retargeting pipeline with automatic updates
 #. Handles cleanup via RAII
 
+If resource cleanup fails during entry rollback or exit, ``cleanup_failed``
+becomes true. Hosts lending OpenXR handles must retain their renderer/session
+while this is true. If stopping the retarget worker was interrupted or failed,
+calling ``__exit__`` again retries that stop before releasing any resources.
+Successful retry clears the failure flag. Failures while unwinding the resource
+stack remain terminal: later entry, stepping and exit raise the stored error.
+An entry failure with successful rollback still permits reuse; the original
+setup error remains primary in either case.
+
 Quick Start
 -----------
 
@@ -172,8 +181,13 @@ Per application frame in pipelined mode, ``step()`` submits the current request
 and returns the latest completed ``RetargeterIO``. On a normal hit this is often
 the previous application frame's output. If that returned output came from
 request N-1, then ``session.last_context`` also comes from N-1: tracker data,
-control messages, execution events, outputs, and context stay self-consistent
-for the completed frame that is returned. Use ``mode="sync"`` when the
+control messages, execution state, outputs, and context stay self-consistent
+for the completed frame that is returned. Reset is an exception: a reset edge
+from replaced pending work or an earlier completed frame remains latched until
+a successful ``step()`` return delivers it through ``last_context``. Repeated
+undelivered reset pulses collapse to one idempotent reset. Reading the latest
+worker result does not consume that edge, and a failed output snapshot does
+not acknowledge it. This is not command-history replay. Use ``mode="sync"`` when the
 application needs exact current-frame sample/compute/return behavior.
 
 The return type is unchanged; inspect ``session.last_step_info`` for age and

pyproject.toml

@@ -50,6 +50,9 @@ dev = [
 ui = [
     "imgui[glfw]>=2.0.0",
 ]
+ovxr = [
+    "ovxr==0.1.0.5842",
+]
 retargeters = [
     "dex-retargeting>=0.4.6,<0.6.0",
     "scipy>=1.15.0",

src/compat/pyproject.toml

@@ -22,6 +22,7 @@ Homepage = "https://github.com/NVIDIA/IsaacCapture"
 [project.optional-dependencies]
 dev = ["isaaccapture[dev]>=1.6.dev0,<1.9"]
 ui = ["isaaccapture[ui]>=1.6.dev0,<1.9"]
+ovxr = ["isaaccapture[ovxr]>=1.6.dev0,<1.9"]
 retargeters = ["isaaccapture[retargeters]>=1.6.dev0,<1.9"]
 retargeters-lite = ["isaaccapture[retargeters-lite]>=1.6.dev0,<1.9"]
 grounding = ["isaaccapture[grounding]>=1.6.dev0,<1.9"]

src/core/python/CMakeLists.txt

@@ -77,6 +77,7 @@ add_custom_target(python_package ALL
     COMMAND ${CMAKE_COMMAND} -E copy "${CMAKE_CURRENT_SOURCE_DIR}/requirements.txt" "${CMAKE_BINARY_DIR}/python_package/$<CONFIG>/"
     COMMAND ${CMAKE_COMMAND} -E copy "${CMAKE_CURRENT_SOURCE_DIR}/requirements-dev.txt" "${CMAKE_BINARY_DIR}/python_package/$<CONFIG>/"
     COMMAND ${CMAKE_COMMAND} -E copy "${CMAKE_CURRENT_SOURCE_DIR}/requirements-ui.txt" "${CMAKE_BINARY_DIR}/python_package/$<CONFIG>/"
+    COMMAND ${CMAKE_COMMAND} -E copy "${CMAKE_CURRENT_SOURCE_DIR}/requirements-ovxr.txt" "${CMAKE_BINARY_DIR}/python_package/$<CONFIG>/"
     COMMAND ${CMAKE_COMMAND} -E copy "${CMAKE_CURRENT_SOURCE_DIR}/requirements-cloudxr.txt" "${CMAKE_BINARY_DIR}/python_package/$<CONFIG>/"
     COMMAND ${CMAKE_COMMAND} -E copy "${CMAKE_CURRENT_SOURCE_DIR}/requirements-retargeters.txt" "${CMAKE_BINARY_DIR}/python_package/$<CONFIG>/"
     COMMAND ${CMAKE_COMMAND} -E copy "${CMAKE_CURRENT_SOURCE_DIR}/requirements-retargeters-lite.txt" "${CMAKE_BINARY_DIR}/python_package/$<CONFIG>/"

src/core/python/pyproject.toml.in

@@ -31,6 +31,7 @@ dynamic = ["dependencies", "optional-dependencies"]
 dependencies = {file = ["requirements.txt"]}
 optional-dependencies.dev = {file = ["requirements-dev.txt"]}
 optional-dependencies.ui = {file = ["requirements-ui.txt"]}
+optional-dependencies.ovxr = {file = ["requirements-ovxr.txt"]}
 optional-dependencies.retargeters = {file = ["requirements-retargeters.txt"]}
 optional-dependencies.retargeters-lite = {file = ["requirements-retargeters-lite.txt"]}
 optional-dependencies.grounding = {file = ["requirements-grounding.txt"]}

src/core/python/requirements-ovxr.txt

@@ -0,0 +1,5 @@
+# SPDX-FileCopyrightText: Copyright (c) 2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved.
+# SPDX-License-Identifier: Apache-2.0
+
+# Application overrides: docs/source/getting_started/ovxr.rst.
+ovxr==0.1.0.5842

src/python/isaaccapture/__init__.py

@@ -60,6 +60,7 @@ def _reserve_closed_standard_fds() -> list[int]:
         deviceio_trackers,
         logging_config,
         oxr,
+        ovxr_session_manager,
         plugin_manager,
         schema,
         teleop_session_manager,
@@ -77,6 +78,7 @@ def _reserve_closed_standard_fds() -> list[int]:
     "deviceio_session",
     "deviceio",
     "oxr",
+    "ovxr_session_manager",
     "plugin_manager",
     "schema",
     "teleop_session_manager",

src/python/isaaccapture/ovxr_session_manager/__init__.py

@@ -0,0 +1,24 @@
+# SPDX-FileCopyrightText: Copyright (c) 2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved.
+# SPDX-License-Identifier: Apache-2.0
+
+"""OVXR presentation over application-owned rendering resources."""
+
+from .ovxr_session import (
+    OvxrSessionBinding,
+    OvxrSessionConfig,
+    OvxrApplicationInterface,
+    OvxrSession,
+    StageOrdinalLedger,
+)
+
+__all__ = [
+    "OvxrSessionBinding",
+    "OvxrSessionConfig",
+    "OvxrApplicationInterface",
+    "OvxrSession",
+    "StageOrdinalLedger",
+]
+
+from .interfaces import OvxrScenePublication, OvxrResourceOwner
+
+__all__ += ["OvxrScenePublication", "OvxrResourceOwner"]