Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion docs/source/device/gamepad.rst
Original file line number Diff line number Diff line change
Expand Up @@ -100,7 +100,7 @@ Usage

Start the plugin with the session through ``TeleopSessionConfig.plugins``, from the copy bundled
with ``isaaccapture`` (wheel installs included), and read it with ``GamepadSource``, as
:code-file:`examples/teleop/python/gamepad_printer_example.py` does:
:code-file:`examples/teleop/python/isaaccapture_examples/teleop/gamepad_printer_example.py` does:

.. code-block:: python

Expand Down
2 changes: 1 addition & 1 deletion docs/source/device/spatial_controllers/spacemouse.rst
Original file line number Diff line number Diff line change
Expand Up @@ -66,7 +66,7 @@ Usage

Start the plugin with the session through ``TeleopSessionConfig.plugins``, from the copy bundled
with ``isaaccapture`` (wheel installs included), and read it with ``SpaceMouseSource``, as
:code-file:`examples/teleop/python/spacemouse_printer_example.py` does:
:code-file:`examples/teleop/python/isaaccapture_examples/teleop/spacemouse_printer_example.py` does:

.. code-block:: python

Expand Down
10 changes: 5 additions & 5 deletions docs/source/device/trackers/index.rst
Original file line number Diff line number Diff line change
Expand Up @@ -252,8 +252,8 @@ axis inputs. Uses standard OpenXR action bindings.
- Examples:

- :code-file:`examples/retargeting/python/isaaccapture_examples/retargeting/sources_example.py`
- :code-file:`examples/teleop/python/locomotion_retargeting_example.py`
- :code-file:`examples/teleop/python/gripper_retargeting_example_simple.py`
- :code-file:`examples/teleop/python/isaaccapture_examples/teleop/locomotion_retargeting_example.py`
- :code-file:`examples/teleop/python/isaaccapture_examples/teleop/gripper_retargeting_example_simple.py`

FullBodyTracker
~~~~~~~~~~~~~~~
Expand Down Expand Up @@ -434,7 +434,7 @@ utility internally.
- Examples:

- :code-file:`examples/schemaio/pedal_printer.cpp`
- :code-file:`examples/teleop/python/foot_pedal_locomotion_example.py`
- :code-file:`examples/teleop/python/isaaccapture_examples/teleop/foot_pedal_locomotion_example.py`

.. note::

Expand Down Expand Up @@ -505,5 +505,5 @@ For end-to-end usage patterns combining trackers with a ``DeviceIOSession``, see
For higher-level usage with the teleop session manager and retargeting, see:

- :code-file:`examples/retargeting/python/isaaccapture_examples/retargeting/sources_example.py`
- :code-file:`examples/teleop/python/gripper_retargeting_example_simple.py`
- :code-file:`examples/teleop/python/locomotion_retargeting_example.py`
- :code-file:`examples/teleop/python/isaaccapture_examples/teleop/gripper_retargeting_example_simple.py`
- :code-file:`examples/teleop/python/isaaccapture_examples/teleop/locomotion_retargeting_example.py`
10 changes: 6 additions & 4 deletions docs/source/references/cloudxr.rst
Original file line number Diff line number Diff line change
Expand Up @@ -213,13 +213,15 @@ for themselves and need no sourcing.
Run teleop examples against the service
---------------------------------------

Nothing to pass. The examples under ``examples/teleop/python/`` attach to
whatever runtime is serving, so with a service running they use it:
Nothing to pass. The examples under ``examples/teleop`` attach to
whatever runtime is serving, so with a service running they use it. Install
them once from the repository root, as in :ref:`run-teleop-example`:

.. code-block:: bash

uv pip install -e ./examples/teleop
python -m isaaccapture.cloudxr.service start --accept-eula
python examples/teleop/python/gripper_retargeting_example_simple.py
python -m isaaccapture_examples.teleop.gripper_retargeting_example_simple
Comment thread
ivany-nv marked this conversation as resolved.
Resolved

The example leaves the service running when it exits, so the headset stays
connected and the next run reattaches to the same session.
Expand All @@ -242,7 +244,7 @@ other than CloudXR:

export XR_RUNTIME_JSON=/path/to/your/openxr.json
export ISAAC_TELEOP_DISABLE_CXR_ENV_CHECKS=1
python examples/teleop/python/gripper_retargeting_example_simple.py --no-launch-cloudxr-runtime
python -m isaaccapture_examples.teleop.gripper_retargeting_example_simple --no-launch-cloudxr-runtime

Run the service in a container or CI
------------------------------------
Expand Down
2 changes: 1 addition & 1 deletion docs/source/references/retargeting/index.rst
Original file line number Diff line number Diff line change
Expand Up @@ -248,7 +248,7 @@ Available Retargeters
For a complete hand retargeting example (e.g. ``HandsSource`` + ``DexHandRetargeter`` or
``TriHandMotionControllerRetargeter``, connect and compute), see
``g1_trihand_retargeting_example.py`` and ``dex_bimanual_example.py`` in the
:code-dir:`examples/teleop/python` directory.
:code-dir:`examples/teleop/python/isaaccapture_examples/teleop` directory.

.. dropdown:: LocomotionRootCmdRetargeter

Expand Down
10 changes: 6 additions & 4 deletions docs/source/references/retargeting/joint_space.rst
Original file line number Diff line number Diff line change
Expand Up @@ -102,17 +102,19 @@ and flatten ``ee_pose`` + ``gripper_command`` into the env's task-space action l
Run the example
---------------

The repo ships ``examples/teleop/python/joint_space_device_example.py``:
The repo ships ``examples/teleop/python/isaaccapture_examples/teleop/joint_space_device_example.py``:

.. code-block:: console

$ uv pip install -e ./examples/teleop

# Consumes the so101_leader plugin over OpenXR (source cloudxr.env first):
$ python joint_space_device_example.py --launch-plugin --mode joint --frames 8
$ python joint_space_device_example.py --launch-plugin --mode ee --urdf /path/to/so101_new_calib.urdf
$ python -m isaaccapture_examples.teleop.joint_space_device_example --launch-plugin --mode joint --frames 8
$ python -m isaaccapture_examples.teleop.joint_space_device_example --launch-plugin --mode ee --urdf /path/to/so101_new_calib.urdf

``so101_new_calib.urdf`` is not shipped in this repo. It is ``Simulation/SO101/so101_new_calib.urdf``
in `TheRobotStudio/SO-ARM100 <https://github.com/TheRobotStudio/SO-ARM100>`_; pass its path to
``--urdf``. Run the commands from ``examples/teleop/python/``.
``--urdf``. Run the commands from the repository root.

Validate
--------
Expand Down
19 changes: 11 additions & 8 deletions docs/source/teleop/quick_start.rst
Original file line number Diff line number Diff line change
Expand Up @@ -116,7 +116,7 @@ env file and start the service with it:
python -m isaaccapture.cloudxr.service start --cloudxr-env-config ./custom.env

The same ``--cloudxr-env-config`` flag is available on the teleop examples under
``examples/teleop/python/``, which register CloudXR's launcher arguments through
``examples/teleop``, which register CloudXR's launcher arguments through
``CloudXRLauncher.add_launcher_arguments()`` — but it applies only when the
example is the one starting the runtime. With a service already running the
example attaches to it instead, and prints which settings it had to ignore (see
Expand Down Expand Up @@ -332,13 +332,15 @@ running the CloudXR runtime and wss proxy in containerized environment; or using
6. Run a teleop example
------------------------

Run the simplified gripper retargeting example. This demonstrates the full
Install the teleop examples into the environment from step 2, then run the
simplified gripper retargeting example. This demonstrates the full
pipeline: reading XR controller input via CloudXR, retargeting it through the
``GripperRetargeter``, and printing the resulting gripper command values:

.. code-block:: bash

python examples/teleop/python/gripper_retargeting_example_simple.py
uv pip install -e ./examples/teleop
python -m isaaccapture_examples.teleop.gripper_retargeting_example_simple

Once running, squeeze the **right controller trigger** to control the gripper.
``GripperRetargeter`` emits binary commands: ``+1.00`` means open and ``-1.00``
Expand All @@ -365,7 +367,7 @@ chosen duration, measured after session startup, pass ``--duration``:

.. code-block:: bash

python examples/teleop/python/gripper_retargeting_example_simple.py --duration 20
python -m isaaccapture_examples.teleop.gripper_retargeting_example_simple --duration 20

``--duration 0`` also runs until Ctrl+C. Both a timed run and Ctrl+C announce
that the session is closing, then print ``Session closed.`` after cleanup.
Expand All @@ -377,12 +379,13 @@ For proportional controller input, read ``ControllerInputIndex.TRIGGER_VALUE``
or use ``SO101GripperRetargeter`` for SO-101 jaw closedness in ``[0, 1]``
(``0`` open, ``1`` closed); see :doc:`../references/retargeting/so101`.

To try other examples, see ``examples/teleop/python/`` — for instance:
To try other examples, run another module of ``isaaccapture_examples.teleop``
the same way; :code-file:`examples/teleop/README.md` lists them all. For instance:

- ``se3_retargeting_example.py`` — maps hand or controller poses to
- ``se3_retargeting_example`` — maps hand or controller poses to
end-effector poses (absolute or relative), with the same ``--duration`` and Ctrl+C behavior
- ``dex_bimanual_example.py`` — bimanual dexterous hand retargeting
- ``gripper_retargeting_example.py`` — full gripper example with more
- ``dex_bimanual_example`` — bimanual dexterous hand retargeting
- ``gripper_retargeting_example`` — full gripper example with more
configuration options

Next steps
Expand Down
12 changes: 8 additions & 4 deletions docs/source/teleop/teleop_session.rst
Original file line number Diff line number Diff line change
Expand Up @@ -456,14 +456,18 @@ Examples
Complete Examples
^^^^^^^^^^^^^^^^^

#. **Simplified Gripper Example**: ``examples/teleop/python/gripper_retargeting_example_simple.py``
#. **Simplified Gripper Example**: ``examples/teleop/python/isaaccapture_examples/teleop/gripper_retargeting_example_simple.py``
-- Shows the minimal configuration approach and demonstrates auto-creation
of input sources. Run it from the project root, with the environment set up
as in the :doc:`Quick Start <quick_start>`:
of input sources. Install ``isaaccapture`` into the active environment first:
from your source build as in
:doc:`step 4 of Build from source </development/build_from_source/index>`, or
from the package index as in the :doc:`Quick Start <quick_start>`. Then install
this example from the project root:

.. code-block:: bash

python examples/teleop/python/gripper_retargeting_example_simple.py
uv pip install -e ./examples/teleop
python -m isaaccapture_examples.teleop.gripper_retargeting_example_simple

Before vs After
^^^^^^^^^^^^^^^
Expand Down
8 changes: 0 additions & 8 deletions examples/teleop/CMakeLists.txt

This file was deleted.

55 changes: 55 additions & 0 deletions examples/teleop/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,55 @@
<!--
SPDX-FileCopyrightText: Copyright (c) 2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved.
SPDX-License-Identifier: Apache-2.0
-->

# Teleop Examples

Each example reads hands, controllers or a device through `TeleopSession` and
prints the result; most retarget it first. Each connects through CloudXR and
starts the runtime itself if none is serving.

Install `isaaccapture` into the active environment first: from your source
build as in [Running an example](../README.md#running-an-example), or from the
package index as in the
[Quick Start](https://nvidia.github.io/IsaacCapture/main/teleop/quick_start.html).
Then install this example from the repository root:

```bash
uv pip install -e ./examples/teleop
python -m isaaccapture_examples.teleop.gripper_retargeting_example_simple
```

Every module runs the same way, as
`python -m isaaccapture_examples.teleop.<module>`; `--help` lists its options.

| Module | What it shows |
| --- | --- |
| `gripper_retargeting_example_simple` | Controller trigger to gripper command, with the minimal `TeleopSession` setup. |
| `gripper_retargeting_example` | A hand-built gripper and end-effector graph combined into one action. |
| `isaac_lab_gripper_example` | Thumb-index pinch to gripper command (hand tracking). |
| `se3_retargeting_example` | Hand or controller pose to an absolute or relative end-effector pose. |
| `dex_bimanual_example` | Both hands to robot hand joints, with one `DexHandRetargeter` per hand. |
| `full_bimanual_reordering_example` | Arms and hands flattened into one Isaac Lab action tensor. |
| `g1_trihand_retargeting_example` | Motion controllers to G1 three-finger hand joints. |
| `locomotion_retargeting_example` | Controller sticks to robot base commands. |
| `foot_pedal_locomotion_example` | A 3-axis foot pedal to robot base commands. |
| `joint_space_device_example` | An SO-101 leader arm to an Isaac Lab action. |
| `gamepad_printer_example` | Gamepad axes and buttons, and what the gamepad retargeters make of them. |
| `spacemouse_printer_example` | SpaceMouse axes and buttons. |
| `keyboard_printer_example` | Keys from a small GLFW window. |
| `keyboard_terminal_example` | Keys from the terminal, with no window. |

The bimanual examples default to the dex-retargeting YAML and URDF files in
[`config/dex_retargeting/`](python/isaaccapture_examples/teleop/config/dex_retargeting/),
which install with the package. Their `--enable-tuning` window needs ImGui from
the `ui` extra. Of the supported Pythons, `imgui` 2.0.0 has wheels only for
3.11: on 3.12 it builds from source and needs a C++ toolchain, and on 3.13 it
does not build at all:

```bash
uv pip install -e './examples/teleop[ui]'
```

The retargeters these examples use are documented in
[Retargeting](https://nvidia.github.io/IsaacCapture/main/references/retargeting/index.html).
32 changes: 32 additions & 0 deletions examples/teleop/pyproject.toml
Original file line number Diff line number Diff line change
@@ -0,0 +1,32 @@
# SPDX-FileCopyrightText: Copyright (c) 2025-2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved.
# SPDX-License-Identifier: Apache-2.0

[build-system]
requires = ["hatchling"]
build-backend = "hatchling.build"

[project]
# The dist name mirrors the import path, so an installed example claims no bare
# top-level name in site-packages.
name = "isaaccapture-examples-teleop"
version = "0.1.0"
description = "Isaac Teleop Retargeting Examples"
requires-python = ">=3.11,<3.14"
dependencies = [
"isaaccapture[cloudxr,retargeters]",
"glfw>=2.7", # keyboard_printer_example: the window and its platform constants
]

[project.optional-dependencies]
# The bimanual examples' --enable-tuning window. Optional because imgui does not
# install on every supported Python; see README.md.
ui = ["isaaccapture[ui]"]

# `only-include` + `sources`, not `packages`, and no __init__.py above
# teleop -- see the packaging rules in examples/README.md.
#
# config/ lives inside the package because the bimanual examples resolve it
# from __file__, so the YAML and URDF must travel with them.
[tool.hatch.build.targets.wheel]
only-include = ["python/isaaccapture_examples/teleop"]
sources = ["python"]
Original file line number Diff line number Diff line change
@@ -0,0 +1,2 @@
# SPDX-FileCopyrightText: Copyright (c) 2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved.
# SPDX-License-Identifier: Apache-2.0
Original file line number Diff line number Diff line change
@@ -1,10 +1,10 @@
# SPDX-FileCopyrightText: Copyright (c) 2025 NVIDIA CORPORATION & AFFILIATES. All rights reserved.
# SPDX-FileCopyrightText: Copyright (c) 2025-2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved.
# SPDX-License-Identifier: Apache-2.0

"""
Dex BiManual Retargeting Example

Demonstrates using DexBiManualRetargeter to control two hands simultaneously.
Drives both hands at once with one DexHandRetargeter per hand.
"""

import sys
Expand All @@ -22,7 +22,6 @@
)
from isaaccapture.retargeting_engine.interface import OutputCombiner
from isaaccapture.teleop_session_manager import TeleopSession, TeleopSessionConfig
from isaaccapture.retargeting_engine_ui import MultiRetargeterTuningUIImGui


def main():
Expand Down Expand Up @@ -50,10 +49,21 @@ def main():
"--right-params", type=str, help="Path to right hand tunable parameters (JSON)"
)
parser.add_argument(
"--enable-tuning", action="store_true", help="Enable retargeting tuning UI"
"--enable-tuning",
action="store_true",
help="Enable retargeting tuning UI (needs the ui extra)",
)
CloudXRLauncher.add_launcher_arguments(parser)
args = parser.parse_args()
# Before the retargeters load and CloudXR starts: imgui is the optional `ui` extra.
if args.enable_tuning:
try:
from isaaccapture.retargeting_engine_ui import MultiRetargeterTuningUIImGui
except ImportError:
parser.error(
"--enable-tuning needs the ui extra: "
"uv pip install -e './examples/teleop[ui]'"
)

# Check for config files
# Note: These paths are placeholders. In a real scenario, you'd point to your robot's config.
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -30,7 +30,9 @@
)


PLUGIN_ROOT_DIR = Path(__file__).resolve().parent.parent.parent.parent / "plugins"
# Six levels up is the repo root in a checkout or editable install. It has no
# plugins/ directory, so this example runs without the synthetic-hands plugin.
PLUGIN_ROOT_DIR = Path(__file__).resolve().parents[5] / "plugins"
PLUGIN_NAME = "controller_synthetic_hands"
PLUGIN_ROOT_ID = "synthetic_hands"

Expand Down
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
# SPDX-FileCopyrightText: Copyright (c) 2025 NVIDIA CORPORATION & AFFILIATES. All rights reserved.
# SPDX-FileCopyrightText: Copyright (c) 2025-2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved.
# SPDX-License-Identifier: Apache-2.0

"""
Expand Down Expand Up @@ -29,7 +29,6 @@
)
from isaaccapture.retargeting_engine.interface import OutputCombiner
from isaaccapture.teleop_session_manager import TeleopSession, TeleopSessionConfig
from isaaccapture.retargeting_engine_ui import MultiRetargeterTuningUIImGui


def main():
Expand All @@ -50,10 +49,21 @@ def main():
"--right-config", type=str, help="Path to right hand retargeting config (YAML)"
)
parser.add_argument(
"--enable-tuning", action="store_true", help="Enable retargeting tuning UI"
"--enable-tuning",
action="store_true",
help="Enable retargeting tuning UI (needs the ui extra)",
)
CloudXRLauncher.add_launcher_arguments(parser)
args = parser.parse_args()
# Before the retargeters load and CloudXR starts: imgui is the optional `ui` extra.
if args.enable_tuning:
try:
from isaaccapture.retargeting_engine_ui import MultiRetargeterTuningUIImGui
except ImportError:
parser.error(
"--enable-tuning needs the ui extra: "
"uv pip install -e './examples/teleop[ui]'"
)

# Config paths (similar to dex_bimanual_example)
config_dir = Path(__file__).parent / "config" / "dex_retargeting"
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -26,7 +26,9 @@
from isaaccapture.retargeting_engine.interface import OutputCombiner


PLUGIN_ROOT_DIR = Path(__file__).resolve().parent.parent.parent.parent / "plugins"
# Six levels up is the repo root in a checkout or editable install. It has no
# plugins/ directory, so this example runs without the synthetic-hands plugin.
PLUGIN_ROOT_DIR = Path(__file__).resolve().parents[5] / "plugins"
PLUGIN_NAME = "controller_synthetic_hands"
PLUGIN_ROOT_ID = "synthetic_hands"

Expand Down
Loading
Loading