Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
7 changes: 4 additions & 3 deletions AGENTS.md
Original file line number Diff line number Diff line change
Expand Up @@ -117,9 +117,10 @@ the logging packages.
**The one exception is a program that imports no `isaaccapture` at all**, and
is meant to keep it that way: there are no handlers of ours for it to
duplicate, and reaching for `logging_config` would add the dependency the
program exists without. `examples/camera_viz/camera_streamer.py` is the
case — a sender-only camera box runs it with no CUDA/Vulkan/OpenXR runtime
installed, so `import isaaccapture` would fail outright. Keep the logger
program exists without. The camera_viz streamer,
`examples/camera_viz/python/isaaccapture_examples/camera_viz/camera_streamer.py`,
is the case — a sender-only camera box runs it with no CUDA/Vulkan/OpenXR
runtime installed, so `import isaaccapture` would fail outright. Keep the logger
names under `isaaccapture.` anyway; the naming rule above is a convention,
not a runtime coupling.
- **Configure through the public API instead:** `logging_config.set_console_level()`,
Expand Down
78 changes: 43 additions & 35 deletions docs/source/references/camera_streaming.rst
Original file line number Diff line number Diff line change
Expand Up @@ -72,12 +72,14 @@ There is no need to install the ``isaaccapture`` pip package yourself. ``setup``
sample's own environment: ``isaaccapture[cloudxr]`` — the ``cloudxr`` extra is not optional here,
since XR is the default display mode and the viewer launches the runtime itself — plus every
other Python dependency, into an isolated ``.venv/`` via ``uv`` (no ``--system-site-packages``).
It resolves a release matching the checkout's major/minor version, falling back to a release
candidate and then, after asking, to a source build
of the surrounding checkout (see :code-file:`scripts/_install_deps.sh
For ``isaaccapture`` it resolves a release matching the checkout's major/minor version, falling
back to a release candidate and then, after asking, to a source build of the surrounding
checkout (see :code-file:`scripts/_install_deps.sh
<examples/camera_viz/scripts/_install_deps.sh>`; ``--wheel`` and ``--build-from-source`` override
the choice). When required system packages are missing, it prints the exact ``apt-get`` line
to approve — declining, or a non-interactive run, aborts.
the choice). It also installs the sample itself there as an editable package, which is what
``python -m isaaccapture_examples.camera_viz`` imports. When required system packages are
missing, it prints the exact ``apt-get`` line to approve — declining, or a non-interactive run,
aborts.

Setup selects CuPy for the toolkit at ``CUDA_PATH``, if set. Otherwise it takes the first CUDA
12 or 13 toolkit among those of each ``nvcc`` on ``PATH`` and ``/usr/local/cuda``: first one
Expand Down Expand Up @@ -196,7 +198,8 @@ path, such as ``configs/replay.yaml``:
* - Command
- Purpose
* - ``./camera_viz.sh run CONFIG [VIEWER_FLAGS]``
- Run the viewer; flags after ``CONFIG`` are forwarded to ``camera_viz.py``.
- Run the viewer; flags after ``CONFIG`` are forwarded to
``python -m isaaccapture_examples.camera_viz``.
* - ``./camera_viz.sh loopback CONFIG``
- Run a local sender and receiver to test RTP. Requires ``setup --with-rtp``.
* - ``./camera_viz.sh deploy CONFIG [DEPLOY_FLAGS]``
Expand All @@ -212,7 +215,8 @@ path, such as ``configs/replay.yaml``:
The ``service-*`` commands manage the robot's RTP sender; CloudXR has separate
:ref:`service commands <camera-cloudxr-runtime>`.

Run ``.venv/bin/python camera_viz.py --help`` for viewer and CloudXR launcher flags.
Run ``.venv/bin/python -m isaaccapture_examples.camera_viz --help`` for viewer and CloudXR
launcher flags.

.. _camera-streaming-web-client:

Expand Down Expand Up @@ -602,8 +606,8 @@ service, activate the example venv (or any environment with ``isaaccapture``) an
python -m isaaccapture.cloudxr.service status
python -m isaaccapture.cloudxr.service stop

Run ``.venv/bin/python camera_viz.py --help`` for the rest (install dir, env-config file,
WSS proxy toggle).
Run ``.venv/bin/python -m isaaccapture_examples.camera_viz --help`` for the rest (install dir,
env-config file, WSS proxy toggle).

Runtime settings themselves are declared in the config rather than exported. The viewer ships
defaults (pose wait off, runtime foveation on), and ``display.cloudxr`` overrides them per
Expand Down Expand Up @@ -643,7 +647,7 @@ client-reconstructed streaming.
Split mode — robot → workstation over RTP
-----------------------------------------

Split mode runs the capture side on the robot (``camera_streamer.py``) and ships RTP H.264 to
Split mode runs the capture side on the robot (the ``camera_streamer`` module) and ships RTP H.264 to
the viewer on the workstation (``source: rtp``).

.. warning::
Expand Down Expand Up @@ -686,13 +690,13 @@ interactively keeps it out of shell history and the command line.
packages or JetPack CUDA wiring prompt for approval on the robot.
3. Render ``~/.config/systemd/user/camera-streamer.service``. ``--streaming-host IP`` (or
``STREAMING_HOST``) injects ``--host IP`` into its ``ExecStart`` without changing the YAML.
4. Enable lingering once with ``sudo loginctl enable-linger``, then enable and start the user
service with ``systemctl --user enable --now``.
4. Enable lingering once with ``sudo loginctl enable-linger``, then enable the user service and
restart it, so a redeploy replaces a streamer that is already running.

``--no-service`` stops after source sync and dependency installation. In that case,
``--streaming-host`` does not configure a running sender; pass ``--host`` when starting
``camera_streamer.py`` manually. The sender retries across unplug, SDK errors, and network
blips. Operate the installed unit with:
``python -m isaaccapture_examples.camera_viz.camera_streamer`` manually. The sender retries
across unplug, SDK errors, and network blips. Operate the installed unit with:

.. code-block:: bash

Expand Down Expand Up @@ -825,6 +829,8 @@ Troubleshooting

- **Python cannot find dependencies** — use the sample's ``.venv/bin/python``, or select an
environment with :ref:`setup --venv <camera-streaming-external-venv>`.
``No module named 'isaaccapture_examples'`` means the sample is not installed in that
environment; rerun ``./camera_viz.sh setup`` (with the same ``--venv``, if you used one).
- **No video on Orin.** When the CloudXR runtime runs on Jetson Orin, set
**Video Codec** to **H.264** in the CloudXR web client. See
:ref:`connect-xr-headset`.
Expand Down Expand Up @@ -875,36 +881,38 @@ Televiz as the compositor at the end of the chain:
:class: code-100col

camera_viz/
├── camera_viz.sh — CLI: setup / loopback / run / deploy / service-*
├── camera_viz.py — receiver / viewer (drives a Televiz VizSession)
├── camera_streamer.py — robot-side RTP sender (per-camera supervisor)
├── config.py — YAML parsing and validation
├── display.py — VizSession + one layer per surface
├── cloudxr_env.py — display.cloudxr → runtime env file
├── dashboard.py — terminal status panel
├── controls/ — XR controller bindings, shapes, stereo geometry, HUD
├── pipeline/ — source ABC + threaded runner
├── placements/ — XR lock-mode strategies (world / head / lazy / gimbal)
├── sources/ — V4L2 / OAK-D / ZED / video replay / synthetic / rtp_h264
├── transports/ — RTP sender + receiver (native + GStreamer)
├── codec/ — native NVENC / NVDEC pybind module
├── configs/ — one YAML per source kind
├── test_data/ — sample replay clip (Git LFS)
├── camera_viz.sh — CLI: setup / loopback / run / deploy / service-*
├── configs/ — one YAML per source kind
├── test_data/ — sample replay clip (Git LFS)
├── python/isaaccapture_examples/camera_viz/
│ ├── camera_viz.py — receiver / viewer (drives a Televiz VizSession)
│ ├── camera_streamer.py — robot-side RTP sender (per-camera supervisor)
│ ├── config.py — YAML parsing and validation
│ ├── display.py — VizSession + one layer per surface
│ ├── cloudxr_env.py — display.cloudxr → runtime env file
│ ├── cuda_env.py — applies the CUDA_PATH setup records
│ ├── dashboard.py — terminal status panel
│ ├── controls/ — XR controller bindings, shapes, stereo geometry, HUD
│ ├── pipeline/ — source ABC + threaded runner
│ ├── placements/ — XR lock-mode strategies (world / head / lazy / gimbal)
│ ├── sources/ — V4L2 / OAK-D / ZED / video replay / synthetic / rtp_h264
│ ├── transports/ — RTP sender + receiver (native + GStreamer)
│ └── codec/ — native NVENC / NVDEC pybind module
└── scripts/
├── _install_deps.sh — installer (setup + deploy)
└── camera-streamer.service.in — systemd unit template

- **Sources** (:code-dir:`sources/ <examples/camera_viz/sources>`) implement a common source ABC and
hand frames to a threaded runner in :code-dir:`pipeline/ <examples/camera_viz/pipeline>`. Each
- **Sources** (:code-dir:`sources/ <examples/camera_viz/python/isaaccapture_examples/camera_viz/sources>`) implement a common source ABC and
hand frames to a threaded runner in :code-dir:`pipeline/ <examples/camera_viz/python/isaaccapture_examples/camera_viz/pipeline>`. Each
source produces GPU frames where possible — e.g. the ZED source uses ``retrieve_image(MEM.GPU)`` so
BGRA8 stays in VRAM and a CUDA kernel channel-swaps into contiguous RGBA with no host round-trip.
- **The viewer** (:code-file:`camera_viz.py <examples/camera_viz/camera_viz.py>`) creates a
- **The viewer** (:code-file:`camera_viz.py <examples/camera_viz/python/isaaccapture_examples/camera_viz/camera_viz.py>`) creates a
``VizSession`` and adds one ``QuadLayer`` per enabled camera, then submits each frame to its layer
and calls ``render()`` once per frame. Stereo cameras submit both eyes.
- **Transport** (:code-dir:`transports/ <examples/camera_viz/transports>`) carries the split mode:
- **Transport** (:code-dir:`transports/ <examples/camera_viz/python/isaaccapture_examples/camera_viz/transports>`) carries the split mode:
an RTP H.264 sender on the robot and a receiver on the workstation, with a native NVENC/NVDEC codec
module (:code-dir:`codec/ <examples/camera_viz/codec>`) or a GStreamer fallback.
- **Placement** (:code-dir:`placements/ <examples/camera_viz/placements>`) holds the XR lock-mode
module (:code-dir:`codec/ <examples/camera_viz/python/isaaccapture_examples/camera_viz/codec>`) or a GStreamer fallback.
- **Placement** (:code-dir:`placements/ <examples/camera_viz/python/isaaccapture_examples/camera_viz/placements>`) holds the XR lock-mode
strategies. Placement is application policy — Televiz only renders a layer at whatever pose the app
sets each frame.

Expand Down
28 changes: 17 additions & 11 deletions examples/camera_viz/camera_viz.sh
Original file line number Diff line number Diff line change
Expand Up @@ -163,6 +163,10 @@ _require_local_config() {
log_error "no venv at $LOCAL_VENV — run ./camera_viz.sh setup first"
exit 1
}
"$LOCAL_VENV/bin/python" -c 'import isaaccapture_examples.camera_viz' >/dev/null 2>&1 || {
log_error "camera_viz is not installed in $LOCAL_VENV — rerun ./camera_viz.sh setup"
exit 1
}
}

# RTP deps are opt-in at setup time (``--with-rtp``), so a venv provisioned
Expand Down Expand Up @@ -198,7 +202,7 @@ PY
}

# ──────────────────────────────────────────────────────────────────────
# run (the viewer; args after CONFIG forward to camera_viz.py, e.g. --mode xr)
# run (the viewer; args after CONFIG forward to the viewer, e.g. --mode xr)
# ──────────────────────────────────────────────────────────────────────

cmd_run() {
Expand All @@ -209,7 +213,7 @@ cmd_run() {
_require_rtp_deps
fi
log_step "Starting camera_viz — Ctrl-C to exit"
"$LOCAL_VENV/bin/python" "$HERE/camera_viz.py" "$@"
"$LOCAL_VENV/bin/python" -m isaaccapture_examples.camera_viz "$@"
}

# ──────────────────────────────────────────────────────────────────────
Expand Down Expand Up @@ -240,11 +244,11 @@ cmd_loopback() {
trap '_loopback_cleanup' EXIT

log_step "Starting camera_streamer → 127.0.0.1 (background)"
"$LOCAL_VENV/bin/python" "$HERE/camera_streamer.py" "$1" --host 127.0.0.1 &
"$LOCAL_VENV/bin/python" -m isaaccapture_examples.camera_viz.camera_streamer "$1" --host 127.0.0.1 &
_LOOPBACK_SENDER_PID=$!

log_step "Starting camera_viz (foreground) — Ctrl-C to exit"
"$LOCAL_VENV/bin/python" "$HERE/camera_viz.py" "$_LOOPBACK_RECV_CONFIG"
"$LOCAL_VENV/bin/python" -m isaaccapture_examples.camera_viz "$_LOOPBACK_RECV_CONFIG"
}

# ──────────────────────────────────────────────────────────────────────
Expand All @@ -256,7 +260,7 @@ cmd_deploy() {
# anything we don't recognize would land in REMOTE_REST (where the
# positional CONFIG lives).
local no_service=false
# ``--streaming-host`` → injected as ``--host`` on camera_streamer.py's
# ``--streaming-host`` → injected as ``--host`` on the streamer's
# CLI inside the rendered systemd unit. Lets you keep the YAML at
# 127.0.0.1 for loopback and override only when deploying to a robot.
local streaming_host="${STREAMING_HOST:-}"
Expand Down Expand Up @@ -304,10 +308,10 @@ cmd_deploy() {
local manual_host_flag=""
if [[ -n "$streaming_host" ]]; then
manual_host_flag=" --host $streaming_host"
log_info "(--streaming-host has no effect in --no-service mode — pass it to camera_streamer.py yourself.)"
log_info "(--streaming-host has no effect in --no-service mode — pass it to the streamer yourself.)"
fi
log_info "Run manually with:"
log_info " ssh $REMOTE_USER@$REMOTE_HOST 'cd ~/camera_viz && .venv/bin/python camera_streamer.py $config$manual_host_flag'"
log_info " ssh $REMOTE_USER@$REMOTE_HOST 'cd ~/camera_viz && .venv/bin/python -m isaaccapture_examples.camera_viz.camera_streamer $config$manual_host_flag'"
log_info "Re-run without --no-service when you're ready to install the systemd unit."
return 0
fi
Expand Down Expand Up @@ -357,8 +361,10 @@ REMOTE
)
ssh_run_tty "$linger_cmd"

log_step "Enabling + starting service"
ssh_run "systemctl --user enable --now ${SERVICE_NAME}.service"
# restart, not `enable --now`: a redeploy must replace a streamer that is
# already running the previous unit.
log_step "Enabling + (re)starting service"
ssh_run "systemctl --user enable ${SERVICE_NAME}.service && systemctl --user restart ${SERVICE_NAME}.service"
log_ok "deployed."

log_info "Tail logs with: ./camera_viz.sh service-logs --host $REMOTE_HOST --user $REMOTE_USER"
Expand Down Expand Up @@ -441,9 +447,9 @@ REMOTE (Jetson robot)
deploy [--host H --user U [--password P]]
[--streaming-host IP] [--no-service] CONFIG
rsync source, install deps, install + start
systemd user service running camera_streamer.py.
systemd user service running the streamer.
--no-service stops after deps so you can run
camera_streamer.py by hand first.
the streamer by hand first.
--streaming-host injects ``--host IP`` into the
unit's ExecStart so the sender streams there
regardless of streaming.host in the YAML. Same
Expand Down
28 changes: 24 additions & 4 deletions examples/camera_viz/pyproject.toml
Original file line number Diff line number Diff line change
Expand Up @@ -7,8 +7,14 @@
# ``camera_viz.sh setup``; pass ``setup --wheel <path>`` to use a locally
# built wheel instead.

[build-system]
requires = ["hatchling"]
build-backend = "hatchling.build"

[project]
name = "camera-viz"
# The dist name mirrors the import path, so an installed example claims no bare
# top-level name in site-packages.
name = "isaaccapture-examples-camera-viz"
version = "0.0.0"
description = "Televiz camera-feed visualizer"
requires-python = ">=3.11,<3.14"
Expand Down Expand Up @@ -38,13 +44,27 @@ rtp = [
# setup script apt-installs them and uses --system-site-packages),
# not pip — PyGObject from PyPI requires a source-build with cairo
# + girepository dev headers. pybind11 stays in the venv as a build-
# time dep for the codec at ./codec/.
# time dep for the native codec.
"pybind11>=2.11",
]
# The viewer imports isaaccapture and Pillow; the sender must run without
# isaaccapture, so they are not base dependencies.
viewer = ["isaaccapture[cloudxr]", "pillow>=10.0"]

# Native NVENC/NVDEC codec lives in ./codec/. Build with
# ``codec/build.sh``; ``./camera_viz.sh setup`` does this automatically.
# Native NVENC/NVDEC codec lives in ./python/isaaccapture_examples/camera_viz/codec/.
# Build with its build.sh; ``./camera_viz.sh setup`` does this automatically.
# Requires CUDA toolkit + NVIDIA driver (libnvidia-encode + libnvcuvid).

# `only-include` + `sources`, not `packages`, and no __init__.py above
# camera_viz -- see the packaging rules in examples/README.md.
#
# The native codec's .so is built in place by codec/build.sh, so it is picked up
# from the source tree rather than shipped by this build. Hatchling reads only
# this directory's .gitignore, so the codec build tree is excluded here.
[tool.hatch.build.targets.wheel]
only-include = ["python/isaaccapture_examples/camera_viz"]
sources = ["python"]
exclude = ["/python/isaaccapture_examples/camera_viz/codec/build", "*.so"]

[tool.uv]
python-preference = "only-managed"
Original file line number Diff line number Diff line change
@@ -0,0 +1,2 @@
# SPDX-FileCopyrightText: Copyright (c) 2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved.
# SPDX-License-Identifier: Apache-2.0
Original file line number Diff line number Diff line change
@@ -0,0 +1,8 @@
# SPDX-FileCopyrightText: Copyright (c) 2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved.
# SPDX-License-Identifier: Apache-2.0

import sys

from .camera_viz import main

sys.exit(main())
Original file line number Diff line number Diff line change
Expand Up @@ -29,15 +29,15 @@

import yaml

import cuda_env
from pipeline import FrameSource
from sources import (
from . import cuda_env
from .pipeline import FrameSource
from .sources import (
PairedFrameSource,
build_local_camera,
resolve_video_paths,
set_verbose,
)
from transports import RtpH264Sender, make_encoder
from .transports import RtpH264Sender, make_encoder

logger = logging.getLogger("isaaccapture.camera_streamer")

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -12,7 +12,7 @@
RTP-mode deployment share one config.

Usage:
python camera_viz.py configs/v4l2.yaml
python -m isaaccapture_examples.camera_viz configs/v4l2.yaml
"""

from __future__ import annotations
Expand All @@ -30,19 +30,16 @@
from isaaccapture import logging_config
from isaaccapture.cloudxr import CloudXRLauncher

import cloudxr_env
import config
import cuda_env
import display
from controls import (
from . import cloudxr_env, config, cuda_env, display
from .controls import (
ControllerControls,
ControlTarget,
controls_config_from_yaml,
make_hud,
)
from dashboard import Dashboard
from pipeline import VizRunner
from sources import resolve_video_paths, set_notify_sink, set_verbose
from .dashboard import Dashboard
from .pipeline import VizRunner
from .sources import resolve_video_paths, set_notify_sink, set_verbose


def _parse_args(argv: Optional[list[str]]):
Expand Down Expand Up @@ -392,7 +389,3 @@ def _on_signal(signum, frame):
# actually in force inspectable while the runtime is up.
settings_stack.close()
return 0


if __name__ == "__main__":
sys.exit(main())
Loading
Loading