Skip to content

Use underscore flags in EgoHand docker wrappers #1400

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
shaosu-nvidia merged 1 commit into from
Oct 29, 2026
Merged

Use underscore flags in EgoHand docker wrappers #1400

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 2 additions & 2 deletions src/python/isaaccapture_egopost/inference/docker/Dockerfile.egohand
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -58,8 +58,8 @@ RUN python -c "from soma import get_assets_dir; get_assets_dir()" \
COPY egohand /workspace/egohand
COPY inference /workspace/inference

# Optional bundled weights. Populated by egohand_inference.sh build --egohand-ckpt /...
# and --hand-detector-weight /... If the files are absent the directory is still
# Optional bundled weights. Populated by egohand_inference.sh build --egohand_ckpt /...
# and --hand_detector_weight /... If the files are absent the directory is still
# created (empty), so the COPY is always valid. At run time the volume-mount
# paths take precedence; bundled paths are the fallback.
COPY weights/ /workspace/weights/
Expand Down
68 changes: 34 additions & 34 deletions src/python/isaaccapture_egopost/inference/docker/README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -38,7 +38,7 @@ needs GitHub access, inference does not.

The image entrypoint is `inference/scripts/run_egohand_inference_pipeline.py`:
RT-DETR hand-bbox detection (on the fly, or read from precomputed JSONs via
`--bbox-dir`) followed by `inference/pipeline/pipeline.py:
`--bbox_dir`) followed by `inference/pipeline/pipeline.py:
EgoHandInferencePipeline`, run directly on decoded video frames.

### Build
Expand All @@ -61,18 +61,18 @@ Weights can be baked into the image so `run` needs fewer flags:

```bash
inference/docker/egohand_inference.sh build <image> \
--egohand-ckpt /path/to/egohand.ckpt \
--hand-detector-weight /path/to/rtdetr.onnx
--egohand_ckpt /path/to/egohand.ckpt \
--hand_detector_weight /path/to/rtdetr.onnx
```

| Flag | Effect |
| ---- | ------ |
| `--egohand-ckpt <path>` | Bundle EgoHand checkpoint (see "HuggingFace fallback" below for what happens when omitted) |
| `--hand-detector <rtdetr\|saved_bboxes>` | Which detector the weight is for (default: `rtdetr`, the only option that takes a weight). `saved_bboxes` needs no weight -- bbox detection is precomputed and supplied at run time via `--bbox-dir`; `--hand-detector-weight` is rejected with this value |
| `--hand-detector-weight <path>` | Bundle detector weight (not needed if every run will use `--bbox-dir`) |
| `--egohand_ckpt <path>` | Bundle EgoHand checkpoint (see "HuggingFace fallback" below for what happens when omitted) |
| `--hand_detector <rtdetr\|saved_bboxes>` | Which detector the weight is for (default: `rtdetr`, the only option that takes a weight). `saved_bboxes` needs no weight -- bbox detection is precomputed and supplied at run time via `--bbox_dir`; `--hand_detector_weight` is rejected with this value |
| `--hand_detector_weight <path>` | Bundle detector weight (not needed if every run will use `--bbox_dir`) |

Camera intrinsics are never bundled at build time -- always pass
`--intrinsics-json` at `run` time (for both the on-the-fly and `--bbox-dir`
`--intrinsics_json` at `run` time (for both the on-the-fly and `--bbox_dir`
paths). This model needs the real calibration to produce correct depth/mesh
scale.

Expand All @@ -82,16 +82,16 @@ scale.
inference/docker/egohand_inference.sh run \
--image <image> \
--ckpt /path/to/egohand.ckpt \
--hand-detector-weight /path/to/rtdetr.onnx \
--hand_detector_weight /path/to/rtdetr.onnx \
--input /path/to/clip.mp4 \
--intrinsics-json /path/to/intrinsics.json \
--intrinsics_json /path/to/intrinsics.json \
--out outputs/inference
```

`--image`, `--input`, and `--intrinsics-json` are required by this wrapper
`--image`, `--input`, and `--intrinsics_json` are required by this wrapper
(fails immediately, before touching Docker, if any is missing).
`--intrinsics-json` is never bundleable at build time (see above) -- always
pass it here. `--ckpt` and `--hand-detector-weight` are optional; see
`--intrinsics_json` is never bundleable at build time (see above) -- always
pass it here. `--ckpt` and `--hand_detector_weight` are optional; see
"HuggingFace fallback" below.

`--input` accepts a single `.mp4` file or a folder of `.mp4` files; per-video
Expand All @@ -105,52 +105,52 @@ run `docker run` yourself with `python inference/scripts/run_egohand_inference_p
and the mounts in "Mount layout inside the container"; see
`inference/scripts/README.md` for the full argument list.

#### Two steps: detect, then reconstruct from saved boxes (`--bbox-dir`)
#### Two steps: detect, then reconstruct from saved boxes (`--bbox_dir`)

```bash
# 1. Detect hand bboxes only
inference/docker/detector.sh run \
--image <detector-image> \
--input /path/to/clip.mp4 \
--intrinsics-json /path/to/intrinsics.json \
--intrinsics_json /path/to/intrinsics.json \
--out /path/to/bboxes

# 2. Reconstruct from the saved boxes
inference/docker/egohand_inference.sh run \
--image <image> \
--ckpt /path/to/egohand.ckpt \
--input /path/to/clip.mp4 \
--bbox-dir /path/to/bboxes \
--intrinsics-json /path/to/intrinsics.json \
--perspective-cropping \
--out outputs/inference
--image <image> \
--ckpt /path/to/egohand.ckpt \
--input /path/to/clip.mp4 \
--bbox_dir /path/to/bboxes \
--intrinsics_json /path/to/intrinsics.json \
--perspective_cropping \
--out outputs/inference
```

`--bbox-dir` skips on-the-fly detection and reads the per-hand bbox JSONs that
`--bbox_dir` skips on-the-fly detection and reads the per-hand bbox JSONs that
`detector.sh run` writes. It is mutually exclusive with
`--hand-detector-weight`; no detector weight is loaded. See
`--hand_detector_weight`; no detector weight is loaded. See
`inference/hand_detector_2d/scripts/README.md` for the JSON layout.

These wrappers run the same pipeline as the host commands, so check the
[main README](../../README.md#reconstruct-with-your-own-2d-bboxes) for the
running rules, such as when `--perspective-cropping` is required for saved
running rules, such as when `--perspective_cropping` is required for saved
boxes and how camera calibration is used
([camera formats](../../README.md#camera-input-formats)).

#### HuggingFace fallback

Checkpoints and detector weights (`--ckpt`, `--hand-detector-weight`,
Checkpoints and detector weights (`--ckpt`, `--hand_detector_weight`,
`--weight`) resolve in this order: mounted at run time, bundled at build time,
downloaded from Hugging Face. Passing none is not an error; the run only fails
if the download fails. Optionally, pass `--hf-token "$HF_TOKEN"` (either
if the download fails. Optionally, pass `--hf_token "$HF_TOKEN"` (either
wrapper) to forward a token as `HF_TOKEN` in the container, e.g. if you hit
anonymous rate limits.

## `detector.sh` -- bbox detection only

The image entrypoint is `inference/hand_detector_2d/scripts/run_hand_detector.py`:
RT-DETRv2 hand-bbox detection only -- no EgoHand model, no SOMA, no torch
even loaded. Pairs with `egohand_inference.sh run --bbox-dir` to feed the JSONs
even loaded. Pairs with `egohand_inference.sh run --bbox_dir` to feed the JSONs
this produces into a full EgoHand inference run without re-detecting, so
detection can be run, reviewed, and cached as its own preprocessing stage.

Expand All @@ -170,27 +170,27 @@ inference/docker/detector.sh run \
--image <image> \
--input /path/to/clip.mp4 \
--out /path/to/bboxes \
--intrinsics-json /path/to/intrinsics.json
--intrinsics_json /path/to/intrinsics.json
```

`--image`, `--input`, `--out`, `--intrinsics-json` are required. `--weight`
`--image`, `--input`, `--out`, `--intrinsics_json` are required. `--weight`
is optional (see "HuggingFace fallback" above). Writes
`<out>/<video_stem>/<video_stem>_{left,right}_bbox.json` -- pass that same
`<out>` path as `--bbox-dir` to a `egohand_inference.sh run`.
`<out>` path as `--bbox_dir` to a `egohand_inference.sh run`.

`--tracking` and `--no-tracking` override the shared tracking setting only when
passed explicitly; `--no-tracking` also disables bbox smoothing. Use the same
input video and `--intrinsics-json` for both stages; see
input video and intrinsics JSON for both stages; see
`inference/hand_detector_2d/scripts/README.md` for camera preprocessing details.

## Mount layout inside the container

| Flag | Container path | Mode |
| ---- | -------------- | ---- |
| `--ckpt` | `/workspace/inputs/<filename>` | ro |
| `--weight` (`detector.sh`) / `--hand-detector-weight` (`egohand_inference.sh`) | `/workspace/inputs/<filename>` | ro |
| `--intrinsics-json` (either wrapper) | `/workspace/inputs/<filename>` | ro |
| `--bbox-dir` (`egohand_inference.sh`) | `/workspace/inputs/bbox` | ro |
| `--weight` (`detector.sh`) / `--hand_detector_weight` (`egohand_inference.sh`) | `/workspace/inputs/<filename>` | ro |
| `--intrinsics_json` (either wrapper) | `/workspace/inputs/<filename>` | ro |
| `--bbox_dir` (`egohand_inference.sh`) | `/workspace/inputs/bbox` | ro |
| `--input` (file) | `/workspace/inputs/video/<filename>` | ro |
| `--input` (folder) | `/workspace/inputs/video` | ro |
| `--out` | `/workspace/outputs/bbox` (`detector.sh`) or `/workspace/outputs/inference` (`egohand_inference.sh`) | rw |
28 changes: 14 additions & 14 deletions src/python/isaaccapture_egopost/inference/docker/detector.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@

# Local helper for building and running the standalone RT-DETRv2 bbox-detector
# Docker image (inference/docker/Dockerfile.detector). Detection only -- no EgoHand
# model, no SOMA. Pair with egohand_inference.sh's --bbox-dir to feed the JSONs
# model, no SOMA. Pair with egohand_inference.sh's --bbox_dir to feed the JSONs
# this produces into a full EgoHand inference run without re-detecting.
# Run from the EgoHand repo root, or any subdirectory -- the script locates the root itself.
# Run with no arguments for usage.
Expand Down Expand Up @@ -34,13 +34,13 @@ usage: inference/docker/detector.sh <build|run> [args]
pass --weight at run time, or omit both to resolve
via HF (nvidia/RT-DETR-Hand-Detector-v1) instead.

run --image <name> --input <path> --out <path> --intrinsics-json <path> [options]
run --image <name> --input <path> --out <path> --intrinsics_json <path> [options]
Run inference/hand_detector_2d/scripts/run_hand_detector.py through the
image entrypoint: bbox detection only, written as
<out>/<video_stem>/<video_stem>_{left,right}_bbox.json -- the layout
egohand_inference.sh run --bbox-dir <out> expects.
egohand_inference.sh run --bbox_dir <out> expects.

Use the same input video and --intrinsics-json for the paired
Use the same input video and intrinsics JSON for the paired
egohand_inference.sh run. This wrapper uses the detector's default
pixel-aspect rectification and does not expose a switch to disable it.
For standard pinhole cropping, match both passes' pixel-aspect settings
Expand All @@ -53,10 +53,10 @@ usage: inference/docker/detector.sh <build|run> [args]
--image <name> Docker image name/tag to run. Required.
--input <path> A video file, or a folder of .mp4 files; mounted read-only.
--out <path> Output dir for bbox JSONs. Required.
--intrinsics-json <path>
--intrinsics_json <path>
Camera intrinsics JSON, mounted read-only. Required --
must match the paired egohand_inference.sh run's
--intrinsics-json (see above).
--intrinsics_json (see above).
--weight <path> RT-DETRv2 ONNX weight, mounted read-only. Not required
if the image was built with --weight -- and if neither
is given, resolved via HF (nvidia/RT-DETR-Hand-Detector-v1)
Expand All @@ -66,9 +66,9 @@ usage: inference/docker/detector.sh <build|run> [args]
Override tracking from the shared rtdetr_docker.yaml
(currently on). --no-tracking also disables bbox smoothing.
--device <dev> Torch-style device, e.g. cuda:1 or cpu. Default: cuda.
--max-seconds <N> Only process the first N seconds.
--frame-batch-size <N> Frames decoded into RAM at once. Default: 2000.
--hf-token <token> Optional. Forwarded as HF_TOKEN in the container
--max_seconds <N> Only process the first N seconds.
--frame_batch_size <N> Frames decoded into RAM at once. Default: 2000.
--hf_token <token> Optional. Forwarded as HF_TOKEN in the container
for authenticated HF downloads (e.g. higher rate
limits). Not needed for normal runs.
EOF
Expand Down Expand Up @@ -133,15 +133,15 @@ case "$cmd" in
--image) IMAGE="$2"; shift 2 ;;
--input) INPUT="$2"; shift 2 ;;
--out) OUT="$2"; shift 2 ;;
--intrinsics-json) INTRINSICS_JSON="$2"; shift 2 ;;
--intrinsics_json) INTRINSICS_JSON="$2"; shift 2 ;;
--weight) WEIGHT="$2"; shift 2 ;;
--hand) HAND="$2"; shift 2 ;;
--tracking) TRACKING="true"; shift 1 ;;
--no-tracking) TRACKING="false"; shift 1 ;;
--device) DEVICE="$2"; shift 2 ;;
--max-seconds) MAX_SECONDS="$2"; shift 2 ;;
--frame-batch-size) FRAME_BATCH_SIZE="$2"; shift 2 ;;
--hf-token) HF_TOKEN_ARG="$2"; shift 2 ;;
--max_seconds) MAX_SECONDS="$2"; shift 2 ;;
--frame_batch_size) FRAME_BATCH_SIZE="$2"; shift 2 ;;
--hf_token) HF_TOKEN_ARG="$2"; shift 2 ;;
*) echo "unknown arg: $1" >&2; usage ;;
esac
done
Expand All @@ -151,7 +151,7 @@ case "$cmd" in
[[ -e "$INPUT" ]] || die "input not found: $INPUT"
[[ -n "$OUT" ]] || die "--out is required"
[[ -n "$INTRINSICS_JSON" ]] \
|| die "--intrinsics-json is required -- must match the paired egohand_inference.sh run's --intrinsics-json, or the boxes land on the wrong pixels (see usage)"
|| die "--intrinsics_json is required -- must match the paired egohand_inference.sh run's --intrinsics_json, or the boxes land on the wrong pixels (see usage)"
[[ -f "$INTRINSICS_JSON" ]] || die "intrinsics JSON not found: $INTRINSICS_JSON"
[[ "$HAND" == "left" || "$HAND" == "right" || "$HAND" == "both" ]] \
|| die "--hand must be left, right, or both (got: $HAND)"
Expand Down
Loading
Loading