Skip to content

Simplify camera processor plumbing and count None read_output as published - #6

Open
mataylor-nvidia wants to merge 2 commits into
temiroff:atemirov/camera-tensor-processorsfrom
mataylor-nvidia:review/pr-8871-cleanup
Open

mataylor-nvidia wants to merge 2 commits into
temiroff:atemirov/camera-tensor-processorsfrom
mataylor-nvidia:review/pr-8871-cleanup

Conversation

@mataylor-nvidia

Copy link
Copy Markdown

Review follow-ups for isaac-sim#8871. Two commits; merge whichever you agree with.

Simplify camera processor plumbing and correct env_mask docs

  • render_into_cameras keeps (index, render_data, camera_data) in each renderer group instead of re-indexing requests (requests[index][1], read_output(*requests[index][1:])). This is closer to the code on main.
  • _apply_tensor_processors builds the buffers dict once with the capture mask and removes the mask before publishing, instead of merging {**buffers, **mask} twice per processor.
  • The reserved env_mask output check moves from _render_data_types to CameraCfg.validate_config, so the helper only computes data types. SensorBase.__init__ validates the cfg first, so the error still fires before the camera builds anything.
  • Tests:
    • The test that called the private _render_data_types with a SimpleNamespace is folded into test_camera_cfg_rejects_outputs_unsupported_by_renderer, through validate(CameraCfg(...)). That test now also checks that processor inputs are validated against the renderer.
    • The mask assertion is now masks == [[True, True]] * 4 instead of only checking lengths.
  • Docs fix: renderers draw every environment on each capture; none of them receives the mask. Rows that are not due are redrawn from their previous camera pose, so they do not "still hold the previous capture". The docs, the CameraCfg docstring and the changelog now describe env_mask as the environments due for the capture, whose frame and timestamp advance.
  • Wording fixes: "replace an output it reads" instead of "filter an output in place", the missing replace import in the rst example, and the two added bullets merged into one.

Count a None read_output return as a published capture

With the PR as is, a custom renderer whose read_output returns None runs no processors. Its processor outputs never appear in CameraData.output, and the first lookup raises a KeyError that is hard to trace. render_into_cameras now uses read_output(...) is not False, so older renderers keep working; only a renderer that holds back a capture needs to return False. The in-tree renderers still return True explicitly. This changes the Compatibility note in isaac-sim#8871's description.

Open: env_mask with async OVRTX

The mask is the camera's current _is_outdated. Async OVRTX publishes the previous request's capture, so with update_period > 0 and staggered resets the mask belongs to request T while the pixels come from request T-1. A stateful processor would then update the wrong rows. OVRTX already copies pose, intrinsics and frame for the delayed capture in prepare_capture. Fixing this needs the mask there too, which changes the renderer interface, so it is not in this PR. The other option is to document that the mask is exact only for synchronous renderers.

Validation

Windows, CUDA, Newton Warp renderer:

  • source/isaaclab/test/renderers/test_camera_output_contract.py and test_simulation_render_context.py: 49 passed. The PR head had 50; one test was folded into another.
  • pre-commit on the changed files: passed.
  • tools/changelog/cli.py check upstream/main: passed.
  • Not run: Isaac RTX and OVRTX camera tests.

Keep each request's data in its renderer group instead of re-indexing
the request list, build the processor buffers once with the capture mask,
and validate the reserved env_mask output name in CameraCfg.validate_config
so the data-type helper only computes.

Test the reserved name and processor-input validation through
validate(CameraCfg) instead of the private helper, and assert the mask
values and call count rather than only their length.

Renderers draw every environment on each capture, so describe env_mask as
the environments due for the capture rather than rows that still hold the
previous one. Merge the two added changelog bullets for the feature.
Renderers written before read_output returned a flag return None. Treating
that as unpublished silently skipped every tensor processor on those
renderers, leaving processor outputs missing from CameraData.output. Only
renderers that hold back a capture need to return False.
@github-actions github-actions Bot added documentation Improvements or additions to documentation isaac-lab labels Oct 29, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

documentation Improvements or additions to documentation isaac-lab

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant