Reduce Sharpa startup costs and improve RJ45 grasping by maxkra15 · Pull Request #8785 · isaac-sim/IsaacLab

maxkra15 · 2026-10-22T21:00:35Z

Sharpa startup built large nested Python payloads and imported visual geometry that native USD renderers did not use. Keep those payloads compact and load prepared visuals through the public asset root. The RJ45 follow-up improves middle-finger support during seating and stabilizes latch retention after release.

Preserve coffee/teapot NumPy payloads and reuse the existing mesh triangulation helper.
Import coffee/RJ45 Newton visual geometry only when Newton GL is selected.
Load the default GL switch front from three prepared material meshes, preserving all 1,151,285 triangles and authored normals. --backdrop datacenter retains the full rack; Kit/OVRTX retain their native scenery.
Use the public 512-pixel robot texture layer, referencing the existing geometry and physics. Disable unused hydroelastic contact-surface visualization output.
Preserve the calibrated hand reset pose, then decrease the middle MCP-FE target by 0.01 rad (0.57 degrees) through the existing 0.2 s closing ramp. Contact measurements showed the middle pad could carry zero normal force during final seating; the preload supplies support.
Set the rigid coupler entry to two local MJWarp substeps. This resolves the reproduced post-release latch loss in the tested cases while keeping the global substep count, cable simulation rate and control rate unchanged.

Scope against main: 14 files, +89/-54 lines. The switch front (30.5 MB) and texture layer/maps (3.04 MB) are public in the Isaac Sim 6.2 asset bundle. The changes use existing APIs and demo preparation paths.

Validation

Main 9860e2bd8, Newton 42f7def1, RTX PRO 6000 Blackwell GPU. Final configuration at 140262844:

Stock full-cycle RJ45 runs: 5/5 passed — Newton GL seeds 17/23, Kit seeds 17/23, and OVRTX (newton_rtx) seed 17. Each completed insertion, release, regrasp and 59.8–59.9 mm withdrawal. Results use the demo's reported success and finite-state checks, not process exit alone.
A separate Kit seed-17 retention trace stayed captured for 5 simulated seconds after release, with positive axial capture margin of 0.270–0.403 mm and at most 0.041 mm head movement per control step. The original rigid-step configuration reproduced a 9.56 mm jump after the fingers had opened.
Standalone CPU suite: 40 passed, 393 optional native rows skipped. Existing coupling CPU suite: 66 passed. Formatting, lint and changelog checks passed.
Unchanged startup/demo owners retain validation at da243da4e: all 11 supported bounded startup/render launches and the strict Sphinx build passed. Dry coffee delivered 1.242 g; wet coffee delivered 1.244 g with 1.274 g dissolved; teapot completed its 18 s pickup/pour/recovery sequence.

The final native runs used the exact committed environment configuration. Controller guards, gains, friction and success thresholds are unchanged. Two rigid substeps add rigid-solver work; the cable and outer collision rates remain unchanged. Shared-GPU wall times are not a performance comparison.

Remaining limits

Five full cycles establish the tested cases, not broad reliability across seeds or hardware. Existing Newton GL arm UDIM texture-resolution warnings also occur on unmodified main; inspected frames remain usable. CI for the final commit is running separately from the completed local/native checks above.

maxkra15 · 2026-10-22T21:01:03Z

run-ci

greptile-apps · 2026-10-22T21:05:26Z

Confidence Score: 5/5

[Medium impact] The PR appears safe to merge; no actionable defects were established.

Summary

This PR reduces presentation work when the Sharpa demos start.

Coffee starts with a studio in Newton GL and keeps the kitchen optional.
The teapot demo runs with Kit, Newton GL, OVRTX, or no visualizer.
RJ45 Newton GL shows the switch front and ports by default.
Teapot setup assigns its large lists after building the scene config.
Newton GL reuses file textures and can limit their upload size.

No actionable defects were established. maxkra15 explicitly documents that no new tests were added.

Diagram

%%{init: {'theme': 'neutral'}}%%
flowchart TD
  A[Choose visualizers and backdrop] --> B[Launcher resolves viewers and cameras]
  B --> C{Newton GL selected?}
  C -->|Yes| D[Import Newton visual meshes]
  C -->|No| E[Skip visual-only Newton meshes]
  D --> F[Build scene]
  E --> F
  F --> G[GL uploads model textures]
  F --> H[Kit and OVRTX read USD scene]
  G --> I[Reuse decoded textures during upload]

_{Reviews (1) · Last reviewed commit: "Reduce Sharpa demo scene startup costs" · Reviewed by Greptile}

isaaclab-review-bot

Isaac Lab Review Bot

Needs attention: Style consistency (4). Other reviewed categories look good.

PR description: Add the omitted cable-color fix and reproducible base-versus-head coffee/RJ45 startup comparisons with revisions, command, environment count, hardware, backend, warmup, and metric.
Design and architecture: Architecture looks good.
Scope and over-engineering: No over-engineering issues.
API: API changes look good.
Compatibility and deprecation: No breaking changes identified.
Implementation: Implementation looks good.
Style consistency: Added backdrop documentation is inaccurate and duplicated; correct the robot-only room claim and consolidate the RJ45 default description.
Test quality: Test quality looks good.

Minor fixes needed. Posted 4 inline findings.

Automated review; human maintainers own approval decisions.

isaaclab-review-bot · 2026-10-22T21:22:08Z

+    "--backdrop",
+    choices=["auto", "studio", "room"],
+    default="auto",
+    help="Backdrop: auto uses a simple studio for GL and the full room for Kit/RTX.",


🟡 Warning · Style Consistency — Backdrop help overstates room contents

cfg.room is only added when --robot rizon_sharpa is set, so --backdrop room loads just the kitchen island by default. State that the enclosed room applies only to the robot variant.

isaaclab-review-bot · 2026-10-22T21:22:08Z

 ```

+Omitting `--viz` selects Newton GL. Add `--extra video` and `--video` to record a run.
+The default `--backdrop auto` shows the detailed gold switch front and all RJ45


🟡 Warning · Style Consistency — Duplicated backdrop prose in README

Lines 21-24 restate the Newton GL and Kit/RTX backdrop defaults already given above, and break the sentence mid-clause after "In GL,". Keep one statement of the defaults and reflow it.

isaaclab-review-bot · 2026-10-22T21:22:08Z

            self.gui._render_scalar_plots = lambda: None
        self.register_ui_callback(self._render_training_controls, position="side")

+    def set_model(self, model: newton.Model | None) -> None:


🟡 Warning · Style Consistency — New methods precede class properties

set_model and log_mesh sit between init and the camera_speed property, breaking the prescribed order of dunders, properties, then public instance methods. Move both below the property block.

isaaclab-review-bot · 2026-10-22T21:22:08Z

                    "gravity": sim.cfg.gravity,
                },
                window_cfg=self.cfg.window,
+                texture_max_size=getattr(self.cfg.backend, "texture_max_size", None),


🟡 Warning · Style Consistency — getattr default hides backend-only field

The guide forbids getattr defaults for known attributes; only NewtonGLBackendCfg declares texture_max_size, so this shared initializer hides the capability difference. Read it in the GL visualizer or declare it on both backends.

Clarify backdrop documentation and keep GL-only settings at the GL initialization boundary. Move pixel resizing to the shared image owner while preserving the decoded texture cache and pixel behavior. Integrate upstream main at e4df255.

…-startup

maxkra15 · 2026-10-23T13:33:51Z

run-ci

Integrate recorded upstream main while retaining source-owned coffee floor visibility before cloning. Update the existing hard-reset check to verify the configured texture limit reaches the replacement model. Recorded upstream main: e907f82

AntoineRichard

Highlevel feedback. I'm confused by the texture size limitter. Othersize it looks reasonable.

AntoineRichard · 2026-10-26T10:56:17Z

+def resize_image(image: np.ndarray, max_size: int) -> np.ndarray:
+    """Downsample an image with Lanczos filtering while preserving its aspect ratio.
+
+    Args:
+        image: Uint8 RGB or RGBA pixels, shape ``(H, W, C)``. The input is not modified.
+        max_size: Positive integer limiting both output dimensions [px]. The caller validates
+            this limit before processing images.
+
+    Returns:
+        A resized pixel array, or ``image`` itself when both dimensions already fit.
+        Images are never enlarged; resized dimensions follow Pillow's thumbnail rounding.
+    """
+    if max(image.shape[:2]) <= max_size:
+        return image
+    resized = Image.fromarray(image)
+    resized.thumbnail((max_size, max_size), Image.Resampling.LANCZOS)
+    return np.array(resized)


KInd of weird to have this utility here since it's not GPU capable? If it's only used to resize a texture, can we move it where its used?

AntoineRichard · 2026-10-26T10:57:04Z

+            if path not in self._model_texture_cache:
+                image = load_texture(texture)
+                if image is not None and self._texture_max_size is not None:
+                    image = resize_image(image, self._texture_max_size)


Why not use a lower resolution texture? I'm not convinced by this hack here.

AntoineRichard · 2026-10-26T10:59:45Z

Not convinced by the texture max size. Why not use a lower resolution texture to start with?

Integrate upstream main d78054a to retain compatibility with current backend ownership and dependencies.

Integrate upstream main at ce2900d to keep the branch current with shared API and dependency changes. Validate the merged revision with focused checks and full formatting and lint hooks before committing. Remote CI remains separate.

Integrate upstream main at 6d37745 to keep the branch current with shared API and dependency changes. Validate the merged revision with focused checks and all repository formatting and lint hooks. Remote CI remains separate.

Reduce Sharpa demo scene startup costs

0c39a9e

maxkra15 requested a review from a team October 22, 2026 21:00

maxkra15 requested review from AntoineRichard, StafaH, fatimaanes, hujc7, kellyguo11, matthewtrepte, ooctipus and pbarejko as code owners October 22, 2026 21:00

github-actions Bot added documentation Improvements or additions to documentation isaac-lab Related to Isaac Lab team labels Oct 22, 2026

isaaclab-bot Bot added ci:run-docker Trigger the on-demand Docker and GPU CI workflow and removed ci:run-docker Trigger the on-demand Docker and GPU CI workflow labels Oct 22, 2026

isaaclab-review-bot Bot reviewed Oct 22, 2026

View reviewed changes

maxkra15 added 6 commits October 23, 2026 06:25

fix: Address Sharpa startup review feedback

9b6d304

Clarify backdrop documentation and keep GL-only settings at the GL initialization boundary. Move pixel resizing to the shared image owner while preserving the decoded texture cache and pixel behavior. Integrate upstream main at e4df255.

Load a prepared RJ45 switch front

83611cf

Keep Sharpa demo payloads in NumPy arrays

dcf896d

Merge remote-tracking branch 'origin/main' into maxkra15/sharpa-scene…

2dad7d2

…-startup

Simplify Sharpa scene preparation and texture upload

a24a95c

Preserve RJ45 failures before Kit launcher teardown

1d4c8c0

isaaclab-bot Bot added ci:run-docker Trigger the on-demand Docker and GPU CI workflow and removed ci:run-docker Trigger the on-demand Docker and GPU CI workflow labels Oct 23, 2026

AntoineRichard reviewed Oct 26, 2026

View reviewed changes

maxkra15 added 2 commits October 27, 2026 06:13

chore: Sync main into PR branch

f414d7a

Integrate upstream main d78054a to retain compatibility with current backend ownership and dependencies.

chore: Merge main into PR isaac-sim#8785

fe88d0e

Integrate upstream main at ce2900d to keep the branch current with shared API and dependency changes. Validate the merged revision with focused checks and full formatting and lint hooks before committing. Remote CI remains separate.

maxkra15 added 4 commits October 28, 2026 10:48

Simplify Sharpa demo startup preparation

bc39de9

Merge latest main into Sharpa startup rework

c4d2716

Use published Sharpa startup assets

4ae7f30

Merge current PR history into validated startup rework

da243da

maxkra15 changed the title ~~Reduce Sharpa demo scene startup costs~~ Reduce Sharpa demo startup preparation Oct 28, 2026

Preload the middle finger for RJ45 insertion

9b792a9

maxkra15 changed the title ~~Reduce Sharpa demo startup preparation~~ Reduce Sharpa startup costs and improve RJ45 grasping Oct 28, 2026

maxkra15 added 2 commits October 28, 2026 15:29

Refine rigid stepping for RJ45 latch stability

1402628

chore: Merge main into PR isaac-sim#8785

2c07888

Integrate upstream main at 6d37745 to keep the branch current with shared API and dependency changes. Validate the merged revision with focused checks and all repository formatting and lint hooks. Remote CI remains separate.

Navigation Menu

Repository navigation

Reduce Sharpa startup costs and improve RJ45 grasping - #8785

Reduce Sharpa startup costs and improve RJ45 grasping#8785
maxkra15 wants to merge 17 commits into
isaac-sim:mainfrom
maxkra15:maxkra15/sharpa-scene-startup

maxkra15 commented Oct 22, 2026 •

edited

Loading

Uh oh!

maxkra15 commented Oct 22, 2026

Uh oh!

greptile-apps Bot commented Oct 22, 2026 •

edited

Loading

Uh oh!

isaaclab-review-bot Bot left a comment

Uh oh!

isaaclab-review-bot Bot Oct 22, 2026

Uh oh!

isaaclab-review-bot Bot Oct 22, 2026

Uh oh!

isaaclab-review-bot Bot Oct 22, 2026

Uh oh!

isaaclab-review-bot Bot Oct 22, 2026

Uh oh!

maxkra15 commented Oct 23, 2026

Uh oh!

AntoineRichard left a comment

Uh oh!

AntoineRichard Oct 26, 2026

Uh oh!

AntoineRichard Oct 26, 2026

Uh oh!

AntoineRichard Oct 26, 2026

Uh oh!

This branch has not been deployed

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Conversation

maxkra15 commented Oct 22, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Validation

Remaining limits

Uh oh!

maxkra15 commented Oct 22, 2026

Uh oh!

greptile-apps Bot commented Oct 22, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Confidence Score: 5/5

Summary

Diagram

Uh oh!

isaaclab-review-bot Bot left a comment

Choose a reason for hiding this comment

Isaac Lab Review Bot

Uh oh!

isaaclab-review-bot Bot Oct 22, 2026

Choose a reason for hiding this comment

Uh oh!

isaaclab-review-bot Bot Oct 22, 2026

Choose a reason for hiding this comment

Uh oh!

isaaclab-review-bot Bot Oct 22, 2026

Choose a reason for hiding this comment

Uh oh!

isaaclab-review-bot Bot Oct 22, 2026

Choose a reason for hiding this comment

Uh oh!

maxkra15 commented Oct 23, 2026

Uh oh!

AntoineRichard left a comment

Choose a reason for hiding this comment

Uh oh!

AntoineRichard Oct 26, 2026

Choose a reason for hiding this comment

Uh oh!

AntoineRichard Oct 26, 2026

Choose a reason for hiding this comment

Uh oh!

AntoineRichard Oct 26, 2026

Choose a reason for hiding this comment

Uh oh!

This branch has not been deployed

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

maxkra15 commented Oct 22, 2026 •

edited

Loading

greptile-apps Bot commented Oct 22, 2026 •

edited

Loading