Skip to content

Add experimental Franka suction parcel sorting - #8873

Draft
maxkra15 wants to merge 69 commits into
isaac-sim:mainfrom
maxkra15:maximiliank/conveyor-suction-sorting-lean
Draft

maxkra15 wants to merge 69 commits into
isaac-sim:mainfrom
maxkra15:maximiliank/conveyor-suction-sorting-lean

Conversation

@maxkra15

@maxkra15 maxkra15 commented Oct 29, 2026 •

Copy link
Copy Markdown
Contributor

Description

Add IsaacContrib-Conveyor-Suction-Sorting: a Franka with four suction cups sorts twelve differently sized cuboids into five powered outfeeds and physical bins. One policy uses 51 observations and four actions. Newton's seal prototype applies compliant forces; parcels are never attached by writing their poses.

Franka suction sorting

The task reuses the conveyor lifecycle and shared success-guided sampler. Training and playback target twelve deliveries within 240 seconds. Its 96 reset recipes initialize partial physical batches for shorter remaining-batch problems. Pre-sorted inventory earns no receipt reward; playback starts with every parcel unsorted.

Dependencies and review

Depends on #8327, #8331 and #8872. Warehouse sorting and optional PhysX support are excluded.

Original task-only comparison: 25 files, +1,964/−34, including 272 test lines. This immutable comparison excludes the later main merge, MPM inertial fix, and one-line index-allocation cleanup. Current Files changed includes the prerequisite PRs and the separately authored MPM fix, which is preserved.

This draft pins an experimental public extraction of Newton's unmerged surface-gripper prototype and matching MuJoCo/MJWarp 3.16 dependencies.

Validation and limits

  • Native gripper and two receipt/reset checks passed; a missing-physical-inventory negative control fails as intended.
  • The 32-world full-start cohort is byte-identical to the development reference; policy dimensions remain 51/4.
  • Lock consistency, full formatting/lint, changelog and environment-browser checks passed.
  • On this cleaned branch, a warm-started development checkpoint completed 43/96 full batches, with 732 physical correct-bin deliveries and no non-finite states. Evaluation used deterministic actions, two fixed full-start cohorts, and actual receipt masks before reset. This is bounded compatibility evidence, not reliable sorting or a from-scratch learning result.
  • A separate 16-GPU policy trained from scratch reached 26/96 full batches and 558 correct-bin deliveries at update 4,500, up from 0/96 and 137 at update 1,999. The same settings continue toward 6,000 updates, alongside a short stronger-failure-penalty comparison; worker-local sampling history restarts on resume. These are two fixed full-start cohorts, with zero non-finite states. Selected Kit footage contains an incomplete three-delivery episode.

CI currently fails the fixed-only URDF PhysX-variant assertion: the standalone importer emits none/physics, while the test expects physx. The test, converter and importer pins are byte-identical to main. A matching baseline runtime was unavailable locally; no task-dependency workaround or skip was added. ARM rendering also failed: the first attempt exceeded an image threshold by 0.01 percentage point; one retry hit a CUDA illegal-memory-access error in Shadow Hand segmentation. No originating kernel or suction-task regression has been established. Reliable full-batch learning and the upstream gripper dependency remain open.

Keep the custom mesh branch current with develop at 9b6d2d8.\n\nThe unchanged feature diff passed the mesh spawner tests and the\nrepository-wide validation gate before this merge was committed.
Integrate the recorded main revision while preserving the PR scope.

Validate the merged tree with focused tests and full repository hooks.
Integrate upstream main e907f82
to keep this branch current while preserving its feature scope.
Use the clone configuration owner for existing conveyor namespace
defaults. This preserves the public values while satisfying the current
clone-ownership check across the integrated task stack.

Recorded upstream main: e907f82
Integrate upstream main d78054a
to retain compatibility with current backend ownership and dependencies.
Integrate upstream main at ce2900d to keep
the branch current with shared API and dependency changes.

Validate the merged revision with focused checks and full formatting
and lint hooks before committing. Remote CI remains separate.
Reuse the racetrack conveyor lifecycle and public reset state sampler for a physical 12-parcel suction task. Keep the experimental Newton prototype pin explicit and preserve the public solver-observables API.
@github-actions github-actions Bot added documentation Improvements or additions to documentation isaac-lab Related to Isaac Lab team infrastructure labels Oct 29, 2026
Integrate upstream main at 6d37745 to keep
the branch current with shared API and dependency changes.

Validate the merged revision with focused checks and all repository
formatting and lint hooks. Remote CI remains separate.
Keep authored rigid-body mass and inertia available to MuJoCo while
Newton applies kinematic collider mass locally in its MPM solver.
Avoid erasing the pouring target inertials before solver creation.

Retain coverage of shared rigid/MPM construction and the native pouring
regression, including failure proofs before the fix.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

documentation Improvements or additions to documentation infrastructure isaac-lab Related to Isaac Lab team

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant