ViewpointGeneration

A ROS 2 package for automated inspection viewpoint generation from CAD models. Given a surface mesh, the library segments the surface into regions, clusters each region to fit within a camera's field of view, and computes collision-free camera viewpoints for full-surface inspection coverage.

Maintainer: Colin Acton (actonc@uw.edu)

Overview

ViewpointGeneration is structured as a standalone Python library (viewpoint_generation) with a ROS 2 node wrapper (ViewpointGenerationNode) that exposes the library's pipeline through ROS parameters and services.

The pipeline follows these stages:

1. Load CAD Model -- Import an STL/OBJ/PLY/OFF mesh, or a STEP file, with unit conversion. STEP files are tessellated through PythonOCC and additionally carry their B-rep topology (see STEP ingest)
2. Surface Segmentation -- Segment the mesh directly into regions (by triangle face, not a sampled point cloud). Three interchangeable algorithms (selected via regions.algorithm):
   • region_growth -- contiguous regions grown over triangle face-adjacency, using face-normal similarity and a per-face curvature analog (variance of neighboring face normals) as the merge criteria
   • partfield -- semantic parts from PartField, grouped directly from its per-face labels (requires GPU + PartField mounted at /models/PartField)
   • brep -- one region per analytical B-rep face from the CAD model itself (requires a STEP-loaded mesh). No seeds, no thresholds, no GPU: the segmentation is the one the CAD author drew. Small faces are merged into their largest neighbour, and adjacent same-type faces can optionally be unioned
3. Sample Region Point Clouds -- Poisson disk sample each region's own submesh (sized from fov_clustering.point_density); cached to disk and referenced from the results file
4. FOV Clustering -- Subdivide each region's sampled point cloud into clusters that fit within the camera's field of view and depth of field. Two interchangeable algorithms (selected via regions.fov_clustering.algorithm):
   • kmeans -- K-means with Bayesian optimization for cluster count
   • greedy_cover -- greedy set cover: a candidate viewpoint covers a region point only if it passes depth-of-field, field-of-view, and photometric-incidence tests and (when occlusion_check is enabled) has an unoccluded line of sight to that point against the full part mesh, not just the region being clustered. Points the straight-normal predicate leaves occluded get one more chance when rescue_search is enabled: a Monte Carlo hemisphere search (see occlusion_search.py) looks for any unoccluded viewing angle within standard_normal_threshold of that point's normal (angles beyond it are too glancing to image the surface, so they are neither sampled nor accepted), preferring ones close to its true normal. Points still unrescuable are reported per-region as blind_spots rather than silently folded into a cluster (see JSON Results Format below). Each emitted cluster carries its winning candidate's occlusion-validated axis forward as candidate_axis, for reuse in the next stage. Candidate anchors normally come from blue-noise farthest-point sampling; setting structured_candidates instead lays them on a per-region cylindrical grid (elevation = world Z, azimuth = angle around a vertical axis through the part's center, both spaced by candidate_spacing) so viewpoints from different regions tend to land at shared elevations/azimuths -- e.g. rings around a roughly cylindrical part -- at some cost to the greedy algorithm's freedom to pick the most locally-efficient anchor
5. Project Viewpoints -- Compute a camera pose (position + orientation) for each cluster at the configured focal distance. The naive direction (mean of the cluster's own point normals) is refined by the same hemisphere search used above, using candidate_axis (when present) as a guaranteed-good starting point so refinement can never do worse than clustering already proved — this closes a gap where the mean direction was never itself re-validated against occlusion. The winning direction is tagged imaging_mode: photometric if it falls within the photometric incidence cone (fov_normal_threshold, supports photometric-stereo normal-map reconstruction), standard if only an off-cone angle within standard_normal_threshold is occlusion-clear (still gets an image, just not photometric-quality), or inaccessible if no angle inside standard_normal_threshold is occlusion-clear — directions beyond it are too glancing to image the surface and are neither sampled nor considered valid. FOV clustering and viewpoint projection use separate occlusion scenes so the two stages answer different questions. FOV clustering uses a mesh-only RaycastingScene (self-occlusion only): cluster feasibility is pose-independent -- it decides which surface a camera could ever image, regardless of how the part is placed. Viewpoint projection uses a mesh + ground-plane scene, with the ground-plane occluder (just below the table/turntable surface, z=0) transformed into the part's current placement (the object_frame → model_frame pose), so projection respects the actual environment and no emitted viewpoint's camera position or line-of-sight ends up underground at the part's current pose. Because Open3D's RaycastingScene is append-only, the ground plane is toggled by rebuilding the scene with or without it (_make_raycasting_scene(include_ground=...))
6. Optimize Traversal -- Solve a TSP to order viewpoints for efficient robot motion

Results are saved as a JSON file.

Package Structure

viewpoint_generation/
├── viewpoint_generation/           # Core Python library
│   ├── viewpoint_generation.py     # Main ViewpointGeneration class
│   ├── region_growth.py            # Region growing segmentation
│   ├── partfield_segmentation.py   # PartField part segmentation (GPU, subprocess)
│   ├── fov_clustering.py           # Field-of-view clustering
│   ├── viewpoint_projection.py     # Viewpoint pose computation
│   ├── occlusion_search.py         # Monte Carlo hemisphere occlusion search (blind-spot rescue + direction refinement)
│   ├── brep_segmentation.py        # B-rep topology segmentation (third algorithm)
│   ├── mesh_utils.py               # Shared triangle-mesh helpers (submesh extraction; re-exports cad_loader.mesh_io)
│   ├── plan_io.py                  # Plan file naming, stage inference, read/write, describe_plan
│   ├── visualizer.py               # Open3D visualization
│   ├── gui_node.py                 # GUI node implementation
│   ├── foxglove_scene.py           # Plan -> numpy scene for the Foxglove viz (no ROS, no GUI)
│   └── assets/
│       ├── materials.py            # Visualization materials
│       └── planning_volume.stl     # Planning volume mesh
├── nodes/                          # ROS 2 node executables
│   ├── viewpoint_generation_node.py
│   ├── task_planning_node.py
│   ├── viewpoint_traversal_node.py
│   ├── gui.py
│   └── foxglove_viz_node.py        # Publishes each stage's results for Foxglove's 3D panel
├── launch/
│   ├── bringup.launch.py           # Full system bringup
│   ├── viewpoint_generation.launch.py
│   └── viewpoint_traversal.launch.py
├── test/                           # pytest suites (see "Tests" below)
├── package.xml
├── setup.py
└── requirements.txt

A companion package, viewpoint_generation_interfaces/, defines the ROS 2 message, service, and action interfaces.

The PLM parts catalog, the part picker and the STEP loader live in the plm_bridge repository (public, BSD-3-Clause): this package depends on its plm_interfaces (part selection, plan recording) and cad_loader (STEP and mesh loading) packages, and its bringup starts the plm_bridge node when that is installed.

Tests

test/ holds tests for plan_io: stage inference, file naming, every plan file shape it reads (3.0, 2.0 with its context mapped to 3.0, the 1.0 envelope, bare results) and describe_plan(), the producer's side of the plan contract with plm_bridge. They need neither ROS nor a network. The catalog's tests moved to plm_bridge, and the STEP loader's to cad_loader. Run them inside the container:

cd viewpoint_generation
python3 -m pytest test/ --deselect test/test_flake8.py::test_flake8 \
    --deselect test/test_pep257.py::test_pep257

The ament flake8/pep257 checks are deselected because the package predates them and fails them throughout.

Dependencies

Python

Listed in requirements.txt:

• numpy (<1.25.0)
• open3d
• pytransform3d
• bayesian-optimization
• matplotlib
• requests

For STEP ingest and the brep segmentation algorithm: the cad_loader package (plm_bridge repository), which needs cadquery-ocp (the maintained OpenCascade Python bindings, imported as OCP). It imports them lazily, so the rest of the package works without them -- only loading a .stp/.step file requires them, and the error names the fix.

For the optional partfield segmentation algorithm: a CUDA GPU and the PartField checkout mounted at /models/PartField (with its model/model_objaverse.ckpt checkpoint and PartField's own dependencies — torch, lightning, torch-scatter, etc.). In this repo's Docker image these are installed via docker/requirements.txt and the mount is provided by docker-compose.yaml.

ROS 2

• sensor_msgs
• geometry_msgs
• plm_interfaces, cad_loader (plm_bridge repository)
• ros2launch
• MoveIt 2 (for viewpoint traversal)
• zivid_camera (zivid-ros, Zivid 2+ MR60 driver)

Library Usage

The core library can be used independently of ROS:

from viewpoint_generation.viewpoint_generation import ViewpointGeneration

vg = ViewpointGeneration()

# Load mesh
vg.set_mesh_file("part.stl", units="mm")

# Configure and run pipeline
# vg.set_segmentation_algorithm("partfield")   # optional: use PartField instead
vg.segment_regions()           # segments the mesh, then samples a point cloud per region
vg.fov_clustering()
vg.project_viewpoints()

Configuration

All algorithm parameters are exposed as dataclass configs with sensible defaults:

RegionGrowingConfig -- grows regions over the mesh's triangle face-adjacency graph (two faces are neighbors only if they share an edge), so growth can never bridge across a fold, thin wall, or gap the way a spatial-radius search over a sampled point cloud could. "Curvature" here is a per-face analog: how much a face's normal deviates from its adjacent faces' normals. Adjacency is computed against a vertex-welded copy of the mesh (weld_epsilon), not the raw input: STL has no vertex-sharing guarantee, and a mesh exported with each triangle's vertices stored independently (common for CAD-authored STLs) would otherwise silently read as disconnected 1-2-triangle islands, one per facet, with no way to detect the real edges between them.

Parameter	Default	Description
seed_threshold	0.1	Curvature threshold for seed face selection
min_cluster_size	10	Minimum faces per region
max_cluster_size	100000	Maximum faces per region
normal_angle_threshold	pi/3	Maximum normal deviation (radians) for region membership
curvature_threshold	0.1	Maximum curvature difference for region membership
weld_epsilon	1e-6 m	Distance within which vertices are merged before computing face adjacency

Tuning for low-poly / faceted parts. seed_threshold's default (0.1) assumes some triangles sit far enough from any edge to read as near-zero curvature. On a coarse or heavily-faceted mesh (few triangles per flat facet), every triangle can be adjacent to a real edge, so nothing clears the default threshold and segmentation silently returns zero regions. Raise seed_threshold/curvature_threshold to comfortably exceed the mesh's actual per-face curvature range (print RegionGrowing.face_curvatures after preprocess_mesh to check it) and lower min_cluster_size to the smallest facet you still want kept as its own region -- normal_angle_threshold remains what separates genuinely distinct facets, since real part edges are almost always well above it.

PartFieldSegmentationConfig (used when segmentation_algorithm == 'partfield')

Parameter	Default	Description
num_parts	12	Number of parts to segment the mesh into
use_agglo	True	Agglomerative clustering (spatially connected parts) instead of KMeans
option	0	Face-adjacency graph for agglomerative clustering: 0=naive, 1=face-MST, 2=cc-MST
with_knn	False	Augment the face-adjacency graph with kNN connections (agglomerative only)

PartField runs as a subprocess (partfield_inference.py then run_part_clustering.py) against an exported copy of the loaded mesh. Its per-face part labels are grouped directly into regions (a list of triangle indices per part) with no further projection step. Mesh preprocessing/remeshing is left disabled so face ordering is preserved. Note: PartField produces no noise faces (every face is assigned a part).

FOVClusteringConfig

Parameter	Default	Description
fov_diameter	0.03 m	Camera field of view diameter at focal distance
dof	0.02 m	Depth of field
point_density	0.5	Target points per square millimeter, both for sampling each region's point cloud and for K-means cluster evaluation. Only needs to resolve FOV coverage/packing decisions, not final inspection imagery, so this can stay low
lambda_weight	1.0	Weight for out-of-FOV penalty in cost function (K-means only)
beta_weight	1.0	Weight for packing efficiency in cost function (K-means only)
point_weight	1.0	Weight for point positions in K-means
normal_weight	1.0	Weight for point normals in K-means
algorithm	kmeans	Clustering algorithm: kmeans (Bayesian-optimised K-means) or greedy_cover (greedy set cover)
fov_normal_threshold	π/4 rad	Max surface-normal incidence angle for photometric-stereo imaging (greedy_cover coverage predicate)
standard_normal_threshold	π/3 rad	Max surface-normal incidence angle for standard imaging; between fov_normal_threshold and this angle a viewpoint is standard-tier, and beyond it the view is too glancing to capture surface information and is treated as inaccessible (neither sampled during the hemisphere search nor considered valid)
candidate_spacing	0.0 m	Anchor spacing for greedy_cover candidate sampling; 0.0 = auto = fov_diameter/2
prune_redundant	true	Remove redundant viewpoints after greedy cover (cannot open coverage holes)
rng_seed	0	Random seed for greedy_cover candidate sampling (reproducibility)
occlusion_check	true	Require unobstructed line-of-sight to the full part mesh (criterion 4 of the greedy_cover coverage predicate). Disabling reproduces the pre-occlusion (criteria 1-3 only) behavior
occlusion_epsilon	1e-4 m	Shrink margin subtracted from the occlusion ray's far bound so a point's own triangle is never mistaken for its own occluder
rescue_search	true	Attempt a Monte Carlo hemisphere search for an alternative viewing angle before giving up on an occluded blind-spot point (greedy_cover only)
rescue_samples	64	Hemisphere samples per blind-spot rescue attempt
structured_candidates	false	Lay greedy_cover candidate anchors on a per-region cylindrical grid (elevation/azimuth around the part center, spaced by candidate_spacing) instead of blue-noise farthest-point sampling, so viewpoints across regions tend to share elevations/azimuths

ViewpointProjectionConfig

Parameter	Default	Description
focal_distance	0.3 m	Distance from viewpoint to surface
hemisphere_points	64	Hemisphere samples for the occlusion-aware direction-refinement search in project_viewpoints (per cluster, not per point)

ROS 2 Interface

ViewpointGenerationNode

Wraps the core library, exposing each pipeline stage as a ROS service and all configuration as ROS parameters.

Services (all std_srvs/srv/Trigger):

Service	Description
/viewpoint_generation/segment_regions	Segment the mesh into regions (region growth or PartField) and sample each region's point cloud
/viewpoint_generation/fov_clustering	Cluster regions by camera FOV
/viewpoint_generation/viewpoint_projection	Generate camera viewpoints
/viewpoint_generation/optimize_traversal	Optimize viewpoint visit order (TSP)

Parameters:

All RegionGrowingConfig, PartFieldSegmentationConfig, FOVClusteringConfig, and ViewpointProjectionConfig fields are declared as ROS parameters with type information and valid ranges (prefixed regions.region_growth., regions.partfield., regions.fov_clustering., and viewpoints.projection. respectively). Additional parameters:

• regions.segmentation_algorithm -- Surface segmentation algorithm: region_growth (default) or partfield
• regions.fov_clustering.algorithm -- FOV clustering algorithm: kmeans (default) or greedy_cover
• model.mesh.file -- Path to the mesh file
• model.mesh.units -- Mesh units (m, cm, mm, in)
• model.point_cloud.file -- Path to a manually-loaded point cloud (optional; independent of the per-region point clouds sampled during segmentation)
• model.point_cloud.units -- Point cloud units Part placement is not authored via parameters. The mesh, regions, and viewpoints are always generated and stored in the mesh's own origin frame (model_frame); the mesh file's origin is never moved. Where the part physically sits is a live TF placement: tsdf_pose registers the part and broadcasts the object_frame -> model_frame transform. That placement is applied only to placement consumers -- the /planning_scene collision object pose, the GUI visualization (Visualizer.apply_model_placement), and the occlusion ground-plane in the raycasting scene (ViewpointGeneration.set_placement) -- and is never baked into the geometry or written to a results file. Downstream viewpoint goals are stamped in model_frame so MoveIt resolves them through the same TF. (The old model.pose.* parameters, which baked a pose into the geometry and rigidly re-transformed / re-saved results on every /tsdf_pose/pose message, have been removed.)
• results.file -- Path to results JSON
• settings.data_path -- Base data directory

Mesh/region/viewpoint selection state lives on the task_planning node (navigation.selected_mesh, navigation.selected_region, navigation.selected_viewpoint), not here. The GUI and visualizer track those parameters; navigation.selected_mesh is visualization-only while navigation.selected_region/navigation.selected_viewpoint also scope robot execution.

Publishers:

• /planning_scene (moveit_msgs/PlanningScene) -- MoveIt planning scene with mesh collision objects

Subscribers:

• /particle_filter/pose (geometry_msgs/PoseStamped) -- Object pose for planning scene updates

Other Nodes

• task_planning_node -- State machine for robot motion control, manages servo/trajectory controller switching. Parameters are namespaced (controllers.*, navigation.*, settings.*) so the GUI renders one tab per namespace. Owns the mesh/region/viewpoint selection parameters (navigation.selected_mesh, navigation.selected_region, navigation.selected_viewpoint, declared with live slider ranges) and navigation.selected_traversal_algorithm; settings.results_file points it at the results JSON to plan/execute over; the GUI/visualizer track these to highlight the selected geometry

• viewpoint_traversal_node -- MoveIt-based motion planning to viewpoints with TSP/VRP optimization and workspace constraints. Exposes move_to_pose_stamped, optimize_traversal, and find_nearest_viewpoint services. The VRP optimizer minimizes an execution-time surrogate by default (vrp_cost_mode='time': per-segment TOTG time from the joint velocity/acceleration limits), or weighted joint-space distance in joint mode; with vrp_validate_topk>1 it plans the top-K candidate tours and keeps the one with the lowest real MoveIt time. The find_nearest_viewpoint service returns the viewpoint in a given region that is closest to the robot's actual current joint state under the same cost model, enabling dynamic entry-point selection at execution time rather than relying on precomputed IK-based distances

• gui_node -- Open3D visualization GUI with interactive mesh, region, cluster, and viewpoint rendering. Visualization is split into two orthogonal axes:

  • Region surface (exclusive) — View → Region Surface selects one coloring for the region surfaces: Solid (uniform region color) or Cluster (colored by owning cluster). Each region is its own exact triangle submesh (segmentation is mesh-native, so no subdivision or nearest-point projection is needed to render it), so selecting a region makes the others semi-transparent to focus on it. Selecting a region also shows only its path, shows the enabled overlays for its viewpoints, and auto-selects its first viewpoint.
  • Viewpoint overlays (inclusive) — View → Viewpoint Overlays independently toggles any combination of per-viewpoint geometries: Viewpoint Marker, FOV Cylinder, Origin Line (surface origin → camera), Frustum, and Origin Marker. FOV Cylinder is a white wireframe cylinder (radius fov_diameter/2, height dof) centered on each cluster's surface target and aligned to its averaged view direction — the literal coverage volume; overlapping FOVs visibly intersect, honestly depicting the covering solution. Origin Marker is a small white sphere at the cluster's surface origin, always white rather than tinted by selection or cluster color. Viewpoint Marker is colored by the viewpoint's imaging_mode: green for photometric, yellow for standard, red for inaccessible (blue overrides all when the viewpoint is selected). This set applies to the non-selected viewpoints.
  • Selected viewpoint overlays — View → Selected Viewpoint Overlays is a parallel, independent set of the same toggles that applies only to the currently-selected viewpoint (drawn with highlight colors). Because the two sets are independent, you can, e.g., show the FOV cylinder for the selected viewpoint only by enabling it here while leaving it off in Viewpoint Overlays.

• foxglove_viz_node (viewpoint_generation_viz) -- the remote alternative to the Open3D GUI's 3D view: publishes the current plan as topics Foxglove's 3D panel draws natively, so a plan can be inspected from anywhere the Foxglove bridge reaches. See Foxglove visualization.

VRP Traversal Optimization Parameters (viewpoint_traversal_node)

The VRP optimizer treats the inspection as a clustered vehicle-routing problem: the arm (starting/ending at its home "depot") must visit every viewpoint, region by region, minimising motion in joint space. vrp_algorithm selects the solver; the mathematical formulation and per-algorithm details are documented in VRP_SYSTEM_REFERENCE.md.

Parameter	Default	Description
vrp_algorithm	''	VRP solver: vrp_greedy, vrp_2opt, vrp_3opt, vrp_ils, vrp_lkh, vrp_aco, vrp_hierarchical, vrp_clustered. Empty → use TSP instead
vrp_cost_mode	time	time = TOTG execution-time surrogate (seconds); joint = weighted joint-space distance (radians)
vrp_max_velocity	0.5	Per-joint max velocity (rad/s) for the time model
vrp_max_acceleration	1.0	Per-joint max acceleration (rad/s²) for the time model
vrp_joint_weights	all 1.0	Length-7 per-joint weights, joint mode only
vrp_validate_topk	1	>1: plan the top-K candidate tours with MoveIt, keep the lowest real execution time
vrp_aco_n_ants / _n_iter	20 / 100	ACO colony size and iteration count
vrp_aco_alpha / _beta / _rho	1.0 / 2.0 / 0.1	ACO pheromone weight / heuristic weight / evaporation rate
vrp_aco_n_jobs	1	>1: run ACO ants across processes (spawn pool)
vrp_clustered_k	6	Candidate entry/exit ports per region for vrp_clustered
vrp_n_turntable_samples	0	>0 enables the multi-config turntable sweep — multiple feasible IK configs per viewpoint, activating config-aware vrp_clustered and weight-independent reachability
vrp_max_configs_per_vp	8	Cap on candidate IK configs kept per viewpoint (sweep)
vrp_config_dedup_tol	0.1	Joint-space L∞ tolerance (rad) for merging near-identical configs (sweep)

Custom Interfaces (viewpoint_generation_interfaces)

Actions:

• ViewpointGeneration.action -- Trigger viewpoint generation (goal/feedback/result as strings)
• InspectRegion.action -- Execute a region inspection task

Services:

• ImportCadModel.srv -- Load a CAD model (file path + units)
• MoveToPoseStamped.srv -- Move robot to a target pose
• OptimizeViewpointTraversal.srv -- Optimize traversal order for a results file
• FindNearestViewpoint.srv -- Given a region index, return the viewpoint in that region nearest to the robot's current joint state (used for dynamic entry selection at execution time)

The catalog's interfaces (part selection, plan recording, DescribePlans) moved to plm_interfaces in the plm_bridge repository.

Messages:

• OrientationControlData.msg -- Orientation control feedback (pitch/yaw/roll errors, PID gains)
• FocusValue.msg -- Focus metric data
• AutofocusData.msg -- Autofocus feedback (images, metrics, ROI, poses)

Launch

Viewpoint Generation Only

ros2 launch viewpoint_generation viewpoint_generation.launch.py \
    object:=my_part \
    data_path:=/path/to/data

Arguments:

• object -- Config/object name
• data_path -- Base data directory
• headless_mode -- Run without GUI (default: false)
• compute_threads -- Cap on CPU threads/worker processes for the segmentation + FOV-clustering compute (default: 6). The node's PartField/torch BLAS threads and the sklearn/loky worker pool otherwise fan out across every core, which saturates the CPU (and swaps) and starves the real-time ros2_control loop when they share a machine. Applied via OMP_/MKL_/OPENBLAS_/NUMEXPR_NUM_THREADS and LOKY_MAX_CPU_COUNT. Keep it well below the core count on a shared box; raise it on a dedicated compute host.

Full System Bringup

ros2 launch viewpoint_generation bringup.launch.py \
    cell:=alpha \
    object:=my_part \
    data_path:=/path/to/data \
    sim:=true

Additional arguments:

• cell -- Inspection cell configuration (alpha or beta)
• sim -- Use fake/simulated hardware
• dual_arm -- Add the second UR5e carrying the Zivid 2+ MR60 (cell beta only; default false). Forwarded to inspection_cell_moveit_config/launch/inspection_cell.launch.py (URDF, controllers, MoveIt groups, Servo) and to inspection_control/launch/admittance_control.launch.py (admittance control for that arm; the joystick's arm-switch button, 10 / right stick click, moves teleop between the arms). It also defines zivid_camera_optical_frame, the frame the Zivid driver publishes in.
• zivid -- With cell enabled, start the Zivid 2+ MR60 driver as /zivid/zivid_camera (default: true). It publishes in zivid_camera_optical_frame; capture with ros2 service call /zivid/capture std_srvs/srv/Trigger. This is the only place the driver starts (the cell launch is passed launch_zivid:=false), since only one client can hold the camera: close Zivid Studio first. zivid_serial_number picks a camera (default: first available). zivid_settings_file is a 3D settings .yml from Zivid Studio; without it, captures fail until settings are set (e.g. /zivid/capture_assistant/suggest_settings).
• catalog -- Launch the PLM catalog node (plm_bridge_ros, from the plm_bridge repository; default: true). If that package is not installed, bringup logs a message and carries on. Like the Foxglove bridge it is not gated on cell: browsing the catalog and caching STEP files is useful with or without cell hardware. It serves whatever is already cached even when the DX_* credentials are absent.
• picker -- Launch the browser part picker alongside the catalog node (default: true, requires catalog:=true)
• picker_port -- Port the picker listens on (default: 5050). With network_mode: host it is reachable at http://localhost:5050.
• catalog_sync_interval -- Seconds between background incremental syncs (default: 300; 0 disables the timer)
• catalog_sync_on_startup -- Run a full sync at bringup (default: false; a full pass over a broad DX_BOOKMARK_SCOPE takes minutes, and the periodic timer and the picker's Sync button cover it)
• plm_backends -- PLM backends the catalog hosts (default: $PLM_BACKENDS, else dx3)
• catalog_mesh_units -- Units announced with a catalog selection (default: mm). Ignored for STEP files, which declare their own units (see STEP ingest); it only matters for a hand-placed non-STEP model
• foxglove_viz -- Start foxglove_viz_node (default: true); see Foxglove visualization.
• headless_mode -- Run without the Open3D GUI (gui_node); starts rqt instead (default: false). Forwarded to viewpoint_generation.launch.py. Use on hosts with no X display.
• compute_threads -- Forwarded to viewpoint_generation.launch.py (see above).
• launch_control -- Launch the ros2_control layer locally (default: true). Set false to run hard-real-time control on a dedicated RT host (bring it up there with inspection_cell_description/launch/inspection_cell_control.launch.py, which is standalone), while this machine runs only MoveIt + perception. The control launch exposes control_lock_memory / control_thread_priority / control_cpu_affinity for SCHED_FIFO priority, mlockall, and core pinning (pin to an isolcpus core on the RT host).
• admittance_config_file -- Admittance/orientation control config (default: admittance_control_coupled_pendulum.yaml). When cell is enabled, bringup includes inspection_control/launch/coupled_pendulum.launch.py, i.e. the Tier 3 coupled-pendulum orientation + admittance pair. To use the legacy point-mass pipeline instead, point this back at admittance_control.yaml and include admittance_control.launch.py.

Foxglove visualization

foxglove_viz_node (nodes/foxglove_viz_node.py, scene building in viewpoint_generation/foxglove_scene.py) publishes the plan for Foxglove's 3D panel. Like the Open3D GUI it stays out of the pipeline. It follows /viewpoint_generation's results.file, model.mesh.file, model.mesh.units and model.mesh.tessellation.* parameters through /parameter_events, and since every stage ends by setting results.file, it redraws after each one.

All topics are latched (transient local) under /viewpoint_generation/viz:

Topic	Type	Content
mesh	foxglove_msgs/SceneUpdate	one entity per region in the GUI's region colours, plus noise and unassigned faces; click a region for its metadata
points	sensor_msgs/PointCloud2	each region's sampled points, rgb = cluster colour, plus region and cluster (traversal position, -1 = unclustered) fields to colour by
blind_spots	sensor_msgs/PointCloud2	points no viewpoint can see (magenta)
viewpoints	foxglove_msgs/SceneUpdate	per viewpoint: camera marker coloured by imaging_mode (green photometric, yellow standard, red inaccessible) and a frustum whose far face is the imaged patch; metadata on click
fov	foxglove_msgs/SceneUpdate	FOV coverage cylinders (fov_diameter x dof) at each surface origin
path	foxglove_msgs/SceneUpdate	per region: viewpoints joined in traversal order, the optimised joint path, unreachable viewpoints
labels	foxglove_msgs/SceneUpdate	region numbers
selection, selection_points	SceneUpdate, PointCloud2	highlight of the selected region (wireframe), cluster points and viewpoint
summary	std_msgs/String	JSON: stage, counts, imaging modes, per-region stats and cluster ids, warnings

Input: select (std_msgs/String, {"region": id, "cluster": id}, -1 = none).

TF: object_frame -> viewpoint_generation/model is the part's placement from /tsdf_pose/pose, the same pose the planning scene uses, and identity until one arrives. So the plan shows up even when tsdf_pose is not running. viewpoint_generation/model -> viewpoint_generation/selected_viewpoint is the selected camera pose (+Z forward).

Parameters: results_file pins the node to one plan file instead of following the pipeline, which is useful for browsing old plans (ros2 param set /viewpoint_generation_viz results_file <plan.json>; '' to follow again). Also traversal_algorithm (which optimised order to show), labels, target_node, topic_prefix, frame_id, parent_frame and placement_topic.

A region's faces are triangle indices, so a STEP mesh is tessellated exactly as the pipeline tessellates it: through cad_loader.step_loader.load_step with the pipeline's model.mesh.tessellation.*. When a plan indexes faces the mesh does not have (it was made with a different tessellation), the node warns, and the warning appears in summary. Stored absolute paths that no longer exist are re-rooted from their last /catalog/ component onto $CATALOG_ROOT.

The Viewpoint Generation Foxglove panel (in the cell's Foxglove extension) shows the summary, runs the stages through the bridge's service whitelist, and steps through regions and clusters by publishing select.

JSON Results Format

Results are saved with the naming convention {stage}_{N}r_{M}c_{V}v_{timestamp}.json, e.g. projected_12r_75c_75v_20260814T162442Z.json. Zero counts are omitted, so a segmentation-only plan is segmented_12r_20260814T162427Z.json. The stage is one of segmented → clustered → projected → ordered, matching the pipeline step that produced the file. Naming, reading, and writing all live in plan_io.py, which both the core class and viewpoint_traversal_node use.

Alongside meshes, a plan carries three self-describing top-level keys:

key	meaning
schema_version	plan format version ("3.0")
plm_context	PLM identity, authored by the catalog: backend, part_id, part_number, revision, change_token, container, title, cell_id, generated_by; plus the producer's stage, generated_at, software_version. (2.0 files used eng_item_id/cestamp/collab_space; read_plan() maps them.)
summary	num_regions, num_clusters, num_viewpoints, and the mesh identity

These sit beside meshes, never wrapping it, so every consumer that reads json.load(...)['meshes'] is unaffected. viewpoint_traversal_node round-trips the whole dict, so an optimized plan inherits its part linkage automatically.

plan_io.infer_stage() derives the stage from content rather than trusting the filename: a region's order becomes an algorithm-keyed dict only after traversal optimization, which is what separates ordered from projected.

Each mesh holds a regions list; each region holds a clusters list. The "0", "1" keys shown below are list indices. A region's order is a list of cluster indices (identity order) until traversal optimization runs, at which point it becomes a dict keyed by TSP algorithm name (see below).

{
  "meshes": [
    {
      "file": "/path/to/mesh.stl",
      "units": "m",
      "pose": {
        "position": [0.0, 0.0, 0.0],
        "orientation_rpy": [0.0, 0.0, 0.0],
        "rotation_center": "origin"
      },
      "material": "unknown",
      "dimensions": "(LxWxH): 0.14 x 0.09 x 0.10 m",
      "surface_area": "Surface Area: 0.03 m^2",
      "regions": [
        {
          "faces": [8994, 199, 2379],
          "point_cloud": {
            "file": "/path/to/region_0_1500points.ply",
            "units": "m",
            "points": 1500
          },
          "clusters": [
            {
              "points": [1234, 5678],
              "candidate_axis": [0.0, 0.0, 1.0],
              "viewpoint": {
                "origin": [0.01, 0.02, 0.03],
                "position": [0.1, 0.2, 0.3],
                "direction": [0.0, 0.0, 1.0],
                "orientation": [0.0, 0.0, 0.0, 1.0],
                "imaging_mode": "photometric"
              }
            }
          ],
          "blind_spots": {
            "points": [17, 42, 88],
            "reason": "occluded"
          },
          "order": {
            "greedy": {
              "order": [0, 2, 1],
              "distance": 0.84,
              "joint_trajectory": {
                "total_time_s": 12.3,
                "total_joint_distance": 4.56,
                "cartesian_waypoints": [[0.1, 0.2, 0.3]],
                "unreachable": [1]
              }
            },
            "vrp_clustered": {
              "order": [0, 2, 1],
              "distance": 3.9,
              "configs": [2, 0, 1],
              "joint_trajectory": {
                "total_time_s": 10.8,
                "total_joint_distance": 4.10,
                "cartesian_waypoints": [[0.1, 0.2, 0.3]],
                "unreachable": []
              }
            },
            "LKH":    {"order": [0, 1, 2], "distance": 0.79}
          }
        }
      ],
      "order": [0, 1, 2],
      "noise_faces": [100, 200],
      "camera_config": {
        "fov_diameter": 0.03,
        "dof": 0.02,
        "focal_distance": 0.3
      }
    }
  ]
}

region.faces is a list of triangle indices into the mesh (meshes[i].file) — the exact set of faces region growth or PartField assigned to that region. region.point_cloud is that region's own submesh, Poisson-disk-sampled and cached to disk during segment_regions(); cluster points are point indices local to that region's own point cloud, not a shared/global one. noise_faces is a list of triangle indices no region claimed (always empty for PartField, since every face gets a part).

cluster.candidate_axis (only produced by fov_clustering.algorithm == 'greedy_cover'; absent for kmeans, which never validates occlusion during clustering) -- the owning candidate's occlusion-validated view axis (unit vector), i.e. a direction already proven, during fov_clustering(), to have unoccluded line of sight to every point in this cluster. project_viewpoints() reuses it as a guaranteed-good starting point for direction refinement (see viewpoint.imaging_mode below) rather than trusting an unvalidated mean-of-normals direction.

region.blind_spots (only produced by fov_clustering.algorithm == 'greedy_cover'; always {"points": []} for kmeans, which has no coverage predicate to fail) -- region-local point indices, in the same indexing space as cluster points, that no candidate viewpoint could cover, even after a Monte Carlo hemisphere search (rescue_search) tried every incidence angle in the outward hemisphere looking for any unoccluded viewing direction, not just the straight surface normal. These points are never assigned to any cluster, so blind_spots.points and the union of a region's clusters[*].points are disjoint. reason (present only when points is non-empty) is "occluded" when every affected point had at least one geometrically-valid candidate that occlusion alone ruled out, "geometric" when none did (e.g. a degenerate surface normal), or "mixed" when the region has both. A region can have blind_spots with zero clusters (the whole region is occluded). This is a genuine inspection coverage gap, not a clustering bug, and is also logged as a warning (region N: <count> points are unreachable (...)) when fov_clustering() runs. The GUI's View → Show Blind Spots toggle renders these points as markers (see below).

Viewpoint fields:

• origin -- Cluster centroid on the surface
• position -- Camera position (centroid projected along direction by focal_distance)
• direction -- Unit vector, the emitted camera axis. Not simply the mean of the cluster's point normals: when a raycasting scene and fov_normal_threshold are available, project_viewpoints() refines it via the same hemisphere search used for blind-spot rescue (starting from candidate_axis when present, so it can never be worse than what clustering already proved), since the raw mean direction is never itself guaranteed occlusion-free
• orientation -- Camera orientation as quaternion (xyzw)
• imaging_mode -- "photometric" if direction falls within fov_normal_threshold of the cluster's true mean surface normal (supports photometric-stereo normal-map reconstruction), "standard" if only an off-cone angle within standard_normal_threshold has unoccluded line of sight (still a usable image, just not photometric-quality), or "inaccessible" if no direction inside standard_normal_threshold is occlusion-clear (any remaining angle is too glancing to capture the surface). The GUI's Viewpoint Marker overlay colors accordingly (green/yellow/red; see below)

camera_config records the FOV geometry the results were generated with (fov_diameter, dof, focal_distance). Besides documenting the capture settings, the visualizer reads fov_diameter/dof from here to draw the FOV Cylinder viewpoint overlay (see above). Results files written before this field existed simply skip that overlay.

All regions/point_cloud/viewpoint coordinates in this file are expressed in the mesh's origin frame (model_frame); the part's physical placement is never baked in (it is a live TF -- see the model.pose.* note under Parameters). Older results files may carry a now-ignored pose field recording a baked-in model.pose.* transform; it is no longer read.

Traversal order:

• Mesh-level order -- Region visit order (a list of region indices).
• Region-level order -- Cluster visit order within that region. Out of FOV clustering this is a plain list (identity order). After optimize_traversal runs it becomes a dict keyed by TSP/VRP algorithm name, where each value is an object {"order": [...], "distance": ...} holding that algorithm's optimized list of cluster indices plus its path metrics. The key is the selected algorithm's name: TSP keys are greedy, 2opt, 3opt, ILS, LKH; VRP keys are vrp_greedy, vrp_2opt, vrp_3opt, vrp_ils, vrp_lkh, vrp_aco, vrp_hierarchical, vrp_clustered (i.e. the vrp_algorithm parameter value). distance units follow vrp_cost_mode — seconds in time mode (execution-time surrogate) or radians in joint mode. Multiple algorithms accumulate in the same dict across runs, so the different optimization results — and their metrics — are stored together in the results file.
• Region-level order.<algorithm>.configs -- Present for any VRP algorithm when the turntable sweep is enabled (vrp_n_turntable_samples > 0), which gives each viewpoint multiple candidate IK configurations. A list parallel to order: configs[k] is the index of the chosen IK configuration for cluster order[k], selected (via a config-selection DP over the whole tour) so joint-space travel is minimised and the arm never has to swing between far IK branches. The trajectory planner plans each segment to this exact configuration (joint-space goal) rather than re-resolving IK from the Cartesian pose. vrp_clustered additionally optimises the config jointly with region order and entry/exit; the other algorithms optimise the order first, then select configs for that fixed tour.
• Region-level order.<algorithm>.joint_trajectory -- moveit_py-computed arm motion cost for that algorithm's cluster order, added by viewpoint_traversal_node's optimize_traversal service whenever the planning component is available:
  • total_time_s / total_joint_distance -- Summed trajectory duration (seconds) and joint-space path length (radians) across all successfully planned segments in the region.
  • cartesian_waypoints -- Flattened list of eoat_camera_link positions (metres) along every successfully planned segment, used to draw the arm's actual path in the GUI (the "Cartesian Path" toggle).
  • unreachable -- Cluster indices (within this region) that a segment could not be planned to. The GUI highlights these viewpoints in the 3D view, and optimize_traversal logs them (region X:Y: viewpoint N unreachable ...) and reports per-region and whole-inspection totals in its summary output.
• Selected algorithm -- Which algorithm's path the visualizer and motion consumers follow when a region's order is a dict is not stored in the results file. It is the selected_traversal_algorithm parameter on the task_planning node (set via the GUI, e.g. by clicking an algorithm under a region's Paths in the tree view), so the same choice drives both the visualized path and the execution order. An empty value falls back to the first available algorithm.

STEP ingest

STEP files load natively, producing the same o3d.geometry.TriangleMesh the STL/OBJ path produces plus a sidecar of B-rep topology. _load_scaled_mesh() dispatches on the file extension, so set_mesh_file("part.stp", ...) is all that is required and every downstream stage is unchanged.

Units are not configurable for STEP. A STEP file declares its own length unit, and OpenCascade converts every file to its working unit (millimetres) on read, so the loader always returns metres and ignores the units argument. set_mesh_file() records m for a STEP mesh, so the results JSON does too. Code that scales what mesh_utils.read_mesh_file() returns must use mesh_utils.mesh_file_units(path, units), which is m for STEP. The visualizer does this, so older results files that recorded mm for a STEP mesh still display at the right size.

Before this was fixed (2026-10-16) the loader treated its units argument as the file's unit, so a STEP was only sized correctly when loaded as mm (the catalog's default), and 1000× too large otherwise. Tessellation deflection was applied in millimetres, so the default 0.0005 meant 0.5 µm rather than 0.5 mm.

step_loader.load_step() returns a StepLoadResult:

Field	Meaning
mesh	Tessellated Open3D mesh, scaled to metres, normals computed
brep_faces	One BRepFace per face: surface type, triangle indices, area, exact centroid and normal
face_adjacency	face_id -> {face_id} from shared B-rep edges (exact CAD topology)
tri_to_brep	Triangle index -> B-rep face id
source_file, units	Provenance; units is always m

Surface types are classified from the analytical surface: plane, cylinder, cone, sphere, torus, bezier, bspline, revolution, extrusion, offset, other.

Vertex merging is load-bearing. OpenCascade tessellates each B-rep face independently with its own vertex pool, so adjacent faces produce coincident but distinct vertices at shared edges. RegionGrowing derives face adjacency from shared vertex indices, so without welding it sees a mesh that falls apart at every B-rep boundary and can never grow a region across one. The loader welds with merge_close_vertices(1e-8) and then rebuilds tri_to_brep and every BRepFace.triangle_indices from the surviving triangles, because welding renumbers (and can drop degenerate) triangles.

Tessellation quality is exposed as ROS parameters under model.mesh.tessellation. (linear_deflection in metres, or a fraction of edge size when relative is set; angular_deflection in radians; relative) and applies on the next model load.

The loader works with cadquery-ocp 7.x and 8.x. OCP 8 moved the TopTools collection classes to OCP.collections and renamed TopoDS.Face_s to TopoDS.Face; _import_occ() falls back to the OCP 8 names.

Everything that reads a model file must go through mesh_utils.read_mesh_file(), which dispatches on the extension and returns the geometry in the file's own units (metres for STEP; see above). o3d.io.read_triangle_mesh() cannot parse STEP and fails silently by returning an empty mesh, so any code path calling it directly renders nothing at all for a STEP-loaded part — which is exactly how the visualizer first went blank. Tessellated STEP meshes are cached by (path, mtime, size) and handed out as copies, since callers scale what they receive.

B-rep segmentation parameters

Declared under regions.brep.:

Parameter	Default	Meaning
min_face_area	1e-6	Faces below this area (m²) merge into their largest neighbour; 0 disables
merge_same_type	false	Union adjacent faces sharing an analytical surface type
max_regions	0	Cap on emitted regions (0 = one per B-rep face)

Parts catalog (plm_bridge)

The PLM parts catalog moved to the plm_bridge repository (packages plm_bridge, plm_bridge_ros, plm_interfaces, cad_loader); its README.md and docs/architecture.md document it. What this package does with it:

• ViewpointGenerationNode subscribes to /catalog/part_selected (plm_interfaces/PartSelected), loads geometry_path, and loads the plan when its status is CURRENT, DOWNLOADED, UPDATED_FROM_REMOTE or SELECTED (an operator's explicit choice, loaded with a warning). A STALE plan is not loaded.
• Plans for a selected part are written to the selection's plan_dir, and their plm_context is the selection's plm_context_json plus the stage, generation time and software version (plan_io).
• After every stage it calls catalog/record_plan, describing the plan itself with plan_io.describe_plan(). The bridge never opens plan files.
• It serves viewpoint_generation/describe_plans (DescribePlans), which the bridge's picker uses to adopt plan files it finds on disk. From the command line, the same description is python3 -m viewpoint_generation.plan_io describe FILE..., which is what PLAN_DESCRIBE_CMD should be for the plm-bridge --adopt-plans CLI.

bringup.launch.py includes plm_bridge_ros/launch/plm_bridge.launch.py when catalog:=true and that package is installed, and logs a message otherwise.

Region provenance in the results JSON

When the mesh came from STEP, each region additionally carries:

• surface_type -- the analytical surface type, or mixed when a region spans several (region_growth/partfield regions get this too, derived from which B-rep faces their triangles came from)
• brep_face_ids -- the B-rep faces the region covers

and the mesh entry carries source_format: "STEP".