Mesh-based Non-collision Constraints

[zhx06]

Summary

Add mesh-based non-collision constraints via sphere-to-SDF

Detailed description

• Introduces CollisionMode.MESH as an alternative to AABB for no-overlap constraints, using greedy sphere decomposition + differentiable Warp SDF queries against actual collision geometry.
• Solver falls back to AABB for pairs where either object lacks a mesh. Validator mirrors this.
• CLI: --collision_mode mesh enables the new path.

Core files

• relations/warp_sdf_kernels.py — differentiable SDF queries on Warp meshes
• relations/warp_mesh_manager.py — sphere decomposition and mesh caching
• relations/relation_solver.py — vectorized mesh collision loss during optimization
• relations/object_placer.py — mesh collision validation at placement time
• relations/relation_loss_strategies.py — per-pair mesh loss for the strategy API

[isaaclab-review-bot]

🤖 Isaac Lab Review Bot — PR #771

Mesh-based Non-collision Constraints

Summary

This PR adds sphere-to-SDF mesh collision support as an alternative to AABB overlap detection. The architecture is clean — CollisionMode.MESH integrates well into the existing NoCollisionLossStrategy dispatch, and the greedy sphere decomposition + Warp SDF kernel approach is sound. The test suite is thorough (542 lines!) with good coverage of dispatch routing, gradient flow, and integration.

Findings

#	Severity	Finding
1	🟡 Warning	Validator creates fresh WarpMeshManager per call — cache never reused
2	🟡 Warning	Scale applied post-transform in extract_trimesh_from_usd may be incorrect for nested prims
3	🔵 Suggestion	object_base.py abstract method has no explicit return None
4	🔵 Suggestion	Sentinel warning pattern on function object is not thread-safe
5	🔵 Suggestion	Consider documenting the rotated-anchor limitation more prominently

See inline comments for details.

---

Update (5e86ed0a): Reviewed incremental changes since 655ac73.

Addressed Findings

• ✅ Finding #1 resolved — _get_cpu_mesh_manager() now lazily creates and caches the WarpMeshManager on the instance, eliminating redundant allocations per validation call. Good fix.
• ✅ Finding #2 resolved — Removed erroneous .T transpose on ComputeLocalToWorldTransform in usd_helpers.py. USD returns row-major matrices; the transpose was producing incorrect vertex transforms for nested prims.

Other Changes

• Validation logic refactored (_validate_placement): Mesh mode now skips AABB validation entirely (else branch). Previously both checks ran in mesh mode — the AABB check was redundant and could produce false negatives for non-convex shapes. Clean improvement.
• Test suite trimmed: Removed test_sphere_count_respects_budget, test_cache_key_differs_for_different_meshes, test_dispatch_falls_back_when_obj_is_none, and test_mesh_zero_loss_separated_cylinders. These removals look intentional (simplified scope / covered elsewhere), though removing cache-key differentiation test reduces regression coverage on the caching layer.

Remaining Observations

• Findings #3–#5 from original review remain unaddressed (low priority, suggestions only).
• The new _get_cpu_mesh_manager uses hasattr check — works fine but Optional attribute initialized in __init__ would be more explicit.

Overall: Good incremental improvement. The two main warnings from the initial review are resolved. No new concerns.

---

Update (729d892c): Reviewed incremental changes since 5e86ed0a.

Changes in this push (2 files)

1. relation_loss_strategies.py — Added parent_pos_resolved.expand(batch_size, -1) before the per-batch loop. This fixes a shape mismatch when parent_pos_resolved is not already batch-expanded (e.g., single parent broadcast to multiple children). Correct fix.

2. warp_mesh_manager.py — Wrapped getattr(obj, "scale", ...) in tuple() for cache key computation. This prevents unhashable types (e.g., numpy arrays or torch tensors returned by .scale) from breaking the dict lookup. Necessary bugfix.

Assessment

Both changes are small, targeted bugfixes. No new concerns introduced. All previous suggestions (#3–#5) remain low-priority and unaddressed.

[greptile-apps]

Greptile Summary

This PR introduces mesh-based non-collision constraints for the placement solver via sphere-to-SDF queries (Warp). It adds CollisionMode.MESH to NoCollisionLossStrategy and RelationSolver, a WarpMeshManager that caches BVH meshes and greedy sphere decompositions, new Warp autograd kernels, and USD mesh extraction. A vectorized multi-mesh SDF path replaces the per-pair Python loop during optimization. All of the issues flagged in the previous review (transpose in USD extraction, stale-manager-per-validation-call, sentinel flag lifetime, batch IndexError for anchors, list-scale hash error, missing child-orientation rotation) appear to have been addressed.

• Adds CollisionMode.MESH to the solver and placer, with AABB as the default; MESH mode uses sphere-to-SDF via Warp BVH for tighter packing, falling back to AABB per pair when a mesh is unavailable.
• Introduces WarpMeshManager with content-hash caching, greedy_sphere_decomposition, and extract_trimesh_from_usd for extracting collision meshes from USD assets.
• Adds a vectorized multi-mesh SDF kernel path in RelationSolver._compute_no_overlap_loss_mesh with a pair-level AABB broadphase gate to skip clearly-separated pairs.

Confidence Score: 3/5

The core MESH-mode solver path can silently miss collision penalties for yaw-rotated anchors, causing the optimizer to converge to positions inside those anchors before the validation pass rejects them.

The vectorized mesh cache stores the anchor's un-rotated local bounding box as the broadphase gate. For any anchor with a non-zero yaw (e.g., a table rotated 45-90 degrees), the unrotated AABB underestimates the anchor's extent on one axis, so the broadphase can declare separated for a child that is actually inside the true footprint. The SDF check is skipped and the pair contributes zero loss; the optimizer has no gradient signal to push the child away. The validation path correctly inflates the anchor bbox, so placed objects are eventually rejected, but without a working loss signal the solver wastes attempts and may consistently fail to find valid placements.

isaaclab_arena/relations/relation_solver.py - specifically _build_vectorized_cache where pair_p_bbox_min/max is stored for anchor objects without applying the anchor's yaw rotation.

Important Files Changed

Filename	Overview
isaaclab_arena/relations/relation_solver.py	Adds vectorized mesh collision cache (_build_vectorized_cache, _compute_no_overlap_loss_mesh) and AABB fallback for mesh-less pairs. AABB broadphase uses un-rotated anchor bbox for yaw-rotated anchors, causing missed collision penalties during optimization.
isaaclab_arena/relations/relation_loss_strategies.py	Adds MESH dispatch to NoCollisionLossStrategy; _compute_mesh_loss now correctly applies net_yaw rotation and uses expand() for batch-size>1 anchor case. Previously-reported bugs addressed.
isaaclab_arena/relations/warp_sdf_kernels.py	New file: single-mesh and multi-mesh SDF autograd kernels with per-solve sentinel warning (reset_sdf_sentinel_warning). Logic looks correct.
isaaclab_arena/relations/warp_mesh_manager.py	New file: WarpMeshManager with content-hash and (usd_path, scale-tuple) caching; greedy sphere decomposition with fallbacks for degenerate meshes. Scale stored as tuple to avoid unhashable-list issue.
isaaclab_arena/relations/object_placer.py	Adds _validate_no_overlap_mesh with cached CPU WarpMeshManager, _centers_in_target_frame, and _effective_yaw; correctly rotates anchor bboxes in the validation path. MESH mode now bypasses AABB validation, resolving the prior AABB-gate issue.
isaaclab_arena/utils/usd_helpers.py	Adds extract_trimesh_from_usd: applies ComputeLocalToWorldTransform via row-vector convention (verts_h @ world_tf, no transpose) then applies scale in world space. Correct for standalone USD assets centered at origin.
isaaclab_arena/assets/object_base.py	Adds abstract get_collision_mesh() returning None by default; implicit return is functionally correct but lacks an explicit return None.
isaaclab_arena/assets/object.py	Adds get_collision_mesh() with lazy USD extraction; deepcopy excludes trimesh to avoid C-pointer pickling issues.
isaaclab_arena/tests/test_mesh_collision.py	New test file: covers sphere decomposition coverage, manager caching, dispatch (mesh vs AABB fallback), end-to-end solver separation, and multi-object scenarios.

Sequence Diagram

sequenceDiagram
    participant OP as ObjectPlacer
    participant RS as RelationSolver
    participant WMM as WarpMeshManager
    participant SDK as warp_sdf_kernels

    OP->>RS: solve(objects, initial_positions, env_bboxes, orientations)
    RS->>RS: _prepare_mesh_collision_cache(state, on_pairs)
    RS->>WMM: "get_warp_mesh(parent_mesh, obj=anchor)"
    RS->>WMM: "get_query_spheres(child_mesh, obj=child)"
    RS->>RS: _build_vectorized_cache (fwd + rev)
    Note over RS: Cache stores pair_p_bbox_min/max unrotated for anchors

    loop optimization iterations
        RS->>RS: _compute_no_overlap_loss_mesh(state)
        RS->>RS: AABB broadphase using pair_p_bbox_min/max
        Note over RS: Rotated-anchor broadphase may false-separate
        RS->>SDK: multi_mesh_sdf(active_centers, mesh_id_array)
        SDK-->>RS: sdf_values
        RS->>RS: "penetration = relu(radii + clearance - sdf)"
        RS->>RS: segment-reduce to pair_mean to total_loss
    end

    RS-->>OP: solved positions
    OP->>OP: _validate_placement to _validate_no_overlap_mesh
    OP->>WMM: get_query_spheres / get_warp_mesh (CPU, cached)
    OP->>OP: _centers_in_target_frame applies src_yaw and tgt_yaw
    OP->>SDK: mesh_sdf(centers_in_b, warp_a)
    SDK-->>OP: sdf_values
    OP-->>OP: overlap check to accept or reject placement

Loading

_{Reviews (6): Last reviewed commit: "supoort for rotation, verification of al..." | Re-trigger Greptile}