diff --git a/docs/source/policy_deployment/03_compass_with_NuRec/compass_navigation_policy_with_NuRec.rst b/docs/source/policy_deployment/03_compass_with_NuRec/compass_navigation_policy_with_NuRec.rst index 0f11d31bc7fd..0dac9decaab6 100644 --- a/docs/source/policy_deployment/03_compass_with_NuRec/compass_navigation_policy_with_NuRec.rst +++ b/docs/source/policy_deployment/03_compass_with_NuRec/compass_navigation_policy_with_NuRec.rst @@ -1,27 +1,63 @@ -Training & Deploying COMPASS Navigation Policy with Real2Sim NuRec -==================================================================== +Training and Deploying COMPASS Policies with NuRec Assets +========================================================= -COMPASS (Cross-embodiment Mobility Policy via Residual RL and Skill Synthesis) trains and -deploys cross-embodiment navigation policies on NuRec Real2Sim assets — photoreal -Gaussian-splat reconstructions of real spaces — in Isaac Lab. +`COMPASS `_ (Cross-embodiment Mobility Policy via Residual RL and +Skill Synthesis) trains navigation policies that can transfer across robot embodiments. Isaac Lab supports +COMPASS training with NuRec scene assets, so policies can be developed in reconstructed real-world +environments instead of relying only on hand-authored simulation scenes. -This tutorial now lives in the **COMPASS handbook**, next to the code it documents, where it -stays in sync with the NuRec support branch. +This page gives a short overview of the supported NuRec assets and example COMPASS training clips. For +installation, asset registration, training, evaluation, export, and ROS 2 deployment instructions, see the +`COMPASS handbook `_. -Where to find it ----------------- +NuRec Assets for COMPASS +------------------------ + +NuRec assets matter for COMPASS because they allow navigation policies to train in simulation on realistic +geometry, visual appearance, and obstacle layouts. This, in turn, should reduce the sim-to-real gap between +training scenes and real operating environments and improve zero-shot transfer. They also make it possible +to evaluate policies in reconstructed simulation scenes that better reflect target deployment spaces. + +Some sample assets compatible with COMPASS training are available in the +`PhysicalAI-Robotics-NuRec dataset on Hugging Face `_. + +Real2Sim Assets +~~~~~~~~~~~~~~~ + +`NVIDIA Neural Reconstruction (NuRec) `_ converts real-world +sensor data into digital simulation-ready assets for training and testing Physical AI agents. The Real2Sim +workflow converts stereo RGB captures into Isaac Sim-ready assets with aligned geometry, collision meshes, +and simulation-ready scene augmentation. The PhysicalAI-Robotics-NuRec dataset includes several robotics +scenes produced with this workflow; two examples are: + +- `Galileo `_ — + Embodiment: `Nova Carter `_, + Sensor Rig: 4 x `Stereo Camera `_ -- **Repository:** `COMPASS on GitHub `_ -- **Branch:** ``samc/support_nurec_assets_isaaclab_3.0`` (NuRec Real2Sim support) -- **Guide:** `docs/handbook/workflows/nurec_real2sim.md `_ in the COMPASS repository. +- `Living Room `_ — + Embodiment: Hand-Held, + Sensor Rig: 1 x `Stereo Camera `_ + +XGRIDS Assets +~~~~~~~~~~~~~ + +COMPASS also supports compatible assets reconstructed with XGRIDS. These assets are post-processed into the +NuRec format used by the COMPASS workflow, including mesh alignment and augmentation with simulation-ready +assets available in Isaac Sim. + +- `Wormhole `_ — + Embodiment: Hand-Held, + Sensor Rig: `XGRIDS PortalCam `_ [2 x fish-eye cameras, 2 x front cameras, 1 x LiDAR] + +COMPASS Workflow +---------------- -.. code-block:: bash +The complete COMPASS workflow is maintained in the COMPASS repository: - git clone https://github.com/NVlabs/COMPASS.git - cd COMPASS - git checkout samc/support_nurec_assets_isaaclab_3.0 - # Open docs/handbook/workflows/nurec_real2sim.md +- **Repository:** `NVlabs/COMPASS `_ +- **Branch:** ``real2sim/isaaclab_3.0`` +- **Guide:** `NuRec asset workflow `_ -The guide covers Isaac Sim / Isaac Lab installation, COMPASS setup, downloading and -registering NuRec Real2Sim scenes, training a residual RL specialist, evaluation, -ONNX / TensorRT export, and ROS2 / sim-to-real deployment. +The guide covers the COMPASS code path for training with NuRec assets generated by the Real2Sim workflow, +including Isaac Sim and Isaac Lab setup, COMPASS installation, asset download and registration, residual RL +specialist training, evaluation, ONNX and TensorRT export, and ROS 2 sim-to-real deployment. diff --git a/pyproject.toml b/pyproject.toml index 84ebbcc3269a..a0797f74d3eb 100644 --- a/pyproject.toml +++ b/pyproject.toml @@ -188,7 +188,7 @@ reportPrivateUsage = "warning" skip = '*.usd,*.usda,*.usdz,*.svg,*.png,_isaac_sim*,*.bib,*.css,*/_build' quiet-level = 0 # the world list should always have words in lower case -ignore-words-list = "haa,slq,collapsable,buss,reacher,thirdparty" +ignore-words-list = "haa,slq,collapsable,buss,reacher,thirdparty,segway" [tool.pytest.ini_options]