Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Original file line number Diff line number Diff line change
@@ -1,27 +1,63 @@
Training & Deploying COMPASS Navigation Policy with Real2Sim NuRec
====================================================================
Training and Deploying COMPASS Policies with NuRec Assets
=========================================================

COMPASS (Cross-embodiment Mobility Policy via Residual RL and Skill Synthesis) trains and
deploys cross-embodiment navigation policies on NuRec Real2Sim assets — photoreal
Gaussian-splat reconstructions of real spaces — in Isaac Lab.
`COMPASS <https://arxiv.org/abs/2502.16372>`_ (Cross-embodiment Mobility Policy via Residual RL and
Skill Synthesis) trains navigation policies that can transfer across robot embodiments. Isaac Lab supports
COMPASS training with NuRec scene assets, so policies can be developed in reconstructed real-world
environments instead of relying only on hand-authored simulation scenes.

This tutorial now lives in the **COMPASS handbook**, next to the code it documents, where it
stays in sync with the NuRec support branch.
This page gives a short overview of the supported NuRec assets and example COMPASS training clips. For
installation, asset registration, training, evaluation, export, and ROS 2 deployment instructions, see the
`COMPASS handbook <https://github.com/NVlabs/COMPASS/blob/real2sim/isaaclab_3.0/docs/handbook/workflows/nurec_real2sim.md>`_.

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔴 CI Blocker — Broken Link (404)

The Check for Broken Links CI job fails because this URL returns HTTP 404:

https://github.com/NVlabs/COMPASS/blob/real2sim/isaaclab_3.0/docs/handbook/workflows/nurec_real2sim.md

Possible causes:

  • The branch real2sim/isaaclab_3.0 may not exist yet on the public repo (private or not yet pushed)
  • The path inside the repo may have changed

Please verify the branch/path exist on the public NVlabs/COMPASS repo, or exclude this URL from the link checker config until the branch is published. This same URL appears again on line 87.


Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔴 Critical: Broken link (404)

This URL returns 404 — the branch real2sim/isaaclab_3.0 does not exist in the COMPASS repository.

The existing branch is samc/support_nurec_assets_isaaclab_3.0. Either:

  1. Update this link to the correct branch, or
  2. Ensure the new branch is created before merging
Suggested change
`COMPASS handbook <https://github.com/NVlabs/COMPASS/blob/samc/support_nurec_assets_isaaclab_3.0/docs/handbook/workflows/nurec_real2sim.md>`_.

Where to find it
----------------
NuRec Assets for COMPASS
------------------------

NuRec assets matter for COMPASS because they allow navigation policies to train in simulation on realistic
geometry, visual appearance, and obstacle layouts. This, in turn, should reduce the sim-to-real gap between
training scenes and real operating environments and improve zero-shot transfer. They also make it possible
to evaluate policies in reconstructed simulation scenes that better reflect target deployment spaces.

Some sample assets compatible with COMPASS training are available in the
`PhysicalAI-Robotics-NuRec dataset on Hugging Face <https://huggingface.co/datasets/nvidia/PhysicalAI-Robotics-NuRec/tree/v0.2>`_.

Real2Sim Assets
~~~~~~~~~~~~~~~

`NVIDIA Neural Reconstruction (NuRec) <https://docs.nvidia.com/nurec/index.html>`_ converts real-world
sensor data into digital simulation-ready assets for training and testing Physical AI agents. The Real2Sim
workflow converts stereo RGB captures into Isaac Sim-ready assets with aligned geometry, collision meshes,
and simulation-ready scene augmentation. The PhysicalAI-Robotics-NuRec dataset includes several robotics
scenes produced with this workflow; two examples are:

- `Galileo <https://huggingface.co/datasets/nvidia/PhysicalAI-Robotics-NuRec/tree/v0.2/nova_carter-galileo>`_ —
Embodiment: `Nova Carter <https://robotics.segway.com/nova-carter/>`_,
Sensor Rig: 4 x `Stereo Camera <https://leopardimaging.com/wp-content/uploads/2024/07/LI-AR0234CS-STEREO-GMSL2-30_Datasheet_V1.8.pdf>`_

- **Repository:** `COMPASS on GitHub <https://github.com/NVlabs/COMPASS>`_
- **Branch:** ``samc/support_nurec_assets_isaaclab_3.0`` (NuRec Real2Sim support)
- **Guide:** `docs/handbook/workflows/nurec_real2sim.md <https://github.com/NVlabs/COMPASS/blob/samc/support_nurec_assets_isaaclab_3.0/docs/handbook/workflows/nurec_real2sim.md>`_ in the COMPASS repository.
- `Living Room <https://huggingface.co/datasets/nvidia/PhysicalAI-Robotics-NuRec/tree/v0.2/hand_hold-endeavor-livingroom>`_ —
Embodiment: Hand-Held,
Sensor Rig: 1 x `Stereo Camera <https://leopardimaging.com/wp-content/uploads/2024/07/LI-AR0234CS-STEREO-GMSL2-30_Datasheet_V1.8.pdf>`_

XGRIDS Assets
~~~~~~~~~~~~~

COMPASS also supports compatible assets reconstructed with XGRIDS. These assets are post-processed into the
NuRec format used by the COMPASS workflow, including mesh alignment and augmentation with simulation-ready
assets available in Isaac Sim.

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔴 Critical: Broken link (404)

Same issue as line 12 — this link also needs to be updated to the correct branch.

Suggested change
- **Guide:** `NuRec asset workflow <https://github.com/NVlabs/COMPASS/blob/samc/support_nurec_assets_isaaclab_3.0/docs/handbook/workflows/nurec_real2sim.md>`_

- `Wormhole <https://huggingface.co/datasets/nvidia/PhysicalAI-Robotics-NuRec/tree/v0.2/xgrid-wormhole>`_ —
Embodiment: Hand-Held,
Sensor Rig: `XGRIDS PortalCam <https://www.xgrids.com/intl/portalcamp>`_ [2 x fish-eye cameras, 2 x front cameras, 1 x LiDAR]

COMPASS Workflow
----------------

.. code-block:: bash
The complete COMPASS workflow is maintained in the COMPASS repository:

git clone https://github.com/NVlabs/COMPASS.git
cd COMPASS
git checkout samc/support_nurec_assets_isaaclab_3.0
# Open docs/handbook/workflows/nurec_real2sim.md
- **Repository:** `NVlabs/COMPASS <https://github.com/NVlabs/COMPASS>`_
- **Branch:** ``real2sim/isaaclab_3.0``
- **Guide:** `NuRec asset workflow <https://github.com/NVlabs/COMPASS/blob/real2sim/isaaclab_3.0/docs/handbook/workflows/nurec_real2sim.md>`_

The guide covers Isaac Sim / Isaac Lab installation, COMPASS setup, downloading and
registering NuRec Real2Sim scenes, training a residual RL specialist, evaluation,
ONNX / TensorRT export, and ROS2 / sim-to-real deployment.
The guide covers the COMPASS code path for training with NuRec assets generated by the Real2Sim workflow,
including Isaac Sim and Isaac Lab setup, COMPASS installation, asset download and registration, residual RL
specialist training, evaluation, ONNX and TensorRT export, and ROS 2 sim-to-real deployment.
2 changes: 1 addition & 1 deletion pyproject.toml
Original file line number Diff line number Diff line change
Expand Up @@ -188,7 +188,7 @@ reportPrivateUsage = "warning"
skip = '*.usd,*.usda,*.usdz,*.svg,*.png,_isaac_sim*,*.bib,*.css,*/_build'
quiet-level = 0
# the world list should always have words in lower case
ignore-words-list = "haa,slq,collapsable,buss,reacher,thirdparty"
ignore-words-list = "haa,slq,collapsable,buss,reacher,thirdparty,segway"


[tool.pytest.ini_options]
Expand Down
Loading