Nvblox Minimal Example

This package demonstrates the minimal setup required to:

1. Receive RGB + Depth data from Isaac Sim
2. Process it through Nvblox to create a 3D voxel map
3. Visualize the result in RViz2

Architecture

┌─────────────────────────────────────────────────────────────────┐
│                    ISAAC SIM (Terminal 1)                       │
│                                                                 │
│  Publishes:                                                     │
│    /front_stereo_camera/left/image_raw     (RGB Image)         │
│    /front_stereo_camera/left/camera_info   (RGB Intrinsics)    │
│    /front_stereo_camera/depth/image_raw    (Depth Image)       │
│    /front_stereo_camera/depth/camera_info  (Depth Intrinsics)  │
│    /tf, /tf_static                         (Transforms)        │
└───────────────────────────┬─────────────────────────────────────┘
                            │
                            │ ROS_DOMAIN_ID=65
                            ▼
┌─────────────────────────────────────────────────────────────────┐
│              DOCKER CONTAINER (Terminal 2)                      │
│                                                                 │
│  ┌─────────────────────────────────────────────────────────┐   │
│  │                   nvblox_node                           │   │
│  │                                                         │   │
│  │  Subscribes:                                            │   │
│  │    /camera_0/color/image  ← remapped from Isaac Sim     │   │
│  │    /camera_0/depth/image  ← remapped from Isaac Sim     │   │
│  │    /tf                                                  │   │
│  │                                                         │   │
│  │  Internal Processing:                                   │   │
│  │    1. TSDF Integration (GPU-accelerated)               │   │
│  │    2. Color Fusion                                      │   │
│  │    3. Voxel Layer Publishing                           │   │
│  │                                                         │   │
│  │  Publishes:                                             │   │
│  │    /nvblox_node/color_layer      (3D Voxels)           │   │
│  │    /nvblox_node/workspace_bounds (Bounding Box)        │   │
│  └─────────────────────────────────────────────────────────┘   │
│                            │                                    │
│                            ▼                                    │
│  ┌─────────────────────────────────────────────────────────┐   │
│  │                      RViz2                              │   │
│  │                                                         │   │
│  │  Displays:                                              │   │
│  │    - NvbloxVoxelBlockLayer (3D reconstruction)         │   │
│  │    - Workspace Bounds (red bounding box)               │   │
│  │    - RGB/Depth images                                  │   │
│  │    - TF frames                                         │   │
│  └─────────────────────────────────────────────────────────┘   │
└─────────────────────────────────────────────────────────────────┘

Prerequisites

• Isaac Sim installed with ROS2 bridge
• Isaac ROS Docker container with nvblox_ros package
• NVIDIA GPU with CUDA support

Usage

Terminal 1: Isaac Sim (Host Machine)

# Set environment variables
export ISAAC_ROS_WS=$HOME/workspaces/isaac_ros-dev
source /opt/ros/humble/setup.bash
export ROS_DOMAIN_ID=65
export FASTRTPS_DEFAULT_PROFILES_FILE=/usr/local/share/middleware_profiles/rtps_udp_profile.xml

# Launch Isaac Sim, open your USD file, and press "Play"
# Ensure your USD scene publishes the required camera topics

Terminal 2: Docker Container

# Enter the Docker container
cd ${ISAAC_ROS_WS}/src/isaac_ros_common && \
    ./scripts/run_dev.sh -i cuda12.8.ros2_humble

# Inside Docker, set environment variables
export ISAAC_ROS_WS=/workspaces/isaac_ros-dev
export ROS_DOMAIN_ID=65
export FASTRTPS_DEFAULT_PROFILES_FILE=/usr/local/share/middleware_profiles/rtps_udp_profile.xml

# Build the package (first time only)
cd /workspaces/isaac_ros-dev
colcon build --packages-select nvblox_minimal_example
source install/setup.bash

# Verify Isaac Sim topics are visible
ros2 topic list | grep front_stereo

# Launch the minimal example
ros2 launch nvblox_minimal_example nvblox_minimal.launch.py

Launch Arguments

Argument	Default	Description
use_sim_time	true	Use simulation time from Isaac Sim
voxel_size	0.02	Voxel size in meters (smaller = more detail)
global_frame	base_link	Reference frame for the 3D map

Example with custom arguments:

ros2 launch nvblox_minimal_example nvblox_minimal.launch.py \
    voxel_size:=0.01 \
    global_frame:=world

Required Isaac Sim Topics

Your Isaac Sim USD scene must publish these topics:

Topic	Message Type	Description
/front_stereo_camera/left/image_raw	sensor_msgs/Image	RGB color image
/front_stereo_camera/left/camera_info	sensor_msgs/CameraInfo	RGB camera intrinsics
/front_stereo_camera/depth/image_raw	sensor_msgs/Image	Depth image (32FC1 or 16UC1)
/front_stereo_camera/depth/camera_info	sensor_msgs/CameraInfo	Depth camera intrinsics
/tf or /tf_static	tf2_msgs	Transform from camera to base_link

Nvblox Output Topics

Topic	Message Type	Description
/nvblox_node/color_layer	nvblox_msgs/VoxelBlockLayer	3D colored voxel map
/nvblox_node/workspace_bounds	visualization_msgs/Marker	Workspace bounding box
/nvblox_node/shapes_to_clear	visualization_msgs/MarkerArray	Regions being cleared

Nvblox Services

Service	Type	Description
/nvblox_node/get_esdf_and_gradient	nvblox_msgs/EsdfAndGradients	Get ESDF for motion planning
/nvblox_node/save_ply	nvblox_msgs/FilePath	Save mesh to PLY file

Troubleshooting

No voxels appearing in RViz

1. Check that Isaac Sim topics are being published:

   ros2 topic hz /front_stereo_camera/left/image_raw
   ros2 topic hz /front_stereo_camera/depth/image_raw

2. Check that TF is being published:

   ros2 run tf2_ros tf2_echo base_link front_stereo_camera_left

3. Check nvblox_node logs for errors:

   ros2 topic echo /rosout --filter "msg.name=='nvblox_node'"

ROS_DOMAIN_ID mismatch

Ensure all terminals (Isaac Sim and Docker) use the same ROS_DOMAIN_ID:

echo $ROS_DOMAIN_ID  # Should be 65

Camera frame not found

If TF transform errors occur, you may need to add a static transform publisher. Uncomment the static_tf node in the launch file and adjust the transform values.

Files

nvblox_minimal_example/
├── CMakeLists.txt
├── package.xml
├── README.md
├── launch/
│   └── nvblox_minimal.launch.py    # Main launch file
├── rviz/
│   └── nvblox_minimal.rviz         # RViz configuration
└── config/
    └── nvblox_params.yaml          # Parameter reference

Comparison with Full Pipeline

This minimal example differs from isaac_sim_workflows.launch.py:

Feature	Minimal Example	Full Pipeline
Robot Segmentation	No	Yes (cuMotion)
Motion Planning	No	Yes (cuMotion + MoveIt)
Object Detection	No	Yes (FoundationPose)
Pick & Place Actions	No	Yes
Depth Source	Raw from Isaac Sim	Segmented (robot masked)

To use robot segmentation (removes robot from depth image), change the depth remapping:

# In launch file, change:
('/camera_0/depth/image', '/front_stereo_camera/depth/image_raw'),
# To:
('/camera_0/depth/image', '/cumotion/camera_1/world_depth'),