How to calibrate the controller-to-wrist transform for a Quest 3 controller mounted on a Manus glove?

[xuanlinh7102-ctrl]

Title: How to calibrate Meta Quest 3 controller mounted on Manus glove to wrist frame?

Hi Isaac Teleop team,

I am trying to use a Meta Quest 3 controller together with a Manus glove. My setup is that the Quest 3 controller is physically mounted on the Manus glove, and I want to use the controller pose to drive the robot wrist/end-effector pose.

However, I am not sure how to calibrate the transform between:

• the Meta Quest 3 controller tracking frame
• the human wrist / Manus glove wrist frame
• the robot end-effector control frame

From reading the code, it looks like some offsets are applied using a hard-coded transform matrix / hard-coded rotation and translation offsets. I understand the idea of applying an offset, but I do not know how that matrix should be generated for my own hardware setup.

Could you provide guidance on the recommended calibration procedure?

Specifically:

1. How should I estimate the fixed transform from the mounted Quest controller frame to the wrist frame?
2. Is there an existing calibration script or workflow in Isaac Teleop for this?
3. Should the offset be defined in the controller local frame, the OpenXR/world frame, or the robot target frame?
4. For Se3RetargeterConfig, how should I correctly set target_offset_roll, target_offset_pitch, target_offset_yaw, and target_offset_x/y/z for a custom controller-on-glove mount?
5. Is the intended workflow to manually tune these values, or to record a reference pose and compute the transform automatically?

My goal is to use Manus glove data for hand/finger retargeting while using the mounted Quest controller pose as a stable wrist pose input for robot arm teleoperation.

Any explanation, example, or recommended calibration method would be very helpful.

Thanks!

[jiwenc-nv]

Good question!

@sgrizan-nv is our in house experts on this. he can provide more details. Might worth adding a doc page for this.