Add Manus glove haptic feedback via XR_NVX1 push-tensor#568
Conversation
|
Important Review skippedAuto incremental reviews are disabled on this repository. Please check the settings in the CodeRabbit UI or the ⚙️ Run configurationConfiguration used: Path: .coderabbit.yaml Review profile: CHILL Plan: Enterprise Run ID: You can disable this status message by setting the Use the checkbox below for a quick retry:
📝 WalkthroughWalkthroughThis PR adds comprehensive haptic device feedback support to Isaac Teleop. It introduces abstract tracker contracts for pushing/reading haptic commands, live and replay implementations for OpenXR controller haptics and Manus glove integration, FlatBuffer serialization for glove commands, vendor-neutral device adapters (controller and glove), tactile-to-haptic retargeting mappers with gain/deadband/saturation shaping and EMA smoothing, a HapticSink component for integrating devices into the retargeting graph, two end-to-end example pipelines (hand pinch proximity and trigger-driven controller haptics), comprehensive test coverage, and build wiring to package Python bindings and examples. Estimated code review effort🎯 4 (Complex) | ⏱️ ~60 minutes ✨ Finishing Touches🧪 Generate unit tests (beta)
|
8842bd4 to
8140af4
Compare
There was a problem hiding this comment.
Actionable comments posted: 5
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@examples/haptic_feedback/python/openxr_controller_haptic_example.py`:
- Around line 214-219: The monitoring dict currently maps
"haptic_left"/"haptic_right" directly from pulse_selectors which stays
unswapped, causing printed haptic sides to be wrong when cross-hand routing is
active; update the construction of monitoring (the monitoring variable) so that
when cross-hand mode is enabled you swap the pulse_selectors used for
"haptic_left" and "haptic_right" (mirroring what is done for trigger_selectors)
so the monitored haptic side matches the actual routed sink inputs.
In `@src/core/schema/fbs/CMakeLists.txt`:
- Around line 49-53: The staging directory copy step (add_custom_target
schema_fb_python using ${CMAKE_COMMAND} -E copy_directory from
${SCHEMA_PYTHON_OUTPUT_DIR} to ${SCHEMA_FB_STAGING_DIR}) can leave stale files
because copy_directory only overlays; change the target to explicitly remove or
clean ${SCHEMA_FB_STAGING_DIR} first (e.g., call ${CMAKE_COMMAND} -E
remove_directory ${SCHEMA_FB_STAGING_DIR} or a remove and re-create sequence)
before creating the directory and copying, ensuring removed/renamed generated
modules are not left behind.
In `@src/haptic_devices/glove.py`:
- Around line 121-136: In HapticGloveDevice.__init__, add a fail-fast
compatibility check between num_fingers and max_payload_size: compute the
minimum required payload (e.g., at least one payload unit per finger or using an
existing MIN_PAYLOAD_PER_FINGER constant) and raise ValueError if
max_payload_size is too small to accommodate num_fingers; reference the
constructor parameters num_fingers and max_payload_size (and introduce a
MIN_PAYLOAD_PER_FINGER constant if none exists) so misconfigured
HapticGloveDevice instances fail immediately rather than later in push/send
methods.
In `@src/plugins/manus/core/manus_hand_tracking_plugin.cpp`:
- Around line 174-181: In ManusTracker::apply_haptic_command, sanitize each
element of the powers array before clamping to [0,1]: for each index used to
populate the existing clamped std::array<float, 5> (the one passed to
CoreSdk_VibrateFingersForGlove), check std::isfinite(powers[i]) and replace
non-finite values with 0.0f (or another safe default) before calling std::clamp,
then pass the sanitized clamped array to CoreSdk_VibrateFingersForGlove; also
add `#include` <cmath> to the file to ensure std::isfinite is available.
In `@src/retargeters/tactile_retargeters.py`:
- Around line 408-427: The constructor currently allows non-positive dimensions
which can lead to runtime failures; add guard checks in the __init__ to validate
that rows, cols, and num_pads are positive integers (e.g., if rows <= 0 or cols
<= 0 or num_pads <= 0: raise ValueError with a clear message referencing the
class/name), and keep the existing reduction validation; apply the same
validation logic to the analogous constructor/block around the code that sets
self._rows/self._cols/self._num_pads/_reduction (the similar section referenced
at lines 490–495) so invalid heatmap dimensions fail fast.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Path: .coderabbit.yaml
Review profile: CHILL
Plan: Enterprise
Run ID: c56d40e6-0e17-4153-82e1-a711d5acdeb2
📒 Files selected for processing (64)
CMakeLists.txtcmake/GenerateFlatBuffers.cmakeexamples/haptic_feedback/CMakeLists.txtexamples/haptic_feedback/python/README.mdexamples/haptic_feedback/python/hand_pinch_haptic_example.pyexamples/haptic_feedback/python/openxr_controller_haptic_example.pyexamples/haptic_feedback/python/pyproject.tomlsrc/core/AGENTS.mdsrc/core/deviceio_base/cpp/inc/deviceio_base/controller_tracker_base.hppsrc/core/deviceio_base/cpp/inc/deviceio_base/haptic_glove_command_pusher_tracker_base.hppsrc/core/deviceio_base/cpp/inc/deviceio_base/haptic_glove_command_reader_tracker_base.hppsrc/core/deviceio_trackers/cpp/CMakeLists.txtsrc/core/deviceio_trackers/cpp/controller_tracker.cppsrc/core/deviceio_trackers/cpp/haptic_glove_command_pusher_tracker.cppsrc/core/deviceio_trackers/cpp/haptic_glove_command_reader_tracker.cppsrc/core/deviceio_trackers/cpp/inc/deviceio_trackers/controller_tracker.hppsrc/core/deviceio_trackers/cpp/inc/deviceio_trackers/haptic_glove_command_pusher_tracker.hppsrc/core/deviceio_trackers/cpp/inc/deviceio_trackers/haptic_glove_command_reader_tracker.hppsrc/core/deviceio_trackers/python/deviceio_trackers_init.pysrc/core/deviceio_trackers/python/tracker_bindings.cppsrc/core/live_trackers/cpp/CMakeLists.txtsrc/core/live_trackers/cpp/inc/live_trackers/live_deviceio_factory.hppsrc/core/live_trackers/cpp/live_controller_tracker_impl.cppsrc/core/live_trackers/cpp/live_controller_tracker_impl.hppsrc/core/live_trackers/cpp/live_deviceio_factory.cppsrc/core/live_trackers/cpp/live_haptic_glove_command_pusher_tracker_impl.cppsrc/core/live_trackers/cpp/live_haptic_glove_command_pusher_tracker_impl.hppsrc/core/live_trackers/cpp/live_haptic_glove_command_reader_tracker_impl.cppsrc/core/live_trackers/cpp/live_haptic_glove_command_reader_tracker_impl.hppsrc/core/oxr_utils/cpp/inc/oxr_utils/oxr_funcs.hppsrc/core/python/CMakeLists.txtsrc/core/python/pyproject.toml.insrc/core/python/requirements.txtsrc/core/replay_trackers/cpp/CMakeLists.txtsrc/core/replay_trackers/cpp/inc/replay_trackers/replay_deviceio_factory.hppsrc/core/replay_trackers/cpp/replay_controller_tracker_impl.hppsrc/core/replay_trackers/cpp/replay_deviceio_factory.cppsrc/core/replay_trackers/cpp/replay_haptic_glove_command_pusher_tracker_impl.cppsrc/core/replay_trackers/cpp/replay_haptic_glove_command_pusher_tracker_impl.hppsrc/core/replay_trackers/cpp/replay_haptic_glove_command_reader_tracker_impl.cppsrc/core/replay_trackers/cpp/replay_haptic_glove_command_reader_tracker_impl.hppsrc/core/retargeting_engine/python/deviceio_source_nodes/__init__.pysrc/core/retargeting_engine/python/deviceio_source_nodes/haptic_sink.pysrc/core/retargeting_engine/python/tensor_types/__init__.pysrc/core/retargeting_engine/python/tensor_types/indices.pysrc/core/retargeting_engine/python/tensor_types/tactile_types.pysrc/core/retargeting_engine_tests/python/test_haptic_devices.pysrc/core/retargeting_engine_tests/python/test_haptic_glove.pysrc/core/retargeting_engine_tests/python/test_haptic_sink.pysrc/core/retargeting_engine_tests/python/test_tactile_retargeters.pysrc/core/schema/fbs/CMakeLists.txtsrc/core/schema/fbs/haptic_glove.fbssrc/haptic_devices/CMakeLists.txtsrc/haptic_devices/__init__.pysrc/haptic_devices/glove.pysrc/haptic_devices/interface.pysrc/haptic_devices/openxr_controller.pysrc/plugins/manus/CMakeLists.txtsrc/plugins/manus/core/manus_hand_tracking_plugin.cppsrc/plugins/manus/inc/core/manus_glove_collection.hppsrc/plugins/manus/inc/core/manus_hand_tracking_plugin.hppsrc/plugins/manus/tests/CMakeLists.txtsrc/plugins/manus/tests/python/test_manus_glove_collection_id.pysrc/retargeters/tactile_retargeters.py
8140af4 to
a9b3541
Compare
a9b3541 to
5f22738
Compare
5f22738 to
8e7deb6
Compare
Glove-path review fixes (CodeRabbit): - Manus consumer: std::clamp passes NaN / +/-Inf through unchanged, so apply_haptic_command now guards each finger power with std::isfinite before clamping. A non-finite power can no longer reach CoreSdk_VibrateFingersForGlove (add <cmath>). - TactileHeatmapToFingerPower / TactileHeatmapToWristPulse: validate rows/cols/num_pads >= 1 in __init__ so a misconfigured heatmap fails fast at construction instead of hitting an empty reduction at run time. Add a regression test for each. The remaining CodeRabbit items no longer apply after the generic push-device redesign: the flatc-Python schema staging target and the per-vendor HapticGloveDevice payload-size check are both gone (the glove path uses a fixed generous payload default and logs oversize pushes once), and the cross-hand controller-example note is controller scope.
Glove-path review fixes (CodeRabbit): - Manus consumer: std::clamp passes NaN / +/-Inf through unchanged, so apply_haptic_command now guards each finger power with std::isfinite before clamping. A non-finite power can no longer reach CoreSdk_VibrateFingersForGlove (add <cmath>). - TactileHeatmapToFingerPower / TactileHeatmapToWristPulse: validate rows/cols/num_pads >= 1 in __init__ so a misconfigured heatmap fails fast at construction instead of hitting an empty reduction at run time. Add a regression test for each. The remaining CodeRabbit items no longer apply after the generic push-device redesign: the flatc-Python schema staging target and the per-vendor HapticGloveDevice payload-size check are both gone (the glove path uses a fixed generous payload default and logs oversize pushes once), and the cross-hand controller-example note is controller scope.
73a9cb6 to
6009ec2
Compare
There was a problem hiding this comment.
Actionable comments posted: 4
♻️ Duplicate comments (1)
src/plugins/manus/core/manus_hand_tracking_plugin.cpp (1)
251-258:⚠️ Potential issue | 🟠 Major | ⚡ Quick winKeep the tensor reader optional.
Unconditionally pushing
m_haptic_readerintotrackersmakesXR_NVX1_tensor_dataa hard requirement forOpenXRSession. On runtimes without that extension,initialize()drops into the catch path and disables all OpenXR-based Manus behavior, not just haptics. Gate reader creation on extension support/capability so hand injection and controller wrist fallback still come up when haptics are unavailable.🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@src/plugins/manus/core/manus_hand_tracking_plugin.cpp` around lines 251 - 258, The code unconditionally creates and pushes m_haptic_reader (core::HapticCommandReaderTracker with MANUS_GLOVE_COLLECTION_ID) into trackers which forces XR_NVX1_tensor_data as a required OpenXR extension; change to make the tensor reader optional by first checking whether the runtime advertises the NVX1 tensor data capability/extension and only then creating m_haptic_reader and pushing it into trackers — ensure when the extension is absent you skip creation (leave m_haptic_reader null) so initialize() and OpenXRSession still succeed and other Manus features (hand injection, controller wrist fallback) remain available.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@src/core/deviceio_trackers/cpp/tensor_push_tracker.cpp`:
- Around line 31-34: The TensorPushTracker::push implementation must validate
payload size against max_payload_size() before calling into the impl; add an
explicit check in TensorPushTracker::push that if payload.size() >
max_payload_size() you reject the call (e.g., throw a std::invalid_argument or a
defined API error) with a clear message that includes payload.size() and
max_payload_size(), and only call static_cast<const
ITensorPushTrackerImpl&>(session.get_tracker_impl(*this)).push(payload) when the
size is valid.
In `@src/core/retargeting_engine_tests/python/test_haptic_devices.py`:
- Around line 228-231: Remove assertions that look for literal b"left"/b"right"
in the serialized payload and instead decode each serialized HapticCommand from
recorder.pushes; for each decoded HapticCommand (use the test's HapticCommand
decoder), assert the side flag (e.g., is_left or side bit) and that the
power/strength fields for the corresponding actuators are present/within
expected ranges; apply the same replacement for the similar checks at the other
block (lines 271-277) so tests validate the semantic side bit and power values
rather than the old string-based wire format.
In `@src/core/retargeting_engine_tests/python/test_haptic_sink.py`:
- Around line 120-128: The test test_connect_rejects_mismatched_upstream_type is
too broad by catching Exception; update the assertion to only accept the
mismatch-related errors (e.g. TypeError or AssertionError) raised by the
compatibility check in HapticSink.connect when connecting a
ControllerHapticPulse()-accepting _RecordingDevice to a ValueInput("upstream",
TactileVector(5)) output. Replace pytest.raises(Exception) with
pytest.raises((TypeError, AssertionError)) so only the expected type-mismatch
failures from HapticSink.connect (or the underlying TensorType checks) will make
the test pass.
In `@src/haptic_devices/controller.py`:
- Around line 67-77: The constructor __init__ currently accepts arbitrary
endpoint names into _endpoints while flush() only handles "left" and "right",
causing silent no-ops for unsupported endpoints; validate the provided endpoints
in __init__ against the supported set {"left","right"} and raise a clear
ValueError if any unsupported endpoint is present (mention the invalid names in
the message), or alternatively normalize inputs to the supported set; update
_endpoints, _pending and _error_logged initialization to only include validated
endpoints and ensure any code that references _endpoints or flush() (e.g. flush)
will operate only on the validated values.
---
Duplicate comments:
In `@src/plugins/manus/core/manus_hand_tracking_plugin.cpp`:
- Around line 251-258: The code unconditionally creates and pushes
m_haptic_reader (core::HapticCommandReaderTracker with
MANUS_GLOVE_COLLECTION_ID) into trackers which forces XR_NVX1_tensor_data as a
required OpenXR extension; change to make the tensor reader optional by first
checking whether the runtime advertises the NVX1 tensor data
capability/extension and only then creating m_haptic_reader and pushing it into
trackers — ensure when the extension is absent you skip creation (leave
m_haptic_reader null) so initialize() and OpenXRSession still succeed and other
Manus features (hand injection, controller wrist fallback) remain available.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Path: .coderabbit.yaml
Review profile: CHILL
Plan: Enterprise
Run ID: 95694869-c854-472a-b5ea-39ca95cb1fc9
📒 Files selected for processing (71)
AGENTS.mdCMakeLists.txtdocs/source/device/haptic_feedback.rstdocs/source/device/index.rstexamples/haptic_feedback/CMakeLists.txtexamples/haptic_feedback/python/README.mdexamples/haptic_feedback/python/controller_haptic_example.pyexamples/haptic_feedback/python/hand_pinch_haptic_example.pyexamples/haptic_feedback/python/pyproject.tomlsrc/core/AGENTS.mdsrc/core/deviceio_base/cpp/inc/deviceio_base/controller_tracker_base.hppsrc/core/deviceio_base/cpp/inc/deviceio_base/haptic_command_reader_tracker_base.hppsrc/core/deviceio_base/cpp/inc/deviceio_base/tensor_push_tracker_base.hppsrc/core/deviceio_trackers/cpp/CMakeLists.txtsrc/core/deviceio_trackers/cpp/controller_tracker.cppsrc/core/deviceio_trackers/cpp/haptic_command_reader_tracker.cppsrc/core/deviceio_trackers/cpp/inc/deviceio_trackers/controller_tracker.hppsrc/core/deviceio_trackers/cpp/inc/deviceio_trackers/haptic_command_reader_tracker.hppsrc/core/deviceio_trackers/cpp/inc/deviceio_trackers/tensor_push_tracker.hppsrc/core/deviceio_trackers/cpp/tensor_push_tracker.cppsrc/core/deviceio_trackers/python/deviceio_trackers_init.pysrc/core/deviceio_trackers/python/tracker_bindings.cppsrc/core/live_trackers/cpp/CMakeLists.txtsrc/core/live_trackers/cpp/inc/live_trackers/live_deviceio_factory.hppsrc/core/live_trackers/cpp/live_controller_tracker_impl.cppsrc/core/live_trackers/cpp/live_controller_tracker_impl.hppsrc/core/live_trackers/cpp/live_deviceio_factory.cppsrc/core/live_trackers/cpp/live_haptic_command_reader_tracker_impl.cppsrc/core/live_trackers/cpp/live_haptic_command_reader_tracker_impl.hppsrc/core/live_trackers/cpp/live_tensor_push_tracker_impl.cppsrc/core/live_trackers/cpp/live_tensor_push_tracker_impl.hppsrc/core/oxr_utils/cpp/inc/oxr_utils/oxr_funcs.hppsrc/core/python/CMakeLists.txtsrc/core/python/deviceio_init.pysrc/core/python/pyproject.toml.insrc/core/replay_trackers/cpp/CMakeLists.txtsrc/core/replay_trackers/cpp/inc/replay_trackers/replay_deviceio_factory.hppsrc/core/replay_trackers/cpp/replay_controller_tracker_impl.hppsrc/core/replay_trackers/cpp/replay_deviceio_factory.cppsrc/core/replay_trackers/cpp/replay_haptic_command_reader_tracker_impl.cppsrc/core/replay_trackers/cpp/replay_haptic_command_reader_tracker_impl.hppsrc/core/replay_trackers/cpp/replay_tensor_push_tracker_impl.cppsrc/core/replay_trackers/cpp/replay_tensor_push_tracker_impl.hppsrc/core/retargeting_engine/python/deviceio_source_nodes/__init__.pysrc/core/retargeting_engine/python/deviceio_source_nodes/haptic_sink.pysrc/core/retargeting_engine/python/deviceio_source_nodes/sink_interface.pysrc/core/retargeting_engine/python/interface/retargeter_subgraph.pysrc/core/retargeting_engine/python/tensor_types/__init__.pysrc/core/retargeting_engine/python/tensor_types/indices.pysrc/core/retargeting_engine/python/tensor_types/tactile_types.pysrc/core/retargeting_engine_tests/python/test_haptic_devices.pysrc/core/retargeting_engine_tests/python/test_haptic_sink.pysrc/core/retargeting_engine_tests/python/test_tactile_retargeters.pysrc/core/schema/fbs/haptic_command.fbssrc/core/schema/python/CMakeLists.txtsrc/core/schema/python/haptic_command_bindings.hsrc/core/schema/python/schema_init.pysrc/core/schema/python/schema_module.cppsrc/core/teleop_session_manager/python/config.pysrc/core/teleop_session_manager/python/teleop_session.pysrc/core/teleop_session_manager_tests/python/test_teleop_session.pysrc/haptic_devices/CMakeLists.txtsrc/haptic_devices/__init__.pysrc/haptic_devices/controller.pysrc/haptic_devices/glove.pysrc/haptic_devices/interface.pysrc/haptic_devices/push_tensor.pysrc/plugins/manus/core/inc/manus/manus_glove_collection.hppsrc/plugins/manus/core/inc/manus/manus_hand_tracking_plugin.hppsrc/plugins/manus/core/manus_hand_tracking_plugin.cppsrc/retargeters/tactile_retargeters.py
✅ Files skipped from review due to trivial changes (11)
- examples/haptic_feedback/CMakeLists.txt
- src/plugins/manus/core/inc/manus/manus_glove_collection.hpp
- docs/source/device/index.rst
- src/core/deviceio_base/cpp/inc/deviceio_base/haptic_command_reader_tracker_base.hpp
- src/core/replay_trackers/cpp/replay_haptic_command_reader_tracker_impl.hpp
- src/core/schema/fbs/haptic_command.fbs
- src/core/AGENTS.md
- src/core/deviceio_base/cpp/inc/deviceio_base/tensor_push_tracker_base.hpp
- AGENTS.md
- examples/haptic_feedback/python/pyproject.toml
- examples/haptic_feedback/python/README.md
🚧 Files skipped from review as they are similar to previous changes (15)
- src/core/python/CMakeLists.txt
- src/core/python/pyproject.toml.in
- src/core/live_trackers/cpp/CMakeLists.txt
- src/core/retargeting_engine/python/tensor_types/indices.py
- src/core/live_trackers/cpp/live_controller_tracker_impl.hpp
- src/haptic_devices/CMakeLists.txt
- src/core/retargeting_engine/python/tensor_types/tactile_types.py
- src/core/deviceio_trackers/cpp/CMakeLists.txt
- CMakeLists.txt
- src/core/retargeting_engine/python/tensor_types/init.py
- src/core/replay_trackers/cpp/CMakeLists.txt
- src/core/deviceio_base/cpp/inc/deviceio_base/controller_tracker_base.hpp
- src/core/oxr_utils/cpp/inc/oxr_utils/oxr_funcs.hpp
- src/core/live_trackers/cpp/live_controller_tracker_impl.cpp
- src/retargeters/tactile_retargeters.py
| void TensorPushTracker::push(const ITrackerSession& session, const std::vector<uint8_t>& payload) const | ||
| { | ||
| static_cast<const ITensorPushTrackerImpl&>(session.get_tracker_impl(*this)).push(payload); | ||
| } |
There was a problem hiding this comment.
Reject oversized payloads before dispatch.
push() documents payload.size() <= max_payload_size() but never enforces it. That lets callers pass an invalid buffer into the impl layer, where the failure mode becomes truncation/assert/runtime-specific behavior instead of a predictable API error.
Proposed fix
void TensorPushTracker::push(const ITrackerSession& session, const std::vector<uint8_t>& payload) const
{
+ if (payload.size() > max_payload_size_)
+ {
+ throw std::invalid_argument("TensorPushTracker: payload size exceeds max_payload_size");
+ }
static_cast<const ITensorPushTrackerImpl&>(session.get_tracker_impl(*this)).push(payload);
}🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@src/core/deviceio_trackers/cpp/tensor_push_tracker.cpp` around lines 31 - 34,
The TensorPushTracker::push implementation must validate payload size against
max_payload_size() before calling into the impl; add an explicit check in
TensorPushTracker::push that if payload.size() > max_payload_size() you reject
the call (e.g., throw a std::invalid_argument or a defined API error) with a
clear message that includes payload.size() and max_payload_size(), and only call
static_cast<const
ITensorPushTrackerImpl&>(session.get_tracker_impl(*this)).push(payload) when the
size is valid.
Glove-path review fixes (CodeRabbit): - Manus consumer: std::clamp passes NaN / +/-Inf through unchanged, so apply_haptic_command now guards each finger power with std::isfinite before clamping. A non-finite power can no longer reach CoreSdk_VibrateFingersForGlove (add <cmath>). - TactileHeatmapToFingerPower / TactileHeatmapToWristPulse: validate rows/cols/num_pads >= 1 in __init__ so a misconfigured heatmap fails fast at construction instead of hitting an empty reduction at run time. Add a regression test for each. The remaining CodeRabbit items no longer apply after the generic push-device redesign: the flatc-Python schema staging target and the per-vendor HapticGloveDevice payload-size check are both gone (the glove path uses a fixed generous payload default and logs oversize pushes once), and the cross-hand controller-example note is controller scope.
6009ec2 to
424cc3d
Compare
Glove-path review fixes (CodeRabbit): - Manus consumer: std::clamp passes NaN / +/-Inf through unchanged, so apply_haptic_command now guards each finger power with std::isfinite before clamping. A non-finite power can no longer reach CoreSdk_VibrateFingersForGlove (add <cmath>). - TactileHeatmapToFingerPower / TactileHeatmapToWristPulse: validate rows/cols/num_pads >= 1 in __init__ so a misconfigured heatmap fails fast at construction instead of hitting an empty reduction at run time. Add a regression test for each. The remaining CodeRabbit items no longer apply after the generic push-device redesign: the flatc-Python schema staging target and the per-vendor HapticGloveDevice payload-size check are both gone (the glove path uses a fixed generous payload default and logs oversize pushes once), and the cross-hand controller-example note is controller scope.
424cc3d to
1420c89
Compare
Build the cross-process device-output path as a vendor-neutral layer so any out-of-process haptic device (glove, exoskeleton) reuses one transport and one wire schema instead of a per-vendor tracker and FlatBuffer. Core transport (C++): - HapticCommand schema (endpoint: string, values: [float]). - TensorPushTracker: generic producer that pushes opaque payloads over XR_NVX1_push_tensor; HapticCommandReaderTracker: the paired consumer. - Base/facade/live/replay impls, factory wiring, and pybind bindings, plus a pack_haptic_command() encoder on the schema module. Python device + retargeting: - PushTensorHapticDevice (IHapticDevice) serializes per-endpoint values into a HapticCommand and pushes them; haptic_glove_device() binds it to FingerPowerVector. - FingerPowerVector / FingerIndex tensor types and the FingerPower tactile retargeters (TactileVector/Heatmap -> FingerPower, FingerPower -> pulse). Reference integrations: - Manus plugin consumes HapticCommand via HapticCommandReaderTracker and drives CoreSdk_VibrateFingersForGlove. - hand_pinch_haptic_example.py: pinch proximity -> per-finger vibration. OpenXR vocabulary stays in the live/runtime layer; device adapters, schemas, and retargeters remain vendor-neutral.
Production review pass over the glove device-output path: - Run clang-format on the push/reader trackers, factories, and the Manus plugin. CI runs clang-format-14 and rejects unformatted C++ (pre-commit does not check it), so these files needed formatting. - Replace the Manus literal finger count with a kManusFingerCount constant so the actuator count reads as a parameter a different glove would change. - Log the replay-mode push drop once instead of every frame. - Drop version-referencing comment framing (no "v1" labels; describe the current behavior directly).
Glove-path review fixes (CodeRabbit): - Manus consumer: std::clamp passes NaN / +/-Inf through unchanged, so apply_haptic_command now guards each finger power with std::isfinite before clamping. A non-finite power can no longer reach CoreSdk_VibrateFingersForGlove (add <cmath>). - TactileHeatmapToFingerPower / TactileHeatmapToWristPulse: validate rows/cols/num_pads >= 1 in __init__ so a misconfigured heatmap fails fast at construction instead of hitting an empty reduction at run time. Add a regression test for each. The remaining CodeRabbit items no longer apply after the generic push-device redesign: the flatc-Python schema staging target and the per-vendor HapticGloveDevice payload-size check are both gone (the glove path uses a fixed generous payload default and logs oversize pushes once), and the cross-hand controller-example note is controller scope.
1420c89 to
af8ac30
Compare
There was a problem hiding this comment.
Actionable comments posted: 2
♻️ Duplicate comments (2)
src/plugins/manus/core/manus_hand_tracking_plugin.cpp (1)
251-259:⚠️ Potential issue | 🟠 Major | ⚡ Quick winMake haptic reader registration conditional to avoid hard-failing OpenXR on runtimes without tensor-data support.
Registering
m_haptic_readerunconditionally turns haptics into a mandatory extension dependency for session creation. That can disable otherwise-working hand/controller injection on runtimes that don’t advertise the tensor-data extension.🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@src/plugins/manus/core/manus_hand_tracking_plugin.cpp` around lines 251 - 259, The haptic reader (m_haptic_reader created with core::HapticCommandReaderTracker and pushed into trackers) is registered unconditionally which makes XR_NVX1_tensor_data a hard session dependency; change this to only create and register m_haptic_reader if the runtime advertises tensor-data support (check the session/instance extension list for "XR_NVX1_tensor_data" or use the project's extension-query helper), otherwise leave m_haptic_reader null and do not push it into trackers; also update any later code that dereferences m_haptic_reader to guard against null before use and emit a debug/log message when haptics are skipped.src/core/deviceio_trackers/cpp/tensor_push_tracker.cpp (1)
31-34:⚠️ Potential issue | 🟠 Major | ⚡ Quick winEnforce payload-size bounds before forwarding to the impl.
Line 31 accepts arbitrary payloads, but
max_payload_size_is never enforced before dispatch on Line 33. Oversized payloads should be rejected here to keep API behavior deterministic.Suggested fix
void TensorPushTracker::push(const ITrackerSession& session, const std::vector<uint8_t>& payload) const { + if (payload.size() > max_payload_size_) + { + throw std::invalid_argument("TensorPushTracker: payload size exceeds max_payload_size"); + } static_cast<const ITensorPushTrackerImpl&>(session.get_tracker_impl(*this)).push(payload); }🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@src/core/deviceio_trackers/cpp/tensor_push_tracker.cpp` around lines 31 - 34, TensorPushTracker::push currently forwards any payload without checking max_payload_size_; add a bounds check at the start of TensorPushTracker::push that compares payload.size() against the member max_payload_size_ and rejects oversized payloads (e.g., by throwing a clear exception such as std::length_error or std::invalid_argument with a message including payload.size() and max_payload_size_). If within bounds, continue to call session.get_tracker_impl(*this).push(payload) as before; ensure the check is performed before the static_cast/forward so the impl never receives an oversized buffer.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@src/retargeters/tactile_retargeters.py`:
- Around line 669-685: TactileVectorToControllerPulse.__init__ accepts
num_taxels with no validation which can cause invalid shapes later; add an
explicit check in __init__ that num_taxels is an int >= 1 (and optionally raise
TypeError if not int) and raise a clear ValueError for bad values; update any
related attributes (self._num_taxels) only after validation so
input_spec/reshape logic uses a validated value (refer to
TactileVectorToControllerPulse.__init__, self._num_taxels, input_spec and
reshape paths).
- Around line 789-809: TactileHeatmapToControllerPulse.__init__ currently allows
non-positive rows, cols, and num_pads; add early validation in the constructor
(TactileHeatmapToControllerPulse.__init__) to raise ValueError if rows <= 0,
cols <= 0, or num_pads <= 0 with a clear message including the parameter name
and the provided value so invalid heatmap dimensions are rejected immediately;
keep the existing reduction validation and then assign self._rows, self._cols,
and self._num_pads only after these checks.
---
Duplicate comments:
In `@src/core/deviceio_trackers/cpp/tensor_push_tracker.cpp`:
- Around line 31-34: TensorPushTracker::push currently forwards any payload
without checking max_payload_size_; add a bounds check at the start of
TensorPushTracker::push that compares payload.size() against the member
max_payload_size_ and rejects oversized payloads (e.g., by throwing a clear
exception such as std::length_error or std::invalid_argument with a message
including payload.size() and max_payload_size_). If within bounds, continue to
call session.get_tracker_impl(*this).push(payload) as before; ensure the check
is performed before the static_cast/forward so the impl never receives an
oversized buffer.
In `@src/plugins/manus/core/manus_hand_tracking_plugin.cpp`:
- Around line 251-259: The haptic reader (m_haptic_reader created with
core::HapticCommandReaderTracker and pushed into trackers) is registered
unconditionally which makes XR_NVX1_tensor_data a hard session dependency;
change this to only create and register m_haptic_reader if the runtime
advertises tensor-data support (check the session/instance extension list for
"XR_NVX1_tensor_data" or use the project's extension-query helper), otherwise
leave m_haptic_reader null and do not push it into trackers; also update any
later code that dereferences m_haptic_reader to guard against null before use
and emit a debug/log message when haptics are skipped.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Path: .coderabbit.yaml
Review profile: CHILL
Plan: Enterprise
Run ID: 56096cf1-c154-4d5d-ad7e-95cddbae375a
📒 Files selected for processing (42)
examples/haptic_feedback/python/README.mdexamples/haptic_feedback/python/hand_pinch_haptic_example.pysrc/core/deviceio_base/cpp/inc/deviceio_base/haptic_command_reader_tracker_base.hppsrc/core/deviceio_base/cpp/inc/deviceio_base/tensor_push_tracker_base.hppsrc/core/deviceio_trackers/cpp/CMakeLists.txtsrc/core/deviceio_trackers/cpp/haptic_command_reader_tracker.cppsrc/core/deviceio_trackers/cpp/inc/deviceio_trackers/haptic_command_reader_tracker.hppsrc/core/deviceio_trackers/cpp/inc/deviceio_trackers/tensor_push_tracker.hppsrc/core/deviceio_trackers/cpp/tensor_push_tracker.cppsrc/core/deviceio_trackers/python/deviceio_trackers_init.pysrc/core/deviceio_trackers/python/tracker_bindings.cppsrc/core/live_trackers/cpp/CMakeLists.txtsrc/core/live_trackers/cpp/inc/live_trackers/live_deviceio_factory.hppsrc/core/live_trackers/cpp/live_deviceio_factory.cppsrc/core/live_trackers/cpp/live_haptic_command_reader_tracker_impl.cppsrc/core/live_trackers/cpp/live_haptic_command_reader_tracker_impl.hppsrc/core/live_trackers/cpp/live_tensor_push_tracker_impl.cppsrc/core/live_trackers/cpp/live_tensor_push_tracker_impl.hppsrc/core/python/deviceio_init.pysrc/core/replay_trackers/cpp/CMakeLists.txtsrc/core/replay_trackers/cpp/inc/replay_trackers/replay_deviceio_factory.hppsrc/core/replay_trackers/cpp/replay_deviceio_factory.cppsrc/core/replay_trackers/cpp/replay_haptic_command_reader_tracker_impl.cppsrc/core/replay_trackers/cpp/replay_haptic_command_reader_tracker_impl.hppsrc/core/replay_trackers/cpp/replay_tensor_push_tracker_impl.cppsrc/core/replay_trackers/cpp/replay_tensor_push_tracker_impl.hppsrc/core/retargeting_engine/python/tensor_types/__init__.pysrc/core/retargeting_engine/python/tensor_types/indices.pysrc/core/retargeting_engine/python/tensor_types/tactile_types.pysrc/core/retargeting_engine_tests/python/test_haptic_devices.pysrc/core/retargeting_engine_tests/python/test_tactile_retargeters.pysrc/core/schema/fbs/haptic_command.fbssrc/core/schema/python/CMakeLists.txtsrc/core/schema/python/haptic_command_bindings.hsrc/core/schema/python/schema_init.pysrc/core/schema/python/schema_module.cppsrc/haptic_devices/glove.pysrc/haptic_devices/push_tensor.pysrc/plugins/manus/core/inc/manus/manus_glove_collection.hppsrc/plugins/manus/core/inc/manus/manus_hand_tracking_plugin.hppsrc/plugins/manus/core/manus_hand_tracking_plugin.cppsrc/retargeters/tactile_retargeters.py
✅ Files skipped from review due to trivial changes (5)
- src/core/schema/python/CMakeLists.txt
- src/core/deviceio_trackers/python/deviceio_trackers_init.py
- src/haptic_devices/glove.py
- examples/haptic_feedback/python/README.md
- src/core/live_trackers/cpp/live_haptic_command_reader_tracker_impl.hpp
🚧 Files skipped from review as they are similar to previous changes (2)
- src/core/deviceio_trackers/cpp/CMakeLists.txt
- src/core/retargeting_engine/python/tensor_types/init.py
There was a problem hiding this comment.
Caution
Inline review comments failed to post. This is likely due to GitHub's internal server error or limits when posting large numbers of comments. If you are seeing this consistently it is likely a permissions issue. Please check "Moderation" -> "Code review limits" under your organization settings.
Actionable comments posted: 2
♻️ Duplicate comments (2)
src/plugins/manus/core/manus_hand_tracking_plugin.cpp (1)
251-259:⚠️ Potential issue | 🟠 Major | ⚡ Quick winMake haptic reader registration conditional to avoid hard-failing OpenXR on runtimes without tensor-data support.
Registering
m_haptic_readerunconditionally turns haptics into a mandatory extension dependency for session creation. That can disable otherwise-working hand/controller injection on runtimes that don’t advertise the tensor-data extension.🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@src/plugins/manus/core/manus_hand_tracking_plugin.cpp` around lines 251 - 259, The haptic reader (m_haptic_reader created with core::HapticCommandReaderTracker and pushed into trackers) is registered unconditionally which makes XR_NVX1_tensor_data a hard session dependency; change this to only create and register m_haptic_reader if the runtime advertises tensor-data support (check the session/instance extension list for "XR_NVX1_tensor_data" or use the project's extension-query helper), otherwise leave m_haptic_reader null and do not push it into trackers; also update any later code that dereferences m_haptic_reader to guard against null before use and emit a debug/log message when haptics are skipped.src/core/deviceio_trackers/cpp/tensor_push_tracker.cpp (1)
31-34:⚠️ Potential issue | 🟠 Major | ⚡ Quick winEnforce payload-size bounds before forwarding to the impl.
Line 31 accepts arbitrary payloads, but
max_payload_size_is never enforced before dispatch on Line 33. Oversized payloads should be rejected here to keep API behavior deterministic.Suggested fix
void TensorPushTracker::push(const ITrackerSession& session, const std::vector<uint8_t>& payload) const { + if (payload.size() > max_payload_size_) + { + throw std::invalid_argument("TensorPushTracker: payload size exceeds max_payload_size"); + } static_cast<const ITensorPushTrackerImpl&>(session.get_tracker_impl(*this)).push(payload); }🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@src/core/deviceio_trackers/cpp/tensor_push_tracker.cpp` around lines 31 - 34, TensorPushTracker::push currently forwards any payload without checking max_payload_size_; add a bounds check at the start of TensorPushTracker::push that compares payload.size() against the member max_payload_size_ and rejects oversized payloads (e.g., by throwing a clear exception such as std::length_error or std::invalid_argument with a message including payload.size() and max_payload_size_). If within bounds, continue to call session.get_tracker_impl(*this).push(payload) as before; ensure the check is performed before the static_cast/forward so the impl never receives an oversized buffer.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@src/retargeters/tactile_retargeters.py`:
- Around line 669-685: TactileVectorToControllerPulse.__init__ accepts
num_taxels with no validation which can cause invalid shapes later; add an
explicit check in __init__ that num_taxels is an int >= 1 (and optionally raise
TypeError if not int) and raise a clear ValueError for bad values; update any
related attributes (self._num_taxels) only after validation so
input_spec/reshape logic uses a validated value (refer to
TactileVectorToControllerPulse.__init__, self._num_taxels, input_spec and
reshape paths).
- Around line 789-809: TactileHeatmapToControllerPulse.__init__ currently allows
non-positive rows, cols, and num_pads; add early validation in the constructor
(TactileHeatmapToControllerPulse.__init__) to raise ValueError if rows <= 0,
cols <= 0, or num_pads <= 0 with a clear message including the parameter name
and the provided value so invalid heatmap dimensions are rejected immediately;
keep the existing reduction validation and then assign self._rows, self._cols,
and self._num_pads only after these checks.
---
Duplicate comments:
In `@src/core/deviceio_trackers/cpp/tensor_push_tracker.cpp`:
- Around line 31-34: TensorPushTracker::push currently forwards any payload
without checking max_payload_size_; add a bounds check at the start of
TensorPushTracker::push that compares payload.size() against the member
max_payload_size_ and rejects oversized payloads (e.g., by throwing a clear
exception such as std::length_error or std::invalid_argument with a message
including payload.size() and max_payload_size_). If within bounds, continue to
call session.get_tracker_impl(*this).push(payload) as before; ensure the check
is performed before the static_cast/forward so the impl never receives an
oversized buffer.
In `@src/plugins/manus/core/manus_hand_tracking_plugin.cpp`:
- Around line 251-259: The haptic reader (m_haptic_reader created with
core::HapticCommandReaderTracker and pushed into trackers) is registered
unconditionally which makes XR_NVX1_tensor_data a hard session dependency;
change this to only create and register m_haptic_reader if the runtime
advertises tensor-data support (check the session/instance extension list for
"XR_NVX1_tensor_data" or use the project's extension-query helper), otherwise
leave m_haptic_reader null and do not push it into trackers; also update any
later code that dereferences m_haptic_reader to guard against null before use
and emit a debug/log message when haptics are skipped.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Path: .coderabbit.yaml
Review profile: CHILL
Plan: Enterprise
Run ID: 56096cf1-c154-4d5d-ad7e-95cddbae375a
📒 Files selected for processing (42)
examples/haptic_feedback/python/README.mdexamples/haptic_feedback/python/hand_pinch_haptic_example.pysrc/core/deviceio_base/cpp/inc/deviceio_base/haptic_command_reader_tracker_base.hppsrc/core/deviceio_base/cpp/inc/deviceio_base/tensor_push_tracker_base.hppsrc/core/deviceio_trackers/cpp/CMakeLists.txtsrc/core/deviceio_trackers/cpp/haptic_command_reader_tracker.cppsrc/core/deviceio_trackers/cpp/inc/deviceio_trackers/haptic_command_reader_tracker.hppsrc/core/deviceio_trackers/cpp/inc/deviceio_trackers/tensor_push_tracker.hppsrc/core/deviceio_trackers/cpp/tensor_push_tracker.cppsrc/core/deviceio_trackers/python/deviceio_trackers_init.pysrc/core/deviceio_trackers/python/tracker_bindings.cppsrc/core/live_trackers/cpp/CMakeLists.txtsrc/core/live_trackers/cpp/inc/live_trackers/live_deviceio_factory.hppsrc/core/live_trackers/cpp/live_deviceio_factory.cppsrc/core/live_trackers/cpp/live_haptic_command_reader_tracker_impl.cppsrc/core/live_trackers/cpp/live_haptic_command_reader_tracker_impl.hppsrc/core/live_trackers/cpp/live_tensor_push_tracker_impl.cppsrc/core/live_trackers/cpp/live_tensor_push_tracker_impl.hppsrc/core/python/deviceio_init.pysrc/core/replay_trackers/cpp/CMakeLists.txtsrc/core/replay_trackers/cpp/inc/replay_trackers/replay_deviceio_factory.hppsrc/core/replay_trackers/cpp/replay_deviceio_factory.cppsrc/core/replay_trackers/cpp/replay_haptic_command_reader_tracker_impl.cppsrc/core/replay_trackers/cpp/replay_haptic_command_reader_tracker_impl.hppsrc/core/replay_trackers/cpp/replay_tensor_push_tracker_impl.cppsrc/core/replay_trackers/cpp/replay_tensor_push_tracker_impl.hppsrc/core/retargeting_engine/python/tensor_types/__init__.pysrc/core/retargeting_engine/python/tensor_types/indices.pysrc/core/retargeting_engine/python/tensor_types/tactile_types.pysrc/core/retargeting_engine_tests/python/test_haptic_devices.pysrc/core/retargeting_engine_tests/python/test_tactile_retargeters.pysrc/core/schema/fbs/haptic_command.fbssrc/core/schema/python/CMakeLists.txtsrc/core/schema/python/haptic_command_bindings.hsrc/core/schema/python/schema_init.pysrc/core/schema/python/schema_module.cppsrc/haptic_devices/glove.pysrc/haptic_devices/push_tensor.pysrc/plugins/manus/core/inc/manus/manus_glove_collection.hppsrc/plugins/manus/core/inc/manus/manus_hand_tracking_plugin.hppsrc/plugins/manus/core/manus_hand_tracking_plugin.cppsrc/retargeters/tactile_retargeters.py
✅ Files skipped from review due to trivial changes (5)
- src/core/schema/python/CMakeLists.txt
- src/core/deviceio_trackers/python/deviceio_trackers_init.py
- src/haptic_devices/glove.py
- examples/haptic_feedback/python/README.md
- src/core/live_trackers/cpp/live_haptic_command_reader_tracker_impl.hpp
🚧 Files skipped from review as they are similar to previous changes (2)
- src/core/deviceio_trackers/cpp/CMakeLists.txt
- src/core/retargeting_engine/python/tensor_types/init.py
🛑 Comments failed to post (2)
src/retargeters/tactile_retargeters.py (2)
669-685:
⚠️ Potential issue | 🟠 Major | ⚡ Quick winValidate
num_taxelsinTactileVectorToControllerPulse.__init__.
num_taxelsis accepted without a lower-bound check, so invalid values can slip intoinput_specand reshape paths instead of failing fast.Proposed fix
def __init__( self, name: str, num_taxels: int, reduction: Literal["max", "mean", "sum"] = "max", gain: float = 1.0, deadband: float = 0.0, saturation: float = 1.0, frequency_hz: float = 0.0, duration_s: float = 0.0, ) -> None: + if num_taxels < 1: + raise ValueError( + f"TactileVectorToControllerPulse '{name}' requires num_taxels >= 1, got {num_taxels}" + ) if reduction not in ("max", "mean", "sum"): raise ValueError( f"TactileVectorToControllerPulse '{name}': unknown reduction '{reduction}'" )📝 Committable suggestion
‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.def __init__( self, name: str, num_taxels: int, reduction: Literal["max", "mean", "sum"] = "max", gain: float = 1.0, deadband: float = 0.0, saturation: float = 1.0, frequency_hz: float = 0.0, duration_s: float = 0.0, ) -> None: if num_taxels < 1: raise ValueError( f"TactileVectorToControllerPulse '{name}' requires num_taxels >= 1, got {num_taxels}" ) if reduction not in ("max", "mean", "sum"): raise ValueError( f"TactileVectorToControllerPulse '{name}': unknown reduction '{reduction}'" ) self._num_taxels = num_taxels self._reduction = reduction🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@src/retargeters/tactile_retargeters.py` around lines 669 - 685, TactileVectorToControllerPulse.__init__ accepts num_taxels with no validation which can cause invalid shapes later; add an explicit check in __init__ that num_taxels is an int >= 1 (and optionally raise TypeError if not int) and raise a clear ValueError for bad values; update any related attributes (self._num_taxels) only after validation so input_spec/reshape logic uses a validated value (refer to TactileVectorToControllerPulse.__init__, self._num_taxels, input_spec and reshape paths).
789-809:
⚠️ Potential issue | 🟠 Major | ⚡ Quick winFail fast on non-positive heatmap dimensions in
TactileHeatmapToControllerPulse.__init__.This constructor currently accepts invalid
rows,cols, andnum_pads, unlike the other heatmap mappers. That can defer bad configuration into runtime behavior instead of rejecting early.Proposed fix
def __init__( self, name: str, rows: int, cols: int, num_pads: int = 1, reduction: Literal["max", "mean", "sum"] = "max", gain: float = 1.0, deadband: float = 0.0, saturation: float = 1.0, frequency_hz: float = 0.0, duration_s: float = 0.0, ) -> None: + if rows < 1 or cols < 1 or num_pads < 1: + raise ValueError( + f"TactileHeatmapToControllerPulse '{name}' requires rows/cols/num_pads >= 1, " + f"got rows={rows}, cols={cols}, num_pads={num_pads}" + ) if reduction not in ("max", "mean", "sum"): raise ValueError( f"TactileHeatmapToControllerPulse '{name}': unknown reduction '{reduction}'" )🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@src/retargeters/tactile_retargeters.py` around lines 789 - 809, TactileHeatmapToControllerPulse.__init__ currently allows non-positive rows, cols, and num_pads; add early validation in the constructor (TactileHeatmapToControllerPulse.__init__) to raise ValueError if rows <= 0, cols <= 0, or num_pads <= 0 with a clear message including the parameter name and the provided value so invalid heatmap dimensions are rejected immediately; keep the existing reduction validation and then assign self._rows, self._cols, and self._num_pads only after these checks.
There was a problem hiding this comment.
♻️ Duplicate comments (1)
src/core/deviceio_trackers/cpp/tensor_push_tracker.cpp (1)
31-34:⚠️ Potential issue | 🔴 CriticalPayload size validation still missing.
The documented precondition
payload.size() <= max_payload_size()is not enforced before forwarding to the impl layer. This was flagged in a previous review and remains unaddressed.🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@src/core/deviceio_trackers/cpp/tensor_push_tracker.cpp` around lines 31 - 34, TensorPushTracker::push currently forwards the payload to the impl without checking the documented precondition; add a size check before calling into ITensorPushTrackerImpl::push: if payload.size() > max_payload_size() then handle the error (e.g., throw std::length_error or assert/log and return) to prevent oversized payloads from reaching session.get_tracker_impl(*this).push(payload). Ensure you reference TensorPushTracker::push, max_payload_size(), ITrackerSession, and ITensorPushTrackerImpl::push when implementing the check so the validation sits in the public-facing push wrapper.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Duplicate comments:
In `@src/core/deviceio_trackers/cpp/tensor_push_tracker.cpp`:
- Around line 31-34: TensorPushTracker::push currently forwards the payload to
the impl without checking the documented precondition; add a size check before
calling into ITensorPushTrackerImpl::push: if payload.size() >
max_payload_size() then handle the error (e.g., throw std::length_error or
assert/log and return) to prevent oversized payloads from reaching
session.get_tracker_impl(*this).push(payload). Ensure you reference
TensorPushTracker::push, max_payload_size(), ITrackerSession, and
ITensorPushTrackerImpl::push when implementing the check so the validation sits
in the public-facing push wrapper.
ℹ️ Review info
⚙️ Run configuration
Configuration used: Path: .coderabbit.yaml
Review profile: CHILL
Plan: Enterprise
Run ID: 674df2b7-6c6b-4b1a-b19b-0b1691066433
📒 Files selected for processing (42)
examples/haptic_feedback/python/README.mdexamples/haptic_feedback/python/hand_pinch_haptic_example.pysrc/core/deviceio_base/cpp/inc/deviceio_base/haptic_command_reader_tracker_base.hppsrc/core/deviceio_base/cpp/inc/deviceio_base/tensor_push_tracker_base.hppsrc/core/deviceio_trackers/cpp/CMakeLists.txtsrc/core/deviceio_trackers/cpp/haptic_command_reader_tracker.cppsrc/core/deviceio_trackers/cpp/inc/deviceio_trackers/haptic_command_reader_tracker.hppsrc/core/deviceio_trackers/cpp/inc/deviceio_trackers/tensor_push_tracker.hppsrc/core/deviceio_trackers/cpp/tensor_push_tracker.cppsrc/core/deviceio_trackers/python/deviceio_trackers_init.pysrc/core/deviceio_trackers/python/tracker_bindings.cppsrc/core/live_trackers/cpp/CMakeLists.txtsrc/core/live_trackers/cpp/inc/live_trackers/live_deviceio_factory.hppsrc/core/live_trackers/cpp/live_deviceio_factory.cppsrc/core/live_trackers/cpp/live_haptic_command_reader_tracker_impl.cppsrc/core/live_trackers/cpp/live_haptic_command_reader_tracker_impl.hppsrc/core/live_trackers/cpp/live_tensor_push_tracker_impl.cppsrc/core/live_trackers/cpp/live_tensor_push_tracker_impl.hppsrc/core/python/deviceio_init.pysrc/core/replay_trackers/cpp/CMakeLists.txtsrc/core/replay_trackers/cpp/inc/replay_trackers/replay_deviceio_factory.hppsrc/core/replay_trackers/cpp/replay_deviceio_factory.cppsrc/core/replay_trackers/cpp/replay_haptic_command_reader_tracker_impl.cppsrc/core/replay_trackers/cpp/replay_haptic_command_reader_tracker_impl.hppsrc/core/replay_trackers/cpp/replay_tensor_push_tracker_impl.cppsrc/core/replay_trackers/cpp/replay_tensor_push_tracker_impl.hppsrc/core/retargeting_engine/python/tensor_types/__init__.pysrc/core/retargeting_engine/python/tensor_types/indices.pysrc/core/retargeting_engine/python/tensor_types/tactile_types.pysrc/core/retargeting_engine_tests/python/test_haptic_devices.pysrc/core/retargeting_engine_tests/python/test_tactile_retargeters.pysrc/core/schema/fbs/haptic_command.fbssrc/core/schema/python/CMakeLists.txtsrc/core/schema/python/haptic_command_bindings.hsrc/core/schema/python/schema_init.pysrc/core/schema/python/schema_module.cppsrc/haptic_devices/glove.pysrc/haptic_devices/push_tensor.pysrc/plugins/manus/core/inc/manus/manus_glove_collection.hppsrc/plugins/manus/core/inc/manus/manus_hand_tracking_plugin.hppsrc/plugins/manus/core/manus_hand_tracking_plugin.cppsrc/retargeters/tactile_retargeters.py
✅ Files skipped from review due to trivial changes (1)
- src/plugins/manus/core/inc/manus/manus_glove_collection.hpp
🚧 Files skipped from review as they are similar to previous changes (6)
- src/core/replay_trackers/cpp/CMakeLists.txt
- src/core/replay_trackers/cpp/inc/replay_trackers/replay_deviceio_factory.hpp
- src/core/live_trackers/cpp/CMakeLists.txt
- src/core/deviceio_trackers/cpp/CMakeLists.txt
- src/core/retargeting_engine/python/tensor_types/init.py
- src/core/retargeting_engine/python/tensor_types/tactile_types.py
Stacks on the OpenXR controller haptics PR. Adds Manus Metagloves
Pro Haptic as the first glove integration on top of a vendor-neutral
haptic-glove adapter, with a Teleop→plugin transport that actually works
across processes.
Why
The Manus plugin runs in its own process. The earlier revision of this
PR tried to ferry haptic commands over
XR_NV_opaque_data_channel, butthat extension is client → runtime only — the runtime cannot fan
those bytes out to a peer process plugin. The right transport is the
Tensor API (
XR_NVX1_push_tensorproducer +XR_NVX1_tensor_dataconsumer), which the runtime fans across OpenXR sessions on the same
systemIdbycollection_id. The existingGeneric3AxisPedalTrackeralready uses this transport in the opposite (plugin → Teleop) direction.
What
Generic / vendor-neutral surface
src/core/schema/fbs/haptic_glove.fbs): newHapticGloveCommand { is_left: bool; powers: [float]; }wire payloadplus the standard
Tracked/Recordwrappers required bySchemaTracker<RecordT, DataTableT>. MCAP recording off in v1.Generic3AxisPedalTrackerpattern):HapticGloveCommandPusherTracker(producer, pybind-exposed) →LiveHapticGloveCommandPusherTrackerImplwrapscore::SchemaPusher.HapticGloveCommandReaderTracker(consumer, C++-only) →LiveHapticGloveCommandReaderTrackerImplwrapscore::SchemaTracker<HapticGloveCommandRecord, HapticGloveCommand>.ReplaySession.live_deviceio_factory+replay_deviceio_factory;DeviceIOSessionauto-aggregates the required extensions.src/haptic_devices/glove.py):HapticGloveDevice(collection_id, num_fingers, max_payload_size, sides)queues per-side
FingerPowerVectorpayloads;HapticGloveSource(name, device)drains the queue insidepoll_tracker(session), encodes via flatc-generated Python,Summary by CodeRabbit
Release Notes
New Features
Documentation