You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Star3 (3)You must be signed in to star a repository
About
[ACM MM 2026] Compass: Degradation-Simulated Reciprocal Learning with Lightweight Needle RWKV for Multimodal Crack Segmentation under Missing Modalities
[ACM MM 2026] Compass: Degradation-Simulated Reciprocal Learning with Lightweight Needle RWKV for Multimodal Crack Segmentation under Missing Modalities
🌟 If this work is useful to you, please give this repository a Star! 🌟
📬 News
2026-11-07: The implementation code of our Compass is now available in this repository. Enjoy!🥰🥰🥰
2026-07-10: 🎉🎉🎉We are delighted to announce that our Compass has been accepted by the ACM MM 2026! The paper and the code will be available soon.
🛠️ Method Overview
In multimodal crack segmentation for industrial facilities, the key challenge is preventing missing modalities from degrading pixel-level performance while maintaining low computational cost. Existing methods struggle to address semantic degradation caused by missing modalities. We propose Compass, a lightweight network for robust crack segmentation under arbitrary missing modalities. Compass comprises Degradation Simulation Distillation (DSD), Needle Block, and Evidential Topology-Preserving Fusion (ETPF). DSD constructs a degradation simulation stream that mimics more severe missing conditions and performs reciprocal distillation with the original stream, decoupling complete perception from degradation adaptation. Within DSD, Feature-Aware Prototype Transmitter (FAPT) performs modality-agnostic prototype-guided feature completion to maintain semantic integrity under incomplete modality conditions. As the lightweight backbone, Needle incorporates crack directional cues into the WKV key modulation through Geometry Conditioned Modulation, and integrates connectivity-aware gating with anisotropic context probing for structure-aware modeling. ETPF fuses multimodal features via Dempster-Shafer evidential combination with uncertainty-gated decoding, preserving crack topology while suppressing unreliable features. Experiments on three datasets demonstrate state-of-the-art (SOTA) performance under diverse missing modality scenarios. Even with 90% depth modality missing on CrackDepth, Compass achieves F1 of 0.8216 and mIoU of 0.8434 with only 2.58M parameters.
🚀 Getting Started
⚙️ Environment Setup
Create and activate the Compass environment using the following commands:
[ACM MM 2026] Compass: Degradation-Simulated Reciprocal Learning with Lightweight Needle RWKV for Multimodal Crack Segmentation under Missing Modalities