A Hybrid Space Model for Misaligned Multi-modality Image Fusion
Yi Xiao, Jia Wang, Zhu Liu, Di Wang, Jinyuan Liu, Risheng Liu
Abstract
Infrared and visible image fusion aims to integrate complementary information, such as thermal saliency from infrared imagery and fine-grained texture details from visible imagery. However, real-world multi-modal misalignment and geometric deformation often introduce severe artifacts. Most existing methods focus on feature extraction within Euclidean space, thereby neglecting the inherent hierarchical structures embedded in multimodal representations. While Euclidean space excels at preserving local structural details and supporting efficient computation, hyperbolic space is naturally suited for modeling hierarchical relationships due to its geometric properties. Building upon these observations, this paper proposes a unified framework that jointly optimizes image registration and fusion through a dual-space architecture. This architecture synergistically combines the local fidelity of Euclidean geometry with the hierarchical modeling capability of hyperbolic geometry to enhance multimodal representation learning. Specifically, this paper introduces Hyperbolic Coupled Contrastive Learning Optimization (HCCLO), which aligns and optimizes the hierarchical structures of infrared and visible embeddings in hyperbolic space. Moreover, this paper designs a task-adaptive dual-space features fusion mechanism, which dynamically balances and fuses Euclidean local features with hyperbolic hierarchical representations, thereby improving adaptability for downstream tasks. Extensive experiments on misaligned multimodal datasets demonstrate that our method achieves state-of-the-art performance, while effectively capturing both spatial dependencies and hierarchical semantics.
Ask about this paper
Your agent reads all of it.
Lune indexed this paper to the last equation, along with the top-tier papers that cite it. Ask a question and the answer quotes them.
Your agent calls
Luneget_paper_fulltext
Free to start. No credit card required.
Terminal
Install the CLIlune papers fulltext 579be445-9ed2-492a-b3d6-eb5b7eca24c3Builds on14
- Target-aware Dual Adversarial Learning and a Multi-scenario Multi-Modality Benchmark to Fuse Infrared and Visible for Object DetectionJinyuan Liu, Xin Fan, Zhanbo Huang, Guanyao Wu et al.CVPR 2022 · 929 citations
- Hyperbolic Neural Networks++Ryohei Shimizu, Yusuke Mukuta, Tatsuya HaradaICLR 2021 · 791 citations
- DDFM: Denoising Diffusion Model for Multi-Modality Image FusionZixiang Zhao, Haowen Bai, Yuanzhi Zhu, Jiangshe Zhang et al.ICCV 2023 · 350 citations
- Multi-interactive Feature Learning and a Full-time Multi-modality Benchmark for Image Fusion and SegmentationJinyuan Liu, Zhu Liu, Guanyao Wu, Long Ma et al.ICCV 2023 · 287 citations
- RFNet: Unsupervised Network for Mutually Reinforcing Multi-modal Image Registration and FusionHan Xu, Jiayi Ma, Jiteng Yuan, Zhuliang Le et al.CVPR 2022 · 161 citations
Related papers
- Text-Driven Fusion for Infrared and Visible Images: Achieving Image Scene Adaptation on Hyperbolic SpaceHuan Kang, Hui Li, Tianyang Xu, Tao Zhou et al.ICML 2026
- FusionRegister: Every Infrared and Visible Image Fusion Deserves RegistrationCongcong Bian, Haolong Ma, Hui Li, Zhongwei Shen et al.CVPR 2026 · 3 citations
- LaRA-Fusion: Latent-Robust Adaptation via Dual-Loop Constraints for Infrared and Visible Image FusionYaru Su, Chaowei Huang, Huangbiao Xu, Xiao KeICML 2026
- Hyperbolic Hierarchical Alignment for Video-Based Visible-Infrared Person Re-IdentificationShuang Li, Changjiang Kuang, Jiaxu Leng, Mingpi Tan et al.ICML 2026
- DetFusion: A Detection-driven Infrared and Visible Image Fusion NetworkYiming Sun, Bing Cao, Pengfei Zhu, Qinghua HuACM MM 2022 · 165 citations
