A Hybrid Space Model for Misaligned Multi-modality Image Fusion
Yi Xiao, Jia Wang, Zhu Liu, Di Wang, Jinyuan Liu, Risheng Liu
摘要
Infrared and visible image fusion aims to integrate complementary information, such as thermal saliency from infrared imagery and fine-grained texture details from visible imagery. However, real-world multi-modal misalignment and geometric deformation often introduce severe artifacts. Most existing methods focus on feature extraction within Euclidean space, thereby neglecting the inherent hierarchical structures embedded in multimodal representations. While Euclidean space excels at preserving local structural details and supporting efficient computation, hyperbolic space is naturally suited for modeling hierarchical relationships due to its geometric properties. Building upon these observations, this paper proposes a unified framework that jointly optimizes image registration and fusion through a dual-space architecture. This architecture synergistically combines the local fidelity of Euclidean geometry with the hierarchical modeling capability of hyperbolic geometry to enhance multimodal representation learning. Specifically, this paper introduces Hyperbolic Coupled Contrastive Learning Optimization (HCCLO), which aligns and optimizes the hierarchical structures of infrared and visible embeddings in hyperbolic space. Moreover, this paper designs a task-adaptive dual-space features fusion mechanism, which dynamically balances and fuses Euclidean local features with hyperbolic hierarchical representations, thereby improving adaptability for downstream tasks. Extensive experiments on misaligned multimodal datasets demonstrate that our method achieves state-of-the-art performance, while effectively capturing both spatial dependencies and hierarchical semantics.
问问这篇 Paper
智能体会读完全文。
Lune 把这篇 Paper 索引到了每一个公式,引用它的顶会 Paper 也一样。你提问,回答直接引用原文。
它引用的顶会 Paper14
- Target-aware Dual Adversarial Learning and a Multi-scenario Multi-Modality Benchmark to Fuse Infrared and Visible for Object DetectionJinyuan Liu, Xin Fan, Zhanbo Huang, Guanyao Wu 等CVPR 2022 · 被引用 929 次
- Hyperbolic Neural Networks++Ryohei Shimizu, Yusuke Mukuta, Tatsuya HaradaICLR 2021 · 被引用 791 次
- DDFM: Denoising Diffusion Model for Multi-Modality Image FusionZixiang Zhao, Haowen Bai, Yuanzhi Zhu, Jiangshe Zhang 等ICCV 2023 · 被引用 350 次
- Multi-interactive Feature Learning and a Full-time Multi-modality Benchmark for Image Fusion and SegmentationJinyuan Liu, Zhu Liu, Guanyao Wu, Long Ma 等ICCV 2023 · 被引用 287 次
- RFNet: Unsupervised Network for Mutually Reinforcing Multi-modal Image Registration and FusionHan Xu, Jiayi Ma, Jiteng Yuan, Zhuliang Le 等CVPR 2022 · 被引用 161 次
相关 Paper
- Text-Driven Fusion for Infrared and Visible Images: Achieving Image Scene Adaptation on Hyperbolic SpaceHuan Kang, Hui Li, Tianyang Xu, Tao Zhou 等ICML 2026
- FusionRegister: Every Infrared and Visible Image Fusion Deserves RegistrationCongcong Bian, Haolong Ma, Hui Li, Zhongwei Shen 等CVPR 2026 · 被引用 3 次
- LaRA-Fusion: Latent-Robust Adaptation via Dual-Loop Constraints for Infrared and Visible Image FusionYaru Su, Chaowei Huang, Huangbiao Xu, Xiao KeICML 2026
- Hyperbolic Hierarchical Alignment for Video-Based Visible-Infrared Person Re-IdentificationShuang Li, Changjiang Kuang, Jiaxu Leng, Mingpi Tan 等ICML 2026
- DetFusion: A Detection-driven Infrared and Visible Image Fusion NetworkYiming Sun, Bing Cao, Pengfei Zhu, Qinghua HuACM MM 2022 · 被引用 165 次
