PIF-Net: Ill-Posed Prior Guided Multispectral and Hyperspectral Image Fusion via Invertible Mamba and Fusion-Aware LoRA
Baisong Li, Xingwang Wang, Haixiao Xu
Abstract
The goal of multispectral and hyperspectral image fusion (MHIF) is to generate high-quality images that simultaneously possess rich spectral information and fine spatial details. However, due to the inherent trade-off between spectral and spatial information and the limited availability of observations, this task is fundamentally ill-posed. Previous studies have not effectively addressed the ill-posed nature caused by data misalignment. To tackle this challenge, we propose a fusion framework named PIF-Net, which explicitly incorporates ill-posed priors to effectively fuse multispectral images and hyperspectral images. To balance global spectral modeling with computational efficiency, we design a method based on an invertible Mamba architecture that maintains information consistency during feature transformation and reconstruction, ensuring stable gradient flow and process reversibility. Furthermore, we introduce a novel fusion module called the Fusion-Aware Low-Rank Adaptation module, which dynamically calibrates spectral and spatial features while keeping the model lightweight. Extensive experiments on multiple benchmark datasets demonstrate that PIF-Net achieves significantly better image restoration performance than current state-of-the-art methods while maintaining model efficiency.
Ask about this paper
Your agent reads all of it.
Lune indexed this paper to the last equation, along with the top-tier papers that cite it. Ask a question and the answer quotes them.
Your agent calls
Luneget_paper_fulltext
Free to start. No credit card required.
Terminal
Install the CLIlune papers fulltext a1a46db6-5ffe-4e6d-b42c-c9d9d3223d30Builds on7
- A ConvNet for the 2020sZhuang Liu, Hanzi Mao, Chao-Yuan Wu, Christoph Feichtenhofer et al.CVPR 2022 · 6,782 citations
- Structure-Preserving Deraining with Residue Channel Prior GuidanceQiaosi Yi, Juncheng Li, Qinyan Dai, Faming Fang et al.ICCV 2021 · 159 citations
- Convolution Meets LoRA: Parameter Efficient Finetuning for Segment Anything ModelZihan Zhong, Zhiqiang Tang, Tong He, Haoyang Fang et al.ICLR 2024 · 91 citations
- U2Net: A General Framework with Spatial-Spectral-Integrated Double U-Net for Image FusionSiran Peng, Chenhao Guo, Xiao Wu, Liang-Jian DengACM MM 2023 · 43 citations
- InternImage: Exploring Large-Scale Vision Foundation Models with Deformable ConvolutionsWenhai Wang, Jifeng Dai, Zhe Chen, Zhenhang Huang et al.CVPR 2023
Related papers
- Deep Blind Hyperspectral Image FusionWu Wang, Weihong Zeng, Yue Huang, Xinghao Ding et al.ICCV 2019 · 106 citations
- M3SR: Multi-Scale Multi-Perceptual Mamba for Efficient Spectral ReconstructionYuze Zhang, Lingjie Li, Qiuzhen Lin, Zhong Ming et al.AAAI 2026
- Cycle-Consistent Mamba-Based Registration-Fusion Joint Network for Unregistered Hyperspectral Image Super-ResolutionQuangui He, Jiahui Qu, Wenqian Dong, Song Xiao et al.ACM MM 2025
- A Novel State Space Model with Local Enhancement and State Sharing for Image FusionZihan Cao, Xiao Wu, Liang-Jian Deng, Yu ZhongACM MM 2024 · 24 citations
- UniFuse: A Unified All-In-One Framework for Multi-Modal Medical Image Fusion Under Diverse Degradations and MisalignmentsDayong Su, Yafei Zhang, Huafeng Li, Jinxing Li et al.ICCV 2025 · 2 citations
