PIF-Net: Ill-Posed Prior Guided Multispectral and Hyperspectral Image Fusion via Invertible Mamba and Fusion-Aware LoRA
Baisong Li, Xingwang Wang, Haixiao Xu
摘要
The goal of multispectral and hyperspectral image fusion (MHIF) is to generate high-quality images that simultaneously possess rich spectral information and fine spatial details. However, due to the inherent trade-off between spectral and spatial information and the limited availability of observations, this task is fundamentally ill-posed. Previous studies have not effectively addressed the ill-posed nature caused by data misalignment. To tackle this challenge, we propose a fusion framework named PIF-Net, which explicitly incorporates ill-posed priors to effectively fuse multispectral images and hyperspectral images. To balance global spectral modeling with computational efficiency, we design a method based on an invertible Mamba architecture that maintains information consistency during feature transformation and reconstruction, ensuring stable gradient flow and process reversibility. Furthermore, we introduce a novel fusion module called the Fusion-Aware Low-Rank Adaptation module, which dynamically calibrates spectral and spatial features while keeping the model lightweight. Extensive experiments on multiple benchmark datasets demonstrate that PIF-Net achieves significantly better image restoration performance than current state-of-the-art methods while maintaining model efficiency.
问问这篇 Paper
智能体会读完全文。
Lune 把这篇 Paper 索引到了每一个公式,引用它的顶会 Paper 也一样。你提问,回答直接引用原文。
它引用的顶会 Paper7
- A ConvNet for the 2020sZhuang Liu, Hanzi Mao, Chao-Yuan Wu, Christoph Feichtenhofer 等CVPR 2022 · 被引用 6,782 次
- Structure-Preserving Deraining with Residue Channel Prior GuidanceQiaosi Yi, Juncheng Li, Qinyan Dai, Faming Fang 等ICCV 2021 · 被引用 159 次
- Convolution Meets LoRA: Parameter Efficient Finetuning for Segment Anything ModelZihan Zhong, Zhiqiang Tang, Tong He, Haoyang Fang 等ICLR 2024 · 被引用 91 次
- U2Net: A General Framework with Spatial-Spectral-Integrated Double U-Net for Image FusionSiran Peng, Chenhao Guo, Xiao Wu, Liang-Jian DengACM MM 2023 · 被引用 43 次
- InternImage: Exploring Large-Scale Vision Foundation Models with Deformable ConvolutionsWenhai Wang, Jifeng Dai, Zhe Chen, Zhenhang Huang 等CVPR 2023
相关 Paper
- Deep Blind Hyperspectral Image FusionWu Wang, Weihong Zeng, Yue Huang, Xinghao Ding 等ICCV 2019 · 被引用 106 次
- M3SR: Multi-Scale Multi-Perceptual Mamba for Efficient Spectral ReconstructionYuze Zhang, Lingjie Li, Qiuzhen Lin, Zhong Ming 等AAAI 2026
- Cycle-Consistent Mamba-Based Registration-Fusion Joint Network for Unregistered Hyperspectral Image Super-ResolutionQuangui He, Jiahui Qu, Wenqian Dong, Song Xiao 等ACM MM 2025
- A Novel State Space Model with Local Enhancement and State Sharing for Image FusionZihan Cao, Xiao Wu, Liang-Jian Deng, Yu ZhongACM MM 2024 · 被引用 24 次
- UniFuse: A Unified All-In-One Framework for Multi-Modal Medical Image Fusion Under Diverse Degradations and MisalignmentsDayong Su, Yafei Zhang, Huafeng Li, Jinxing Li 等ICCV 2025 · 被引用 2 次
