Enhanced Pansharpening Via Quaternion Spatial-Spectral Interactions
Dong Liu, Chunhui Luo, Yuanfei Bao, Gang Yang, Jie Xiao, Xueyang Fu, Zheng-Jun Zha
Abstract
However, many existing methods struggle to fully capture spatial and spectral interactions, limiting their effectiveness. To address this, we propose a novel quaternion-based spatialspectral interaction network that enhances pansharpening by leveraging the compact representation capabilities of quaternions for high-dimensional data. Our method consists of three key components: the quaternion global spectral interaction branch, the quaternion local spatial structure awareness branch, and the quaternion spatialspectral interaction branch. The first applies the quaternion Fourier transform to convert multi-channel features into the frequency domain as a whole, enabling global information interaction while preserving inter-channel dependencies, which aids spectral fidelity. The second uses a customized spatial quaternion representation, combined with a window-shifting strategy, to maintain local spatial dependencies while promoting spatial interactions, which helps inject spatial details. The last integrates the two pathways within the quaternion framework to enrich spatialspectral interactions for richer representations. By utilizing quaternion's multi-dimensional representation and parameter-sharing properties, our method achieves a more compact and efficient cross-resolution, multi-band information integration, significantly improving the quality of the fused image. Extensive experiments validate the proposed method's effectiveness and its superior performance over current SOTA techniques. The code is available at https://github.com/dongli8/QuatPanNet.
Ask about this paper
Your agent reads all of it.
Lune indexed this paper to the last equation, along with the top-tier papers that cite it. Ask a question and the answer quotes them.
Builds on10
- Swin Transformer: Hierarchical Vision Transformer using Shifted WindowsZe Liu, Yutong Lin, Yue Cao, Han Hu et al.ICCV 2021 · 31,683 citations
- Fourmer: An Efficient Global Modeling Paradigm for Image RestorationMan Zhou, Jie Huang, Chun-Le Guo, Chongyi LiICML 2023 · 148 citations
- Pan-Sharpening with Customized Transformer and Invertible Neural NetworkMan Zhou, Jie Huang, Yanchi Fang, Xueyang Fu et al.AAAI 2022 · 130 citations
- Mutual Information-driven Pan-sharpeningMan Zhou, Keyu Yan, Jie Huang, Zihe Yang et al.CVPR 2022 · 113 citations
- PanFlowNet: A Flow-Based Deep Network for Pan-sharpeningGang Yang, Xiangyong Cao, Wenzhe Xiao, Man Zhou et al.ICCV 2023 · 43 citations
Related papers
- Revisiting Spatial-Frequency Information Integration from a Hierarchical Perspective for Panchromatic and Multi-Spectral Image FusionJiangtong Tan, Jie Huang, Naishan Zheng, Man Zhou et al.CVPR 2024 · 27 citations
- Adaptively Learning Low-high Frequency Information Integration for Pan-sharpeningMan Zhou, Jie Huang, Chongyi Li, Hu Yu et al.ACM MM 2022 · 44 citations
- Butterworth as Attention: Anisotropic Spectral Gating for PansharpeningZhenggang Wang, Wang Wu, Lianghuazhe, Tai-Xiang JiangICML 2026
- Hierarchical Dual-Domain Fusion with Frequency-Guided Spatial Modeling for Pan-SharpeningHuangqimei Zheng, Chengyi Pan, Qian Jiang, Wei Zhou et al.AAAI 2026
- Domain-irrelevant Feature Learning for Generalizable Pan-sharpeningYunlong Lin, Zhenqi Fu, Ge Meng, Yingying Wang et al.ACM MM 2023 · 11 citations
