CVPR Findings2026
Deep-to-Shallow Knowledge Transfer:Multi-Scale Self-Distillation with Bidirectional Aware for 3D Brain Segmentation
Ziwei Zhang, Dayu Tan, Xin Peng, Weimin Zhong
Abstract
Many CNN-Transformer hybrid models rely on complex global attention mechanisms to capture long-range dependencies, introducing non-linear computational complexity and significant resource consumption. Although techniques like knowledge distillation and sparse attention improve efficiency, they often fall short of achieving the high segmentation accuracy required for complex tasks. Balancing performance with computational efficiency remains a critical challenge. In this work, we propose a novel 3D multi-modal image segmentation framework, termed MSDBA, which integrates bidirectional spatial perception with multi-scale self-distillation. The bidirectional spatial branch captures long-range context across brain regions, while a nonlinear feature extraction mechanism further enhances the model's ability to learn complex patterns. The multi-scale selfdistillation strategy strengthens hierarchical feature representations and improves semantic information transfer at different resolutions. By combining bidirectional spatial perception with multi-scale self-distilled fusion, our framework reduces the bottleneck of quadratic complexity in volumetric segmentation, while addressing the limitation of insufficient global perception. Extensive experiments on benchmark datasets show that MSDBA outperforms stateof-the-art methods in segmentation accuracy, robustness, and generalization, while maintaining high efficiency and scalability.
Ask Lune
Know this paper in minutes, not an afternoon.
Lune can read every page of it and the top-tier papers that cite it, and gives you its claims, its evidence and its limits, each one cited to the paper it comes from. No skimming, no AI slop.
Lineage
Builds on2
Related papers
- Dynamic Self-adaptive Multiscale Distillation from Pre-trained Multimodal Large Model for Efficient Cross-modal RetrievalZhengyang Liang, Meiyu Liang, Wei Huang, Yawen Li et al.ACM MM 2025
- UCAN: Unified Convolutional Attention Network for Expansive Receptive Fields in Lightweight Super-ResolutionCao Thien Tan, Phan Thi Thu Trang, Do Nghiem Duc, Ho Ngoc Anh et al.CVPR 2026 · 2 citations
- Distilled Cross-Combination Transformer for Image Captioning with Dual Refined Visual FeaturesJunbo Hu, Zhixin LiACM MM 2024 · 8 citations
- A Good Student is Cooperative and Reliable: CNN-Transformer Collaborative Learning for Semantic SegmentationJinjing Zhu, Yunhao Luo, Xu Zheng, Hao Wang et al.ICCV 2023 · 49 citations
- A Dynamic Dual-Processing Object Detection Framework Inspired by the Brain's Recognition MechanismMinying Zhang, Tianpeng Bu, Lulu HuICCV 2023 · 3 citations
