Lune

CVPR Findings2026

Deep-to-Shallow Knowledge Transfer:Multi-Scale Self-Distillation with Bidirectional Aware for 3D Brain Segmentation

Ziwei Zhang, Dayu Tan, Xin Peng, Weimin Zhong

Abstract

Many CNN-Transformer hybrid models rely on complex global attention mechanisms to capture long-range dependencies, introducing non-linear computational complexity and significant resource consumption. Although techniques like knowledge distillation and sparse attention improve efficiency, they often fall short of achieving the high segmentation accuracy required for complex tasks. Balancing performance with computational efficiency remains a critical challenge. In this work, we propose a novel 3D multi-modal image segmentation framework, termed MSDBA, which integrates bidirectional spatial perception with multi-scale self-distillation. The bidirectional spatial branch captures long-range context across brain regions, while a nonlinear feature extraction mechanism further enhances the model's ability to learn complex patterns. The multi-scale selfdistillation strategy strengthens hierarchical feature representations and improves semantic information transfer at different resolutions. By combining bidirectional spatial perception with multi-scale self-distilled fusion, our framework reduces the bottleneck of quadratic complexity in volumetric segmentation, while addressing the limitation of insufficient global perception. Extensive experiments on benchmark datasets show that MSDBA outperforms stateof-the-art methods in segmentation accuracy, robustness, and generalization, while maintaining high efficiency and scalability.

Ask Lune

Know this paper in minutes, not an afternoon.

Lune can read every page of it and the top-tier papers that cite it, and gives you its claims, its evidence and its limits, each one cited to the paper it comes from. No skimming, no AI slop.

Questions to start from

Lune can draw on

the full paper · related work

Get the answer

Free to start. No credit card required.

Lineage

Builds onCited byLeft to right by year; the more cited, the higher.

Builds on2

  1. 3D UX-Net: A Large Kernel Volumetric ConvNet Modernizing Hierarchical Transformer for Medical Image SegmentationHo Hin Lee, Shunxing Bao, Yuankai Huo, Bennett A. LandmanICLR 2023 · 100 citations
  2. KAN: Kolmogorov-Arnold NetworksZiming Liu, Yixuan Wang, Sachin Vaidya, Fabian Ruehle et al.ICLR 2025

Related papers

  1. Dynamic Self-adaptive Multiscale Distillation from Pre-trained Multimodal Large Model for Efficient Cross-modal RetrievalZhengyang Liang, Meiyu Liang, Wei Huang, Yawen Li et al.ACM MM 2025
  2. UCAN: Unified Convolutional Attention Network for Expansive Receptive Fields in Lightweight Super-ResolutionCao Thien Tan, Phan Thi Thu Trang, Do Nghiem Duc, Ho Ngoc Anh et al.CVPR 2026 · 2 citations
  3. Distilled Cross-Combination Transformer for Image Captioning with Dual Refined Visual FeaturesJunbo Hu, Zhixin LiACM MM 2024 · 8 citations
  4. A Good Student is Cooperative and Reliable: CNN-Transformer Collaborative Learning for Semantic SegmentationJinjing Zhu, Yunhao Luo, Xu Zheng, Hao Wang et al.ICCV 2023 · 49 citations
  5. A Dynamic Dual-Processing Object Detection Framework Inspired by the Brain's Recognition MechanismMinying Zhang, Tianpeng Bu, Lulu HuICCV 2023 · 3 citations
Dusk over the sea between two cliffs drawn in fine vertical lines