The Overlooked Matters: Revisiting Background, Prototype, and Activation in Few-Shot Medical Image Segmentation

In few-shot medical image segmentation, most existing methods focus heavily on learning explicit correlations between support and query sets, often overlooking the core demands of the segmentation task itself. In this work, we identify three overlooked yet critical issues that limit current performance: the diversity of background distributions, the degradation of support prototypes, and the over-activation of irrelevant regions. To address these challenges, we propose a novel framework with three lightweight and adaptive modules. First, a background self-distillation module acts as a self-attention-driven agent to cluster and aggregate diverse background features, generating multiple sub-prototypes that enhance foreground-background separation. Second, we introduce a prototype self-anchoring mechanism that leverages a dual-branch correlation mapping and reverse supervision to stabilize support prototype learning and prevent feature degradation. Third, an activation self-calibration module identifies over-activated residuals and applies test-time channel manipulation to suppress noisy activations without additional training. Extensive experiments on standard few-shot medical segmentation benchmarks demonstrate the superiority of our approach over state-of-the-art methods. Our findings suggest that performance gains come not only from better support-query alignment, but also from rethinking and addressing the often neglected aspects of few-shot segmentation.