Entropy-aware Span-Constrained Optimal Transport for Robust Cross-Tokenizer Knowledge Distillation

Existing Cross-Tokenizer Knowledge Distillation (CTKD) methods can fail to outperform simple supervised fine-tuning when vocabulary overlap is low due to severe alignment noise. We identify this phenomenon as the ``Low-Overlap negative transfer regime'' . To overcome this, we propose Entropy-aware Span-Constrained Optimal Transport (E-SCOT) , a robust framework that treats distillation as a sparse transport problem built upon a vocabulary-agnostic ground metric. Unlike prior OT approaches that incur quadratic costs via dense sequence-level optimization, E-SCOT employs span-anchored lexical alignment to construct a deterministic, locality-preserving support set in linear time with respect to sequence length. Furthermore, we introduce Rényi-entropy adaptive reweighting to dynamically concentrate the distillation budget on informative positions exhibiting significant uncertainty-profile gaps. Extensive experiments demonstrate that E-SCOT achieves state-of-the-art performance across diverse model families, effectively eliminating negative transfer even in challenging low-overlap scenarios.