ICLR2025

NetMoE: Accelerating MoE Training through Dynamic Sample Placement

Xinyi Liu, Yujie Wang, Fangcheng Fu, Xupeng Miao, Shenhan Zhu, Xiaonan Nie, Bin Cui

2025年份

摘要

Motivation: Recent studies have demonstrated that expert routing exhibits a certain degree of data locality. To be specific, input tokens may have distinct preferences for experts, and the corresponding distribution is often skewed