RDMA over Ethernet for Distributed Training at Meta Scale

Adithya Gangidi; Rui Miao; Shengbao Zheng; Sai Jayesh Bondu; Guilherme Loch Waltrick Goes; Hany Morsy; Rohit Puri; Mohammad Riftadi; Ashmitha Jeevaraj Shetty; Jingyi Yang; Shuqiang Zhang; Mikel Jimenez Fernandez; Shashidhar Gandham; Hongyi Zeng

doi:10.1145/3651890.3672233

SIGCOMM2024顶会

RDMA over Ethernet for Distributed Training at Meta Scale

Adithya Gangidi, Rui Miao, Shengbao Zheng, Sai Jayesh Bondu, Guilherme Loch Waltrick Goes, Hany Morsy, Rohit Puri, Mohammad Riftadi, Ashmitha Jeevaraj Shetty, Jingyi Yang, Shuqiang Zhang, Mikel Jimenez Fernandez

DOI 出版方

2024年份

171被引次数

45顶会引用

摘要

The rapid growth in both computational density and scale in AI models in recent years motivates the construction of an efficient and reliable dedicated network infrastructure. This paper presents the design, implementation, and operation of Meta's Remote Direct Memory Access over Converged Ethernet (RoCE) networks for distributed AI training.