Hostping: Diagnosing Intra-host Network Bottlenecks in RDMA Servers
Kefei Liu, Zhuo Jiang, Jiao Zhang, Haoran Wei, Xiaolong Zhong, Lizhuang Tan, Tian Pan, Tao Huang
Abstract
Intra-host networking was considered robust in the RDMA (Remote Direct Memory Access) network and received little attention. However, as the RNIC (RDMA NIC) line rate increases rapidly to multi-hundred gigabits, the intra-host network becomes a potential performance bottleneck for network applications. Intra-host network bottlenecks may result in degraded intra-host bandwidth and increased intra-host latency, which can severely impact network performance. However, when intra-host bottlenecks occur, they can hardly be noticed due to the lack of a monitoring system. Furthermore, existing bottleneck diagnosis mechanisms fail to diagnose intra-host bottlenecks efficiently. In this paper, we analyze the symptom of intra-host bottlenecks based on our longterm troubleshooting experience and propose Hostping, the first bottleneck monitoring and diagnosis system dedicated to intra-host networks. The core idea of Hostping is conducting loopback tests between RNICs and endpoints within the host to measure intra-host latency and bandwidth. Hostping not only discovers intra-host bottlenecks we already knew but also reveals six bottlenecks we did not notice before.
Ask about this paper
Your agent reads all of it.
Lune indexed this paper to the last equation, along with the top-tier papers that cite it. Ask a question and the answer quotes them.
Your agent calls
Luneget_paper_fulltext
Free to start. No credit card required.
Terminal
Install the CLIlune papers fulltext ef69fd06-e669-485c-8550-49d0528b74d3Cited by top-tier papers21
- MegaScale: Scaling Large Language Model Training to More Than 10, 000 GPUsZiheng Jiang, Haibin Lin, Yinmin Zhong, Qi Huang et al.NSDI 2024 · 415 citations
- Characterization of Large Language Model Development in the DatacenterQinghao Hu, Zhisheng Ye, Zerui Wang, Guoteng Wang et al.NSDI 2024 · 192 citations
- Crux: GPU-Efficient Communication Scheduling for Deep Learning TrainingJiamin Cao, Yu Guan, Kun Qian, Jiaqi Gao et al.SIGCOMM 2024 · 60 citations
- Minder: Faulty Machine Detection for Large-scale Distributed Model TrainingYangtao Deng, Xiang Shi, Zhuo Jiang, Xingjian Zhang et al.NSDI 2025 · 36 citations
- White-Boxing RDMA with Packet-Granular Software ControlChenxingyu Zhao, Jaehong Min, Ming Liu, Arvind KrishnamurthyNSDI 2025 · 28 citations
Builds on7
- A Unified Architecture for Accelerating Distributed DNN Training in Heterogeneous GPU/CPU ClustersYimin Jiang, Yibo Zhu, Chang Lan, Bairen Yi et al.OSDI 2020 · 390 citations
- When Cloud Storage Meets RDMAYixiao Gao, Qiang Li, Lingbo Tang, Yongqing Xi et al.NSDI 2021 · 228 citations
- Reexamining Direct Cache Access to Optimize I/O Intensive Applications for Multi-hundred-gigabit NetworksAlireza Farshin, Amir Roozbeh, Gerald Q. Maguire Jr., Dejan KosticUSENIX ATC 2020 · 88 citations
- Collie: Finding Performance Anomalies in RDMA SubsystemsXinhao Kong, Yibo Zhu, Huaping Zhou, Zhuo Jiang et al.NSDI 2022 · 86 citations
- How to diagnose nanosecond network latencies in rich end-host stacksRoni Haecki, Radhika Niranjan Mysore, Lalith Suresh, Gerd Zellweger et al.NSDI 2022 · 58 citations
Related papers
- R-Pingmesh: A Service-Aware RoCE Network Monitoring and Diagnostic SystemKefei Liu, Zhuo Jiang, Jiao Zhang, Shixian Guo et al.SIGCOMM 2024 · 21 citations
- Understanding the Host NetworkMidhul Vuppalapati, Saksham Agarwal, Henry Schuh, Baris Kasikci et al.SIGCOMM 2024 · 25 citations
- Towards Lightweight Traffic Forecasting in RDMA Networks: Design and ApplicationCheng Yang, Xiaoning Zhang, Bodong Yan, Sun Xu et al.INFOCOM 2025 · 1 citation
- INSERT: In-Network Stateful End-to-End RDMA TelemetryHyunseok Chang, Walid A. Hanafy, Sarit Mukherjee, Limin WangINFOCOM 2024 · 3 citations
- Software-based Live Migration for RDMAXiaoyu Li, Ran Shu, Yongqiang Xiong, Fengyuan RenSIGCOMM 2025 · 4 citations
