Nextmini: A New Research Testbed for Network Emulation and Experimentation

As large language models are trained on datacenters with tens of thousands of compute nodes and are quickly becoming parts of our daily routines, the need for a flexible, easy-to-use, and high-performance research testbed for emulating and experimenting with new network protocols has become more pressing and relevant than ever. Conventional packet-level simulators are, by their nature of discrete-event simulation, not scalable enough; yet traditional network emulation testbeds, such as Mininet, are also showing their age with respect to the flexibility of implementing new algorithms, ability to run arbitrary application workloads, scalability to a large number of nodes, as well as the freedom of expanding beyond a single cluster to span multiple geographically distributed regions.In this paper, we present Nextmini, a modern, next-generation, high-performance networking research testbed for network emulation and experimentation. Implemented in Rust, it is designed from scratch to be as flexible as possible, accommodating a wider array of resource scheduling algorithms. It supports running arbitrary workloads — such as distributed machine learning training workloads — directly on the emulated network. Its design strikes an excellent balance between flexibility and performance, supporting both performant user-space emulation for maximum flexibility, as well as much higher kernel-level performance when users need such a performance boost. It is scalable to a larger number of nodes with ease in the same cluster, and can be easily expanded to span multiple geographically distributed datacenters. We conducted an extensive array of experiments to evaluate Nextmini’s tent-pole features, and to compare it with Mininet. Our results show both Nextmini’s raw power and its abundance of flexibility.