DynaDojo: An Extensible Platform for Benchmarking Scaling in Dynamical System Identification

Modeling complex dynamical systems poses significant challenges, with traditional methods struggling to work across a variety of systems and scale to highdimensional dynamics. In response, we present DynaDojo, a novel benchmarking platform designed for data-driven dynamical system identification. DynaDojo enables comprehensive evaluation of how an algorithm's performance scales across three key dimensions: (1) the number of training samples provided, (2) the complexity of the dynamical system being modeled, and (3) the training samples required to achieve a target error threshold. Furthermore, DynaDojo enables studying out-ofdistribution generalization (by providing multiple test conditions for each system) and active learning (by supporting closed-loop control). Through its user-friendly and easily extensible API, DynaDojo accommodates a wide range of user-defined Algorithms, Systems, and Challenges (scaling metrics). The platform also prioritizes resource-efficient training for running on a cluster. To showcase its utility, in DynaDojo 0.9, we include implementations of 7 baseline algorithms and 20 dynamical systems, along with many demo notebooks. This work aspires to make DynaDojo a unifying benchmarking platform for system identification, paralleling the role of OpenAI's Gym in reinforcement learning. 1 * equal contribution † corresponding author 1 https://github.com/DynaDojo/dynadojo 37th Conference on Neural Information Processing Systems (NeurIPS 2023) Track on Datasets and Benchmarks.