Katakomba: Tools and Benchmarks for Data-Driven NetHack

NetHack is known as the frontier of reinforcement learning research where learningbased methods still need to catch up to rule-based solutions. One of the promising directions for a breakthrough is using pre-collected datasets similar to recent developments in robotics, recommender systems, and more under the umbrella of offline reinforcement learning (ORL). Recently, a large-scale NetHack dataset was released; while it was a necessary step forward, it has yet to gain wide adoption in the ORL community. In this work, we argue that there are three major obstacles for adoption: resource-wise, implementation-wise, and benchmark-wise. To address them, we develop an open-source library 2 that provides workflow fundamentals familiar to the ORL community: pre-defined D4RL-style tasks, uncluttered baseline implementations, and reliable evaluation tools with accompanying configs and logs synced to the cloud. * Contributed equally. 2 Source code: https://github.com/corl-team/katakomba 3 For a brief introduction to the game, we recommend excellent game wiki, as well as the original publication by Küttler et al. ( 2020 ), which introduced the NetHack environment. 37th Conference on Neural Information Processing Systems (NeurIPS 2023) Track on Datasets and Benchmarks.