COOM: A Game Benchmark for Continual Reinforcement Learning

The advancement of continual reinforcement learning (RL) has been facing various obstacles, including standardized metrics and evaluation protocols, demanding computational requirements, and a lack of widely accepted standard benchmarks. In response to these challenges, we present COOM (Continual DOOM), a continual RL benchmark tailored for embodied pixel-based RL. COOM presents a meticulously crafted suite of task sequences set within visually distinct 3D environments, serving as a robust evaluation framework to assess crucial aspects of continual RL, such as catastrophic forgetting, knowledge transfer, and sample-efficient learning. Following an in-depth empirical evaluation of popular continual learning (CL) methods, we pinpoint their limitations, provide valuable insight into the benchmark and highlight unique algorithmic challenges. This makes our work the first to benchmark image-based CRL in 3D environments with embodied perception. The primary objective of the COOM benchmark is to offer the research community a valuable and cost-effective challenge. It seeks to deepen our comprehension of the capabilities and limitations of current and forthcoming CL methods in an RL setting. The code and environments are open-sourced and accessible on GitHub. Related work Most recent eminent CL platforms and frameworks, such as Sequoia [47] , Avalanche [38], and Continuum [16] learn from static data sets of fixed sizes [35] , and are thus not predominantly intended for RL. In CRL, the data sequence consists of different environments, and data samples are obtained through the agent-environment interaction. Sequoia introduces metrics and baselines aimed at CRL but only consists of simple environments like state-based manipulation tasks from