Towards Online Spatio-Temporal Prediction: A Knowledge Distillation Driven Continual Learning Approach

Spatio-temporal data prediction is a fundamental task in urban computing, benefiting a variety of real-life applications such as traffic forecasting and environmental monitoring. Due to the dynamic and time-involving nature of spatio-temporal data, researchers have increasingly emphasized online prediction. However, existing approaches (e.g., URCL) typically rely on data-replay strategies, which require storing large volumes of historical data to frequently update their models with new inputs. These methods impose substantial costs, including frequent buffer construction, high storage requirements, and increased training complexity. Furthermore, the single-pass nature of online data, combined with the constrained resources of online environments, highlights the urgent need for more efficient and lightweight solutions for online spatio-temporal prediction. To address these challenges, we propose Storm, a knowledge distillation driven continual learning framework. Storm introduces Dynamic Knowledge Distillation (DKD), leveraging an ever-evolving teacher model to train an effective student model. To optimize efficiency, Storm employs a Mixture-of-Experts (MoE) mechanism, which dynamically switches between the original training mode and the DKD mode. This hybrid design enables low-cost online learning while addressing the stabilityplasticity dilemma. To fully leverage single-pass online data, Storm integrates effective data augmentation methods tailored to the dynamic nature of spatio-temporal data. Moreover, Storm incorporates a Gradual Parameter Freezing (GPF) module to progressively reduce computational costs during online training. Extensive experiments conducted on four real-world datasets, evaluated across short-term, medium-term, and long-term prediction horizons, demonstrate the superiority of Storm. Specifically, Storm: (i) provides a general online training extension for various offline spatio-temporal models, and (ii) achieves remarkable improvements, e.g., up to 14.24% accuracy gains while requiring only 0.3% of the training and inference time compared to the state-of-the-art URCL framework. The source code is publicly available at https://github.com/ZJU-DAILY/Storm.