Lyapunov-Stable Adaptive Control for Multimodal Concept Drift

Multimodal learning systems often struggle in non-stationary environments due to concept drift, where changing data distributions can degrade performance. Modality-specific drifts and the lack of mechanisms for continuous, stable adaptation compound this challenge. This paper introduces LS-OGD, a novel adaptive control framework for robust multimodal learning in the presence of concept drift. LS-OGD uses an online controller that dynamically adjusts the model's learning rate and the fusion weights between different data modalities in response to detected drift and evolving prediction errors. We prove that under bounded drift conditions, the LS-OGD system's prediction error is uniformly ultimately bounded and converges to zero if the drift ceases. Additionally, we demonstrate that the adaptive fusion strategy effectively isolates and mitigates the impact of severe modality-specific drift, thereby ensuring system resilience and fault tolerance. These theoretical guarantees establish a principled foundation for developing reliable and continuously adapting multimodal learning systems.

Recent advancements in deep learning have spurred research into concept drift in neural networks and large models. While continual learning and concept drift address learning from non-stationary data, continual learning typically prevents catastrophic forgetting of previous tasks. In contrast, concept drift emphasizes rapid adaptation to evolving data. Some studies, like [38], propose extending concept drift theory to multimodal language models, recognizing their vulnerability to gradual shifts and sudden out-of-distribution events. During pre-training, they introduced a drift adapter module to address drift, highlighting a growing focus on this issue in complex AI systems. However, their approach primarily targets pre-training bias rather than real-time adaptation. Our work stands apart by addressing online drift during deployment while ensuring stability.