EDDI: Explaining Data Drift Using Influence

Data drift poses a significant challenge to the reliability of ML models in real-world applications, as the data distribution may change between training and inference phases. Existing drift monitoring systems have several limitations: (i) they detect either model drift or feature drift but not both; (ii) they rely on a plethora of detection methods, each making its own assumptions regarding the data and the ML models; (iii) they do not explain a drift in relation to the underlying probability distribution. To address these limitations, we propose EDDI, a novel influence-based drift detection and explanation framework that leverages the direct influence of samples on the decision boundary of the deployed predictive model. EDDI represents the first unified framework that utilizes Influence Functions to detect both model and feature drift. At the same time, EDDI provides a novel kind of explanation by revealing the probabilistic source of a drift. The key idea is that the influence distributions between drifted and non-drifted samples differ. Moreover, different drift types exhibit unique distributional influence characteristics, enabling their attribution. Based on a type-specific drift simulation, it is demonstrated that EDDI achieves statistically significant improvements on detection performance up to 15% over baselines across four drift types on 42 time-series classification datasets and reveals the drift type with up to 0.8 AUC. Moreover, we evaluated EDDI on two datasets with naturally occurring drift.