Knowledge Negative Distillation: Circumventing Overfitting to Unlock More Generalizable Deepfake Detection

The rapid advancement of deepfake technology has led to an increasing frequency of crisis incidents stemming from its misuse. However, existing forgery detection methods often suffer from poor cross-domain generalization due to overfitting to specific forgery cues inherent in their training datasets. Through an in-depth analysis, we identify that low-loss, overfitted features hinder models from capturing broadly applicable patterns necessary for effective generalization. To overcome this limitation, we introduce Knowledge Negative Distillation (KND), a simple yet powerful teacher-student framework, designed to encourage the student model to acquire knowledge beyond the teacher's existing scope. Specifically, we guide the student model to avoid the teacher's overfitted features by maximizing a cross-entropy loss computed from the teacher's probability distributions during the training for the target task. Additionally, we propose an adaptive fusion mechanism that integrates the extensible student features with the teacher's features, weighted and guided by their respective probability distributions. Extensive experimental results validate the superior performance of KND, demonstrating state-of-the-art capabilities across multiple benchmarks. Moreover, the extensibility and universality of KND underscore its potential applicability to a broader range of cross-domain problems characterized by significant overfitting challenges.