Forensic-MoE: Exploring Comprehensive Synthetic Image Detection Traces With Mixture of Experts

Recently, synthetic images have evolved incredibly realistic with the development of generative techniques. To avoid the spread of misinformation and identify synthetic content, research on synthetic image detection becomes urgent. Unfortunately, limited to the singular forensic perspective, existing methods struggle to explore sufficient traces encountered with diverse synthetic techniques. In response to this, we argue that different synthetic images encompass a variety of forensic traces, and utilizing multiple experts to explore traces from diverse perspectives will be beneficial. Accordingly, a novel detector with the Mixture of multiple forensic Experts is proposed, named Forensic-MoE. To integrate multiple experts and enhance the knowledge interaction, Forensic-MoE follows an adapter-backbone architecture. Specifically, multiple adapters trained on different synthetic images serve as the trace exploration experts, and they are uniformly integrated into a pretrained backbone model to learn the detection prior and encourage the expert interaction. By guiding multiple experts to align with each other and collaborate, Forensic-MoE can integrate comprehensive and discriminative detection traces from multiple perspectives. Moreover, for the discrimination improvement of each expert, a multi-stage structure is proposed for efficient trace perception, and a patch decentralization strategy is applied to encourage the model's attention on every local region. Code will be available in https: //github.com/fangmq77/Forensic-MoE