Implicit Identity Driven Deepfake Face Swapping Detection
Baojin Huang, Zhongyuan Wang, Jifan Yang, Jiaxin Ai, Qin Zou, Qian Wang, Dengpan Ye
Abstract
In this paper, we consider the face swapping detection from the perspective of face identity. Face swapping aims to replace the target face with the source face and generate the fake face that the human cannot distinguish between real and fake. We argue that the fake face contains the explicit identity and implicit identity, which respectively corresponds to the identity of the source face and target face during face swapping. Note that the explicit identities of faces can be extracted by regular face recognizers. Particularly, the implicit identity of real face is consistent with the its explicit identity. Thus the difference between explicit and implicit identity of face facilitates face swapping detection. Following this idea, we propose a novel implicit identity driven framework for face swapping detection. Specifically, we design an explicit identity contrast (EIC) loss and an implicit identity exploration (IIE) loss, which supervises a CNN backbone to embed face images into the implicit identity space. Under the guidance of EIC, real samples are pulled closer to their explicit identities, while fake samples are pushed away from their explicit identities. Moreover, IIE is derived from the margin-based classification loss function, which encourages the fake faces with known target identities to enjoy intra-class compactness and interclass diversity. Extensive experiments and visualizations on several datasets demonstrate the generalization of our method against the state-of-the-art counterparts.
Ask about this paper
Your agent reads all of it.
Lune indexed this paper to the last equation, along with the top-tier papers that cite it. Ask a question and the answer quotes them.
Cited by top-tier papers50
- Exposing the Deception: Uncovering More Forgery Clues for Deepfake DetectionZhongjie Ba, Qingyu Liu, Zhenguang Liu, Shuang Wu et al.AAAI 2024 · 101 citations
- Can We Leave Deepfake Data Behind in Training Deepfake Detector?Jikang Cheng, Zhiyuan Yan, Ying Zhang, Yuhao Luo et al.NeurIPS 2024 · 85 citations
- DiffusionFake: Enhancing Generalization in Deepfake Detection via Guided Stable DiffusionKe Sun, Shen Chen, Taiping Yao, Hong Liu et al.NeurIPS 2024 · 57 citations
- Locate and Verify: A Two-Stream Network for Improved Deepfake DetectionChao Shuai, Jieming Zhong, Shuang Wu, Feng Lin et al.ACM MM 2023 · 52 citations
- AVFF: Audio-Visual Feature Fusion for Video Deepfake DetectionTrevine Oorloff, Surya Koppisetti, Nicolò Bonettini, Divyaraj Solanki et al.CVPR 2024 · 51 citations
Builds on24
- FaceForensics++: Learning to Detect Manipulated Facial ImagesAndreas Rössler, Davide Cozzolino, Luisa Verdoliva, Christian Riess et al.ICCV 2019 · 2,966 citations
- Leveraging Frequency Analysis for Deep Fake Image RecognitionJoel Frank, Thorsten Eisenhofer, Lea Schönherr, Asja Fischer et al.ICML 2020 · 848 citations
- Everybody Dance NowCaroline Chan, Shiry Ginosar, Tinghui Zhou, Alexei A. EfrosICCV 2019 · 840 citations
- FSGAN: Subject Agnostic Face Swapping and ReenactmentYuval Nirkin, Yosi Keller, Tal HassnerICCV 2019 · 710 citations
- WildDeepfake: A Challenging Real-World Dataset for Deepfake DetectionBojia Zi, Minghao Chang, Jingjing Chen, Xingjun Ma et al.ACM MM 2020 · 443 citations
Related papers
- Information Bottleneck Disentanglement for Identity SwappingGege Gao, Huaibo Huang, Chaoyou Fu, Zhaoyang Li et al.CVPR 2021
- Smooth-Swap: A Simple Enhancement for Face-Swapping with SmoothnessJiseob Kim, Jihoon Lee, Byoung-Tak ZhangCVPR 2022
- BlendFace: Re-designing Identity Encoders for Face-SwappingKaede Shiohara, Xingchao Yang, Takafumi TaketomiICCV 2023 · 83 citations
- SimSwap: An Efficient Framework For High Fidelity Face SwappingRenwang Chen, Xuanhong Chen, Bingbing Ni, Yanhao GeACM MM 2020 · 409 citations
- 3D-Aware Face SwappingYixuan Li, Chao Ma, Yichao Yan, Wenhan Zhu et al.CVPR 2023
