ACL Findings2021
Fingerprinting Fine-tuned Language Models in the Wild
Nirav Diwan, Tanmoy Chakraborty, Zubair Shafiq
Abstract
There are concerns that the ability of language models (LMs) to generate high quality synthetic text can be misused to launch spam, disinformation, or propaganda. Therefore, the research community is actively working on developing approaches to detect whether a given text is organic or synthetic. While this is a useful first step, it is important to be able to further fingerprint the author LM to attribute its origin. Prior work on fingerprinting LMs is limited to attributing synthetic text generated by a handful (usually < 10) of pre-trained LMs. However, LMs such as GPT2 are commonly fine-tuned in a myriad of ways (e.g., on a domain-specific text corpus) before being used to generate synthetic text. It is challenging to fingerprinting fine-tuned LMs because the universe of fine-tuned LMs is much larger in realistic scenarios. To address this challenge, we study the problem of large-scale fingerprinting of fine-tuned LMs in the wild. Using a real-world dataset of synthetic text generated by 108 different fine-tuned LMs, we conduct comprehensive experiments to demonstrate the limitations of existing fingerprinting approaches. Our results show that fine-tuning itself is the most effective in attributing the synthetic text generated by fine-tuned LMs.
Ask Lune
Know this paper in minutes, not an afternoon.
Lune can read every page of it and the top-tier papers that cite it, and gives you its claims, its evidence and its limits, each one cited to the paper it comes from. No skimming, no AI slop.
1 top-tier paper has cited it since. Know what they found before your reviewers do.
Lune can draw on
the full paper · the top-tier papers that cite it · related work
Free to start. No credit card required.
Lineage
- Authorship Attribution for Neural Text Generation cites Plug and Play Language Models: A Simple Approach to Controlled Text Generation
- Window-based Membership Inference Attacks Against Fine-tuned Large Language Models cites Language Models are Few-Shot Learners
Cited by top-tier papers1
Builds on7
- Language Models are Few-Shot LearnersTom B. Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah et al.NeurIPS 2020 · 64,255 citations
- BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and ComprehensionMike Lewis, Yinhan Liu, Naman Goyal, Marjan Ghazvininejad et al.ACL 2020 · 1,224 citations
- Plug and Play Language Models: A Simple Approach to Controlled Text GenerationSumanth Dathathri, Andrea Madotto, Janice Lan, Jane Hung et al.ICLR 2020 · 1,166 citations
- Privacy Risks of General-Purpose Language ModelsXudong Pan, Mi Zhang, Shouling Ji, Min YangS&P 2020 · 291 citations
- Authorship Attribution for Neural Text GenerationAdaku Uchendu, Thai Le, Kai Shu, Dongwon LeeEMNLP 2020 · 110 citations
Related papers
- When Detection Fails: The Power of Fine-Tuned Models to Generate Human-Like Social Media TextHillary Dawkins, Kathleen C. Fraser, Svetlana KiritchenkoACL Findings 2025 · 3 citations
- Matching Pairs: Attributing Fine-Tuned Models to their Pre-Trained Large Language ModelsMyles Foley, Ambrish Rawat, Taesung Lee, Yufang Hou et al.ACL 2023 · 2 citations
- AI 'News' Content Farms Are Easy to Make and Hard to Detect: A Case Study in ItalianGiovanni Puccetti, Anna Rogers, Chiara Alzetta, Felice Dell'Orletta et al.ACL 2024 · 1 citation
- WASA: WAtermark-based Source Attribution for Large Language Model-Generated DataXinyang Lu, Jingtan Wang, Zitong Zhao, Zhongxiang Dai et al.ACL Findings 2025
- Responsible Disclosure of Generative Models Using Scalable FingerprintingNing Yu, Vladislav Skripniuk, Dingfan Chen, Larry S. Davis et al.ICLR 2022 · 118 citations
