SeedPrints: Fingerprints Can Even Tell Which Seed Your Large Language Model Was Trained From
Yao Tong, Haonan Wang, Siquan Li, Kenji Kawaguchi, Tianyang Hu
摘要
Fingerprinting Large Language Models (LLMs) is essential for provenance verification and model attribution. Existing fingerprinting methods are primarily evaluated after fine-tuning, where models have already acquired stable signatures from training data, optimization dynamics, or hyperparameters. However, most of a model's capacity and knowledge are acquired during pretraining rather than downstream fine-tuning, making large-scale pretraining a more fundamental regime for lineage verification. We show that existing fingerprinting methods become unreliable in this regime, as they rely on post-hoc signatures that only emerge after substantial training. This limitation contradicts the classical Galton notion of a fingerprint as an intrinsic and persistent identity. In contrast, we propose a stronger and more intrinsic notion of LLM fingerprinting: SeedPrints, a method that leverages random initialization biases as persistent, seed-dependent identifiers present even before training begins. We show that untrained models exhibit reproducible prediction biases induced by their initialization seed, and that these weak signals remain statistically detectable throughout training, enabling high-confidence lineage verification. Unlike prior techniques that fail during early pretraining or degrade under distribution shifts, SeedPrints remains effective across all training stages, from initialization to large-scale pretraining and downstream adaptation. Experiments on LLaMA-style and Qwen-style models demonstrate seed-level distinguishability and enable birth-to-lifecycle identity verification. Evaluations on large-scale pretraining trajectories and real-world fingerprinting benchmarks further confirm its robustness under prolonged training, domain shifts, and parameter modifications. Together, our results show that initialization itself imprints a unique and persistent identity on LLMs, forming a true "Galtonian" fingerprint. Code is available at https://github.com/YnezT0311/SeedPrints .
问问这篇 Paper
智能体会读完全文。
Lune 把这篇 Paper 索引到了每一个公式,引用它的顶会 Paper 也一样。你提问,回答直接引用原文。
引用它的顶会 Paper1
问问它们各自怎么用它它引用的顶会 Paper9
- Membership Inference Attacks From First PrinciplesNicholas Carlini, Steve Chien, Milad Nasr, Shuang Song 等S&P 2022 · 被引用 1,049 次
- A Watermark for Large Language ModelsJohn Kirchenbauer, Jonas Geiping, Yuxin Wen, Jonathan Katz 等ICML 2023 · 被引用 854 次
- Turning Your Weakness Into a Strength: Watermarking Deep Neural Networks by BackdooringYossi Adi, Carsten Baum, Moustapha Cissé, Benny Pinkas 等USENIX Security 2018 · 被引用 832 次
- Llemma: An Open Language Model for MathematicsZhangir Azerbayev, Hailey Schoelkopf, Keiran Paster, Marco Dos Santos 等ICLR 2024 · 被引用 433 次
- Watermarking Makes Language Models RadioactiveTom Sander, Pierre Fernandez, Alain Durmus, Matthijs Douze 等NeurIPS 2024 · 被引用 68 次
相关 Paper
- Fingerprinting LLMs via Prompt InjectionYuepeng Hu, Zhengyuan Jiang, Mengyuan Li, Osama Ahmed 等ACL 2026 · 被引用 3 次
- Ghost in the Transformer: Detecting Model Reuse with Invariant Spectral SignaturesSuqing Wang, Ziyang Ma, Xinyi Li, Zuchao LiAAAI 2026 · 被引用 1 次
- AWM: Accurate Weight-Matrix Fingerprint for Large Language ModelsBoyi Zeng, Lin Chen, Ziwei He, Xinbing Wang 等ICLR 2026 · 被引用 3 次
- HuRef: HUman-REadable Fingerprint for Large Language ModelsBoyi Zeng, Lizheng Wang, Yuncong Hu, Yi Xu 等NeurIPS 2024 · 被引用 48 次
- Matching Pairs: Attributing Fine-Tuned Models to their Pre-Trained Large Language ModelsMyles Foley, Ambrish Rawat, Taesung Lee, Yufang Hou 等ACL 2023 · 被引用 2 次
