Lune

ACL Findings2022

Training Text-to-Text Transformers with Privacy Guarantees

Natalia Ponomareva, Jasmijn Bastings, Sergei Vassilvitskii

Abstract

Recent advances in NLP often stem from large transformer-based pre-trained models, which rapidly grow in size and use more and more training data. Such models are often released to the public so that end users can fine-tune them on a task dataset. While it is common to treat pre-training data as public, it may still contain personally identifiable information (PII), such as names, phone numbers, and copyrighted material. Recent findings show that the capacity of these models allows them to memorize parts of the training data, and suggest differentially private (DP) training as a potential mitigation. While there is recent work on DP fine-tuning of NLP models, the effects of DP pre-training are less well understood: it is not clear how downstream performance is affected by DP pre-training, and whether DP pre-training mitigates some of the memorization concerns. We focus on T5 and show that by using recent advances in JAX and XLA we can train models with DP that do not suffer a large drop in pre-training utility, nor in training speed, and can still be fine-tuned to high accuracy on downstream tasks (e.g. GLUE). Moreover, we show that T5's span corruption is a good defense against data memorization.

Ask Lune

Know this paper in minutes, not an afternoon.

Lune can read every page of it and the top-tier papers that cite it, and gives you its claims, its evidence and its limits, each one cited to the paper it comes from. No skimming, no AI slop.

7 top-tier papers have cited it since. Know what they found before your reviewers do.

Questions to start from

Lune can draw on

the full paper · the top-tier papers that cite it · related work

Get the answer

Free to start. No credit card required.

Lineage

  • LazyDP: Co-Designing Algorithm-Software for Scalable Training of Differentially Private Recommendation Models cites Large Language Models Can Be Strong Differentially Private Learners
  • LazyDP: Co-Designing Algorithm-Software for Scalable Training of Differentially Private Recommendation Models cites Deep Learning with Differential Privacy
  • LazyDP: Co-Designing Algorithm-Software for Scalable Training of Differentially Private Recommendation Models cites Extracting Training Data from Large Language Models
  • Privacy Side Channels in Machine Learning Systems cites Deep Learning with Differential Privacy
  • Privacy Side Channels in Machine Learning Systems cites Deduplicating Training Data Makes Language Models Better
  • Privacy Side Channels in Machine Learning Systems cites Extracting Training Data from Large Language Models
  • Differentially Private Learning Needs Better Features (or Much More Data) cites Deep Learning with Differential Privacy
  • TensorTEE: Unifying Heterogeneous TEE Granularity for Efficient Secure Collaborative Tensor Computing cites Large Language Models Can Be Strong Differentially Private Learners
  • Large Language Models Can Be Strong Differentially Private Learners cites Differentially Private Learning Needs Better Features (or Much More Data)
  • Large Language Models Can Be Strong Differentially Private Learners cites Deep Learning with Differential Privacy
  • Large Language Models Can Be Strong Differentially Private Learners cites Extracting Training Data from Large Language Models
  • On Provable Copyright Protection for Generative Models cites Differentially Private Learning Needs Better Features (or Much More Data)
  • On Provable Copyright Protection for Generative Models cites Large Language Models Can Be Strong Differentially Private Learners
  • On Provable Copyright Protection for Generative Models cites Deduplicating Training Data Makes Language Models Better
  • On Provable Copyright Protection for Generative Models cites Extracting Training Data from Large Language Models
  • Deduplicating Training Data Makes Language Models Better cites Extracting Training Data from Large Language Models
  • Extracting Training Data from Large Language Models cites Deep Learning with Differential Privacy
Builds onCited byLeft to right by year; the more cited, the higher.

Cited by top-tier papers7

  1. On Provable Copyright Protection for Generative ModelsNikhil Vyas, Sham M. Kakade, Boaz BarakICML 2023 · 120 citations
  2. Privacy Side Channels in Machine Learning SystemsEdoardo Debenedetti, Giorgio Severi, Milad Nasr, Christopher A. Choquette-Choo et al.USENIX Security 2024 · 52 citations
  3. TextFusion: Privacy-Preserving Pre-trained Model Inference via Token FusionXin Zhou, Jinzhu Lu, Tao Gui, Ruotian Ma et al.EMNLP 2022 · 12 citations
  4. TensorTEE: Unifying Heterogeneous TEE Granularity for Efficient Secure Collaborative Tensor ComputingHusheng Han, Xinyao Zheng, Yuanbo Wen, Yifan Hao et al.ASPLOS 2024 · 12 citations
  5. LazyDP: Co-Designing Algorithm-Software for Scalable Training of Differentially Private Recommendation ModelsJuntaek Lim, Youngeun Kwon, Ranggi Hwang, Kiwan Maeng et al.ASPLOS 2024 · 3 citations

Builds on11

  1. Deep Learning with Differential PrivacyMartín Abadi, Andy Chu, Ian J. Goodfellow, H. Brendan McMahan et al.CCS 2016 · 7,620 citations
  2. Extracting Training Data from Large Language ModelsNicholas Carlini, Florian Tramèr, Eric Wallace, Matthew Jagielski et al.USENIX Security 2021 · 2,866 citations
  3. Deduplicating Training Data Makes Language Models BetterKatherine Lee, Daphne Ippolito, Andrew Nystrom, Chiyuan Zhang et al.ACL 2022 · 844 citations
  4. Large Language Models Can Be Strong Differentially Private LearnersXuechen Li, Florian Tramèr, Percy Liang, Tatsunori HashimotoICLR 2022 · 502 citations
  5. Differentially Private Learning Needs Better Features (or Much More Data)Florian Tramèr, Dan BonehICLR 2021 · 325 citations

Related papers

  1. Private Memorization Editing: Turning Memorization into a Defense to Strengthen Data Privacy in Large Language ModelsElena Sofia Ruzzetti, Giancarlo A. Xompero, Davide Venditti, Fabio Massimo ZanzottoACL 2025 · 9 citations
  2. Differentially Private Model CompressionFatemehsadat Mireshghallah, Arturs Backurs, Huseyin A. Inan, Lukas Wutschitz et al.NeurIPS 2022 · 18 citations
  3. Pre-training Differentially Private Models with Limited Public DataZhiqi Bu, Xinwei Zhang, Sheng Zha, Mingyi Hong et al.NeurIPS 2024 · 9 citations
  4. Differentially Private Fine-tuning of Language ModelsDa Yu, Saurabh Naik, Arturs Backurs, Sivakanth Gopi et al.ICLR 2022 · 494 citations
  5. Exploring Memorization in Fine-tuned Language ModelsShenglai Zeng, Yaxin Li, Jie Ren, Yiding Liu et al.ACL 2024
Dusk over the sea between two cliffs drawn in fine vertical lines