EMNLP Findings2022
Finding Memo: Extractive Memorization in Constrained Sequence Generation Tasks
Abstract
Memorization presents a challenge for several constrained Natural Language Generation (NLG) tasks such as Neural Machine Translation (NMT), wherein the proclivity of neural models to memorize noisy and atypical samples reacts adversely with the noisy (web crawled) datasets. However, previous studies of memorization in constrained NLG tasks have only focused on counterfactual memorization, linking it to the problem of hallucinations. In this work, we propose a new, inexpensive algorithm for extractive memorization (exact training data generation under insufficient context) in constrained sequence generation tasks and use it to study extractive memorization and its effects in NMT. We demonstrate that extractive memorization poses a serious threat to NMT reliability by qualitatively and quantitatively characterizing the memorized samples as well as the model behavior in their vicinity. Based on empirical observations, we develop a simple algorithm which elicits non-memorized translations of memorized samples from the same model, for a large fraction of such samples. Finally, we show that the proposed algorithm could also be leveraged to mitigate memorization in the model through finetuning. We have released the code to reproduce our results at https://github.com/vyraun/Finding-Memo.
Ask Lune
Know this paper in minutes, not an afternoon.
Lune can read every page of it and the top-tier papers that cite it, and gives you its claims, its evidence and its limits, each one cited to the paper it comes from. No skimming, no AI slop.
6 top-tier papers have cited it since. Know what they found before your reviewers do.
Lune can draw on
the full paper · the top-tier papers that cite it · related work
Free to start. No credit card required.
Lineage
- Quantifying Memorization Across Neural Language Models cites What Neural Networks Memorize and Why: Discovering the Long Tail via Influence Estimation
- Quantifying Memorization Across Neural Language Models cites Extracting Training Data from Large Language Models
- Memorization Without Overfitting: Analyzing the Training Dynamics of Large Language Models cites Quantifying Memorization Across Neural Language Models
- Memorization Without Overfitting: Analyzing the Training Dynamics of Large Language Models cites What Neural Networks Memorize and Why: Discovering the Long Tail via Influence Estimation
- Memorization Without Overfitting: Analyzing the Training Dynamics of Large Language Models cites Extracting Training Data from Large Language Models
- Memorisation Cartography: Mapping out the Memorisation-Generalisation Continuum in Neural Machine Translation cites Quantifying Memorization Across Neural Language Models
- Memorisation Cartography: Mapping out the Memorisation-Generalisation Continuum in Neural Machine Translation cites Memorization Without Overfitting: Analyzing the Training Dynamics of Large Language Models
- Memorisation Cartography: Mapping out the Memorisation-Generalisation Continuum in Neural Machine Translation cites What Neural Networks Memorize and Why: Discovering the Long Tail via Influence Estimation
- Memorisation Cartography: Mapping out the Memorisation-Generalisation Continuum in Neural Machine Translation cites Extracting Training Data from Large Language Models
- ByGPT5: End-to-End Style-conditioned Poetry Generation with Token-free Language Models cites Quantifying Memorization Across Neural Language Models
- AutomaTikZ: Text-Guided Synthesis of Scientific Vector Graphics with TikZ cites Quantifying Memorization Across Neural Language Models
- AutomaTikZ: Text-Guided Synthesis of Scientific Vector Graphics with TikZ cites ByGPT5: End-to-End Style-conditioned Poetry Generation with Token-free Language Models
- Extracting Training Data from Large Language Models cites What Neural Networks Memorize and Why: Discovering the Long Tail via Influence Estimation
Cited by top-tier papers6
- The Unreasonable Effectiveness of Few-shot Learning for Machine TranslationXavier Garcia, Yamini Bansal, Colin Cherry, George F. Foster et al.ICML 2023 · 133 citations
- AutomaTikZ: Text-Guided Synthesis of Scientific Vector Graphics with TikZJonas Belouadi, Anne Lauscher, Steffen EgerICLR 2024 · 64 citations
- ByGPT5: End-to-End Style-conditioned Poetry Generation with Token-free Language ModelsJonas Belouadi, Steffen EgerACL 2023 · 12 citations
- Memorisation Cartography: Mapping out the Memorisation-Generalisation Continuum in Neural Machine TranslationVerna Dankers, Ivan Titov, Dieuwke HupkesEMNLP 2023
- Revisiting Compositional Generalization Capability of Large Language Models Considering Instruction Following AbilityYusuke Sakai, Hidetaka Kamigaito, Taro WatanabeACL 2025
Builds on10
- Extracting Training Data from Large Language ModelsNicholas Carlini, Florian Tramèr, Eric Wallace, Matthew Jagielski et al.USENIX Security 2021 · 2,866 citations
- What Neural Networks Memorize and Why: Discovering the Long Tail via Influence EstimationVitaly Feldman, Chiyuan ZhangNeurIPS 2020 · 674 citations
- Memorization Without Overfitting: Analyzing the Training Dynamics of Large Language ModelsKushal Tirumala, Aram H. Markosyan, Luke Zettlemoyer, Armen AghajanyanNeurIPS 2022 · 304 citations
- Quantifying Memorization Across Neural Language ModelsNicholas Carlini, Daphne Ippolito, Matthew Jagielski, Katherine Lee et al.ICLR 2023 · 158 citations
- Memorisation versus Generalisation in Pre-trained Language ModelsMichael Tänzer, Sebastian Ruder, Marek ReiACL 2022 · 59 citations
Related papers
- Counterfactual Memorization in Neural Language ModelsChiyuan Zhang, Daphne Ippolito, Katherine Lee, Matthew Jagielski et al.NeurIPS 2023 · 184 citations
- Detecting Hallucinated Content in Conditional Neural Sequence GenerationChunting Zhou, Graham Neubig, Jiatao Gu, Mona T. Diab et al.ACL Findings 2021 · 122 citations
- Mitigating Memorization in Language ModelsMansi Sakarvadia, Aswathy Ajith, Arham Mushtaq Khan, Nathaniel C. Hudson et al.ICLR 2025
- Scalable Extraction of Training Data from Aligned, Production Language ModelsMilad Nasr, Javier Rando, Nicholas Carlini, Jonathan Hayase et al.ICLR 2025
- Syntax-Aware Retrieval Augmented Code GenerationXiangyu Zhang, Yu Zhou, Guang Yang, Taolue ChenEMNLP Findings 2023 · 10 citations
