Lune

EMNLP2021Top-tier venue

Profanity-Avoiding Training Framework for Seq2seq Models with Certified Robustness

Hengtong Zhang, Tianhang Zheng, Yaliang Li, Jing Gao, Lu Su, Bo Li

Abstract

Seq2seq models have demonstrated their incredible effectiveness in a large variety of applications. However, recent research has shown that inappropriate language in training samples and well-designed testing cases can induce seq2seq models to output profanity. These outputs may potentially hurt the usability of seq2seq models and make the end-users feel offended. To address this problem, we propose a training framework with certified robustness to eliminate the causes that trigger the generation of profanity. The proposed training framework leverages merely a short list of profanity examples to prevent seq2seq models from generating a broader spectrum of profanity. The framework is composed of a patterneliminating training component to suppress the impact of language patterns with profanity in the training set, and a trigger-resisting training component to provide certified robustness for seq2seq models against intentionally injected profanity-triggering expressions in test samples. In the experiments, we consider two representative NLP tasks that seq2seq can be applied to, i.e., style transfer and dialogue generation. Extensive experimental results show that the proposed training framework can successfully prevent the NLP models from generating profanity.

Ask Lune

Know this paper in minutes, not an afternoon.

Lune can read every page of it and the top-tier papers that cite it, and gives you its claims, its evidence and its limits, each one cited to the paper it comes from. No skimming, no AI slop.

Questions to start from

Lune can draw on

the full paper · related work

Get the answer

Free to start. No credit card required.

Lineage

  • Randomized Smoothing of All Shapes and Sizes cites Certified Robustness to Adversarial Examples with Differential Privacy
  • TextBugger: Generating Adversarial Text Against Real-world Applications cites Seq2Sick: Evaluating the Robustness of Sequence-to-Sequence Models with Adversarial Examples
  • Is BERT Really Robust? A Strong Baseline for Natural Language Attack on Text Classification and Entailment cites TextBugger: Generating Adversarial Text Against Real-world Applications
Builds onCited byLeft to right by year; the more cited, the higher.

Builds on7

  1. Is BERT Really Robust? A Strong Baseline for Natural Language Attack on Text Classification and EntailmentDi Jin, Zhijing Jin, Joey Tianyi Zhou, Peter SzolovitsAAAI 2020 · 1,333 citations
  2. Certified Robustness to Adversarial Examples with Differential PrivacyMathias Lécuyer, Vaggelis Atlidakis, Roxana Geambasu, Daniel Hsu et al.S&P 2019 · 1,022 citations
  3. TextBugger: Generating Adversarial Text Against Real-world ApplicationsJinfeng Li, Shouling Ji, Tianyu Du, Bo Li et al.NDSS 2019 · 876 citations
  4. Seq2Sick: Evaluating the Robustness of Sequence-to-Sequence Models with Adversarial ExamplesMinhao Cheng, Jinfeng Yi, Pin-Yu Chen, Huan Zhang et al.AAAI 2020 · 268 citations
  5. Randomized Smoothing of All Shapes and SizesGreg Yang, Tony Duan, J. Edward Hu, Hadi Salman et al.ICML 2020 · 237 citations

Related papers

  1. Certified Robustness to Programmable Transformations in LSTMsYuhao Zhang, Aws Albarghouthi, Loris D'AntoniEMNLP 2021 · 8 citations
  2. Precisely the Point: Adversarial Augmentations for Faithful and Informative Text GenerationWenhao Wu, Wei Li, Jiachen Liu, Xinyan Xiao et al.EMNLP 2022 · 4 citations
  3. RealToxicityPrompts: Evaluating Neural Toxic Degeneration in Language ModelsSamuel Gehman, Suchin Gururangan, Maarten Sap, Yejin Choi et al.EMNLP Findings 2020 · 692 citations
  4. People Make Better Edits: Measuring the Efficacy of LLM-Generated Counterfactually Augmented Data for Harmful Language DetectionIndira Sen, Dennis Assenmacher, Mattia Samory, Isabelle Augenstein et al.EMNLP 2023 · 7 citations
  5. Randomized Smoothing with Masked Inference for Adversarially Robust Text ClassificationsHan Cheol Moon, Shafiq R. Joty, Ruochen Zhao, Megh Thakkar et al.ACL 2023 · 6 citations
Dusk over the sea between two cliffs drawn in fine vertical lines