Lune

ACL Findings2022

Combining Feature and Instance Attribution to Detect Artifacts

Pouya Pezeshkpour, Sarthak Jain, Sameer Singh, Byron C. Wallace

Abstract

Training the deep neural networks that dominate NLP requires large datasets. These are often collected automatically or via crowdsourcing, and may exhibit systematic biases or annotation artifacts. By the latter we mean spurious correlations between inputs and outputs that do not represent a generally held causal relationship between features and classes; models that exploit such correlations may appear to perform a given task well, but fail on out of sample data. In this paper we evaluate use of different attribution methods for aiding identification of training data artifacts. We propose new hybrid approaches that combine saliency maps (which highlight "important" input features) with instance attribution methods (which retrieve training samples "influential" to a given prediction). We show that this proposed training-feature attribution can be used to efficiently uncover artifacts in training data when a challenging validation set is available. We also carry out a small user study to evaluate whether these methods are useful to NLP researchers in practice, with promising results. We make code for all methods and experiments in this paper available. 1

Ask Lune

Know this paper in minutes, not an afternoon.

Lune can read every page of it and the top-tier papers that cite it, and gives you its claims, its evidence and its limits, each one cited to the paper it comes from. No skimming, no AI slop.

18 top-tier papers have cited it since. Know what they found before your reviewers do.

Questions to start from

Lune can draw on

the full paper · the top-tier papers that cite it · related work

Get the answer

Free to start. No credit card required.

Lineage

  • First is Better Than Last for Language Data Influence cites Explaining Black Box Predictions and Unveiling Data Artifacts through Influence Functions
  • Understanding Dataset Difficulty with V-Usable Information cites Competency Problems: On Finding and Removing Artifacts in Language Data
  • "Will You Find These Shortcuts?" A Protocol for Evaluating the Faithfulness of Input Salience Methods for Text Classification cites Explaining Black Box Predictions and Unveiling Data Artifacts through Influence Functions
  • "Will You Find These Shortcuts?" A Protocol for Evaluating the Faithfulness of Input Salience Methods for Text Classification cites Do Feature Attribution Methods Correctly Attribute Features?
  • "Will You Find These Shortcuts?" A Protocol for Evaluating the Faithfulness of Input Salience Methods for Text Classification cites Explaining NLP Models via Minimal Contrastive Editing (MiCE)
  • "Will You Find These Shortcuts?" A Protocol for Evaluating the Faithfulness of Input Salience Methods for Text Classification cites A Diagnostic Study of Explainability Techniques for Text Classification
  • Out-of-Distribution Generalization in Natural Language Processing: Past, Present, and Future cites Competency Problems: On Finding and Removing Artifacts in Language Data
  • Out-of-Distribution Generalization in Natural Language Processing: Past, Present, and Future cites "Will You Find These Shortcuts?" A Protocol for Evaluating the Faithfulness of Input Salience Methods for Text Classification
  • SELFEXPLAIN: A Self-Explaining Architecture for Neural Text Classifiers cites Explaining Black Box Predictions and Unveiling Data Artifacts through Influence Functions
Builds onCited byLeft to right by year; the more cited, the higher.

Cited by top-tier papers18

  1. Understanding Dataset Difficulty with V-Usable InformationKawin Ethayarajh, Yejin Choi, Swabha SwayamdiptaICML 2022 · 337 citations
  2. First is Better Than Last for Language Data InfluenceChih-Kuan Yeh, Ankur Taly, Mukund Sundararajan, Frederick Liu et al.NeurIPS 2022 · 39 citations
  3. SELFEXPLAIN: A Self-Explaining Architecture for Neural Text ClassifiersDheeraj Rajagopal, Vidhisha Balachandran, Eduard H. Hovy, Yulia TsvetkovEMNLP 2021 · 39 citations
  4. "Will You Find These Shortcuts?" A Protocol for Evaluating the Faithfulness of Input Salience Methods for Text ClassificationJasmijn Bastings, Sebastian Ebert, Polina Zablotskaia, Anders Sandholm et al.EMNLP 2022 · 29 citations
  5. Out-of-Distribution Generalization in Natural Language Processing: Past, Present, and FutureLinyi Yang, Yaoxian Song, Xuan Ren, Chenyang Lyu et al.EMNLP 2023 · 12 citations

Builds on7

  1. Do Feature Attribution Methods Correctly Attribute Features?Yilun Zhou, Serena Booth, Marco Túlio Ribeiro, Julie ShahAAAI 2022 · 167 citations
  2. A Diagnostic Study of Explainability Techniques for Text ClassificationPepa Atanasova, Jakob Grue Simonsen, Christina Lioma, Isabelle AugensteinEMNLP 2020 · 158 citations
  3. Explaining Black Box Predictions and Unveiling Data Artifacts through Influence FunctionsXiaochuang Han, Byron C. Wallace, Yulia TsvetkovACL 2020 · 91 citations
  4. Explaining NLP Models via Minimal Contrastive Editing (MiCE)Alexis Ross, Ana Marasovic, Matthew E. PetersACL Findings 2021 · 86 citations
  5. Competency Problems: On Finding and Removing Artifacts in Language DataMatt Gardner, William Merrill, Jesse Dodge, Matthew E. Peters et al.EMNLP 2021 · 72 citations

Related papers

  1. Debugging Tests for Model ExplanationsJulius Adebayo, Michael Muelly, Ilaria Liccardi, Been KimNeurIPS 2020 · 209 citations
  2. Self-Supervised Debiasing Using Low Rank RegularizationGeon Yeong Park, Chanyong Jung, Sangmin Lee, Jong Chul Ye et al.CVPR 2024 · 2 citations
  3. Improving Deep Learning Interpretability by Saliency Guided TrainingAya Abdelsalam Ismail, Héctor Corrada Bravo, Soheil FeiziNeurIPS 2021 · 121 citations
  4. AGAIN: Adversarial Training with Attribution Span Enlargement and Hybrid Feature FusionShenglin Yin, Kelu Yao, Sheng Shi, Yangzhou Du et al.CVPR 2023
  5. Scalable Influence and Fact Tracing for Large Language Model PretrainingTyler A. Chang, Dheeraj Rajagopal, Tolga Bolukbasi, Lucas Dixon et al.ICLR 2025 · 1 citation
Dusk over the sea between two cliffs drawn in fine vertical lines