ACL Findings2023
Neighboring Words Affect Human Interpretation of Saliency Explanations
Alon Jacovi, Hendrik Schuff, Heike Adel, Ngoc Thang Vu, Yoav Goldberg
Abstract
Word-level saliency explanations ("heat maps over words") are often used to communicate feature-attribution in text-based models. Recent studies found that superficial factors such as word length can distort human interpretation of the communicated saliency scores. We conduct a user study to investigate how the marking of a word's neighboring words affect the explainee's perception of the word's importance in the context of a saliency explanation. We find that neighboring words have significant effects on the word's importance rating. Concretely, we identify that the influence changes based on neighboring direction (left vs. right) and a-priori linguistic and computational measures of phrases and collocations (vs. unrelated neighboring words). Our results question whether text-based saliency explanations should be continued to be communicated at word level, and inform future research on alternative saliency explanation methods.
Ask Lune
Know this paper in minutes, not an afternoon.
Lune can read every page of it and the top-tier papers that cite it, and gives you its claims, its evidence and its limits, each one cited to the paper it comes from. No skimming, no AI slop.
Lineage
Builds on5
- BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and ComprehensionMike Lewis, Yinhan Liu, Naman Goyal, Marjan Ghazvininejad et al.ACL 2020 · 1,224 citations
- The effectiveness of feature attribution methods and its correlation with automatic evaluation scoresGiang Nguyen, Daeyoung Kim, Anh NguyenNeurIPS 2021 · 128 citations
- Look at the Variance! Efficient Black-box Explanations with Sobol-based Sensitivity AnalysisThomas Fel, Rémi Cadène, Mathieu Chalvidal, Matthieu Cord et al.NeurIPS 2021 · 100 citations
- Explain, Edit, and Understand: Rethinking User Study Design for Evaluating Model ExplanationsSiddhant Arora, Danish Pruthi, Norman M. Sadeh, William W. Cohen et al.AAAI 2022 · 47 citations
- Gradient-based Analysis of NLP Models is ManipulableJunlin Wang, Jens Tuyls, Eric Wallace, Sameer SinghEMNLP Findings 2020 · 43 citations
Related papers
- Graphical Perception of Saliency-based Model ExplanationsYayan Zhao, Mingwei Li, Matthew BergerCHI 2023 · 3 citations
- Generating Hierarchical Explanations on Text Classification via Feature Interaction DetectionHanjie Chen, Guangtao Zheng, Yangfeng JiACL 2020 · 85 citations
- Unraveling the Dilemma of AI Errors: Exploring the Effectiveness of Human and Machine Explanations for Large Language ModelsMarvin Pafla, Kate Larson, Mark HancockCHI 2024 · 16 citations
- Fool Me Once? Contrasting Textual and Visual Explanations in a Clinical Decision-Support SettingMaxime Kayser, Bayar Menzat, Cornelius Emde, Bogdan Bercean et al.EMNLP 2024 · 8 citations
- A Diagnostic Study of Explainability Techniques for Text ClassificationPepa Atanasova, Jakob Grue Simonsen, Christina Lioma, Isabelle AugensteinEMNLP 2020 · 158 citations
