ACL Findings2021
Attending via both Fine-tuning and Compressing
Jie Zhou, Yuanbin Wu, Qin Chen, Xuanjing Huang, Liang He
Abstract
Though being a primary trend for enhancing interpretability of neural networks, attention mechanism's reliability and validity are still under debate. In this paper, we try to purify attention scores to obtain a more faithful explanation of downstream models. Specifically, we propose a framework consisting of a learner and a compressor, which performs finetuning and compressing iteratively to enhance the performance and interpretability of the attention mechanism. The learner focuses on learning better text representations to achieve good decisions by fine-tuning, while the compressor aims to perform compressions over the representations to retain the most useful clues for explanations with a Variational information bottleneck ATtention (VAT) mechanism. Extensive experiments on eight benchmark datasets show the great advantages of our proposed approach in terms of both performance and interpretability.
Ask Lune
Know this paper in minutes, not an afternoon.
Lune can read every page of it and the top-tier papers that cite it, and gives you its claims, its evidence and its limits, each one cited to the paper it comes from. No skimming, no AI slop.
1 top-tier paper has cited it since. Know what they found before your reviewers do.
Lune can draw on
the full paper · the top-tier papers that cite it · related work
Free to start. No credit card required.
Lineage
- Inserting Information Bottleneck for Attribution in Transformers cites Restricting the Flow: Information Bottlenecks for Attribution
- Learning Variational Word Masks to Improve the Interpretability of Neural Text Classifiers cites Restricting the Flow: Information Bottlenecks for Attribution
Cited by top-tier papers1
Builds on3
- Restricting the Flow: Information Bottlenecks for AttributionKarl Schulz, Leon Sixt, Federico Tombari, Tim LandgrafICLR 2020 · 220 citations
- Learning Variational Word Masks to Improve the Interpretability of Neural Text ClassifiersHanjie Chen, Yangfeng JiEMNLP 2020 · 45 citations
- Inserting Information Bottleneck for Attribution in TransformersZhiying Jiang, Raphael Tang, Ji Xin, Jimmy LinEMNLP Findings 2020 · 9 citations
Related papers
- Why Attentions May Not Be Interpretable?Bing Bai, Jian Liang, Guanhua Zhang, Hao Li et al.KDD 2021 · 51 citations
- InterpreTabNet: Distilling Predictive Signals from Tabular Data by Salient Feature InterpretationJacob Yoke Hong Si, Wendy Yusi Cheng, Michael Cooper, Rahul G. KrishnanICML 2024 · 15 citations
- Visual Explanations of Image-Text Representations via Multi-Modal Information Bottleneck AttributionYing Wang, Tim G. J. Rudner, Andrew Gordon WilsonNeurIPS 2023 · 51 citations
- Improving the Faithfulness of Attention-based Explanations with Task-specific Information for Text ClassificationGeorge Chrysostomou, Nikolaos AletrasACL 2021
- Explaining A Black-box By Using A Deep Variational Information Bottleneck ApproachSeo-Jin Bang, Pengtao Xie, Heewook Lee, Wei Wu et al.AAAI 2021 · 33 citations
