ACL Findings2023
Model Interpretability and Rationale Extraction by Input Mask Optimization
Abstract
Concurrent with the rapid progress in neural network-based models in NLP, the need for creating explanations for the predictions of these black-box models has risen steadily. Yet, especially for complex inputs like texts or images, existing interpretability methods still struggle with deriving easily interpretable explanations that also accurately represent the basis for the model's decision. To this end, we propose a new, model-agnostic method to generate extractive explanations for predictions made by neural networks, that is based on masking parts of the input which the model does not consider to be indicative of the respective class. The masking is done using gradient-based optimization combined with a new regularization scheme that enforces sufficiency, comprehensiveness, and compactness of the generated explanation.
Our method achieves state-of-the-art results in a challenging paragraph-level rationale extraction task, showing that this task can be performed without training a specialized model. We further apply our method to image inputs and obtain high-quality explanations for image classifications, which indicates that the objectives for optimizing explanation masks in text generalize to inputs of other modalities.
Ask Lune
Know this paper in minutes, not an afternoon.
Lune can read every page of it and the top-tier papers that cite it, and gives you its claims, its evidence and its limits, each one cited to the paper it comes from. No skimming, no AI slop.
1 top-tier paper has cited it since. Know what they found before your reviewers do.
Lune can draw on
the full paper · the top-tier papers that cite it · related work
Free to start. No credit card required.
Lineage
- Rationalizing Transformer Predictions via End-To-End Differentiable Self-Training cites ERASER: A Benchmark to Evaluate Rationalized NLP Models
- Rationalizing Transformer Predictions via End-To-End Differentiable Self-Training cites Generic Attention-model Explainability for Interpreting Bi-Modal and Encoder-Decoder Transformers
- Rationalizing Transformer Predictions via End-To-End Differentiable Self-Training cites Transformer Interpretability Beyond Attention Visualization
- Generic Attention-model Explainability for Interpreting Bi-Modal and Encoder-Decoder Transformers cites Transformer Interpretability Beyond Attention Visualization
- A Diagnostic Study of Explainability Techniques for Text Classification cites ERASER: A Benchmark to Evaluate Rationalized NLP Models
- Transformer Interpretability Beyond Attention Visualization cites ERASER: A Benchmark to Evaluate Rationalized NLP Models
Cited by top-tier papers1
Builds on4
- Generic Attention-model Explainability for Interpreting Bi-Modal and Encoder-Decoder TransformersHila Chefer, Shir Gur, Lior WolfICCV 2021 · 451 citations
- A Diagnostic Study of Explainability Techniques for Text ClassificationPepa Atanasova, Jakob Grue Simonsen, Christina Lioma, Isabelle AugensteinEMNLP 2020 · 158 citations
- ERASER: A Benchmark to Evaluate Rationalized NLP ModelsJay DeYoung, Sarthak Jain, Nazneen Fatema Rajani, Eric P. Lehman et al.ACL 2020 · 36 citations
- Transformer Interpretability Beyond Attention VisualizationHila Chefer, Shir Gur, Lior WolfCVPR 2021
Related papers
- Generating Hierarchical Explanations on Text Classification via Feature Interaction DetectionHanjie Chen, Guangtao Zheng, Yangfeng JiACL 2020 · 85 citations
- Learn from A Rationalist: Distilling Intermediate Interpretable RationalesJiayi Dai, Randy GoebelICML 2026
- Learning from the Best: Rationalizing Predictions by Adversarial Information CalibrationLei Sha, Oana-Maria Camburu, Thomas LukasiewiczAAAI 2021 · 40 citations
- From Pixels to Perception: Interpretable Predictions via Instance-wise Grouped Feature SelectionMoritz Vandenhirtz, Julia E. VogtICML 2025
- Flexible Instance-Specific Rationalization of NLP ModelsGeorge Chrysostomou, Nikolaos AletrasAAAI 2022 · 17 citations
