Lune

ACL Findings2024

TextGenSHAP: Scalable Post-Hoc Explanations in Text Generation with Long Documents

James Enouen, Hootan Nakhost, Sayna Ebrahimi, Sercan Ö. Arik, Yan Liu, Tomas Pfister

Abstract

Large language models (LLMs) have attracted great interest in many real-world applications; however, their "black-box" nature necessitates scalable and faithful explanations. Shapley values have matured as an explainability method for deep learning, but extending them to LLMs is difficult due to long input contexts and autoregressive output generation. We introduce TextGenSHAP, an efficient posthoc explanation method incorporating LLMspecific techniques, which leads to significant runtime improvements: token-level explanations in minutes not hours, and document-level explanations within seconds. We demonstrate how such explanations can improve end-toend performance of retrieval augmented generation by localizing important words within long documents and reranking passages collected by retrieval systems. On various opendomain question answering benchmarks, we show TextGenSHAP improves the retrieval recall and prediction accuracy significantly.

Ask Lune

Know this paper in minutes, not an afternoon.

Lune can read every page of it and the top-tier papers that cite it, and gives you its claims, its evidence and its limits, each one cited to the paper it comes from. No skimming, no AI slop.

9 top-tier papers have cited it since. Know what they found before your reviewers do.

Questions to start from

Lune can draw on

the full paper · the top-tier papers that cite it · related work

Get the answer

Free to start. No credit card required.

Lineage

  • Multi-Level Explanations for Generative Language Models cites Chain-of-Thought Prompting Elicits Reasoning in Large Language Models
  • FlashAttention: Fast and Memory-Efficient Exact Attention with IO-Awareness cites Language Models are Few-Shot Learners
  • Learning to Understand: Identifying Interactions via the Möbius Transform cites Multi-Level Explanations for Generative Language Models
  • Learning to Understand: Identifying Interactions via the Möbius Transform cites FlashAttention: Fast and Memory-Efficient Exact Attention with IO-Awareness
  • Interpretation Meets Safety: A Survey on Interpretation Methods and Tools for Improving LLM Safety cites Chain-of-Thought Prompting Elicits Reasoning in Large Language Models
  • Language Models are Few-Shot Learners cites The Curious Case of Neural Text Degeneration
  • Chain-of-Thought Prompting Elicits Reasoning in Large Language Models cites Training language models to follow instructions with human feedback
  • Training language models to follow instructions with human feedback cites Language Models are Few-Shot Learners
  • Revitalizing Black-Box Interpretability: Actionable Interpretability for LLMs via Proxy Models cites Multi-Level Explanations for Generative Language Models
Builds onCited byLeft to right by year; the more cited, the higher.

Cited by top-tier papers9

  1. Learning to Understand: Identifying Interactions via the Möbius TransformJustin Singh Kang, Yigit Efe Erginbas, Landon Butler, Ramtin Pedarsani et al.NeurIPS 2024 · 17 citations
  2. Multi-Level Explanations for Generative Language ModelsLucas Monteiro Paes, Dennis Wei, Hyo Jin Do, Hendrik Strobelt et al.ACL 2025 · 16 citations
  3. Bridging Fairness and Explainability: Can Input-Based Explanations Promote Fairness in Hate Speech Detection?Yifan Wang, Mayank Jobanputra, Ji-Ung Lee, Soyoung Oh et al.ICLR 2026 · 3 citations
  4. Revitalizing Black-Box Interpretability: Actionable Interpretability for LLMs via Proxy ModelsJunhao Liu, Haonan Yu, Zhenyu Yan, Xin ZhangACL 2026 · 2 citations
  5. Interpretation Meets Safety: A Survey on Interpretation Methods and Tools for Improving LLM SafetySeongmin Lee, Aeree Cho, Grace C. Kim, Shengyun Peng et al.EMNLP 2025 · 1 citation

Builds on23

  1. Language Models are Few-Shot LearnersTom B. Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah et al.NeurIPS 2020 · 64,255 citations
  2. Training language models to follow instructions with human feedbackLong Ouyang, Jeffrey Wu, Xu Jiang, Diogo Almeida et al.NeurIPS 2022 · 24,707 citations
  3. Chain-of-Thought Prompting Elicits Reasoning in Large Language ModelsJason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma et al.NeurIPS 2022 · 22,562 citations
  4. FlashAttention: Fast and Memory-Efficient Exact Attention with IO-AwarenessTri Dao, Daniel Y. Fu, Stefano Ermon, Atri Rudra et al.NeurIPS 2022 · 5,493 citations
  5. The Curious Case of Neural Text DegenerationAri Holtzman, Jan Buys, Li Du, Maxwell Forbes et al.ICLR 2020 · 4,112 citations

Related papers

  1. TokenShapley: Token Level Context Attribution with Shapley ValueYingtai Xiao, Yuqing Zhu, Sirat Samyoun, Wanrong Zhang et al.ACL Findings 2025 · 2 citations
  2. SyntaxShap: Syntax-aware Explainability Method for Text GenerationKenza Amara, Rita Sevastjanova, Mennatallah El-AssadyACL Findings 2024 · 7 citations
  3. Context Attribution with Multi-Armed Bandit OptimizationDeng Pan, Keerthiram Murugesan, Ting Hua, Nuno Moniz et al.ACL Findings 2026
  4. SPEX: Scaling Feature Interaction Explanations for LLMsJustin Singh Kang, Landon Butler, Abhineet Agarwal, Yigit Efe Erginbas et al.ICML 2025
  5. QPaug: Question and Passage Augmentation for Open-Domain Question Answering of LLMsMinsang Kim, Cheoneum Park, Seung BaekEMNLP Findings 2024 · 5 citations
Dusk over the sea between two cliffs drawn in fine vertical lines