EMNLP Findings2023
Responsible AI Considerations in Text Summarization Research: A Review of Current Practices
Yu Lu Liu, Meng Cao, Su Lin Blodgett, Jackie Chi Kit Cheung, Alexandra Olteanu, Adam Trischler
Abstract
AI and NLP publication venues have increasingly encouraged researchers to reflect on possible ethical considerations, adverse impacts, and other responsible AI issues their work might engender. However, for specific NLP tasks our understanding of how prevalent such issues are, or when and why these issues are likely to arise, remains limited. Focusing on text summarization-a common NLP task largely overlooked by the responsible AI community-we examine research and reporting practices in the current literature. We conduct a multi-round qualitative analysis of 333 summarization papers from the ACL Anthology published between 2020-2022. We focus on how, which, and when responsible AI issues are covered, which relevant stakeholders are considered, and mismatches between stated and realized research goals. We also discuss current evaluation practices and consider how authors discuss the limitations of both prior work and their own work. Overall, we find that relatively few papers engage with possible stakeholders or contexts of use, which limits their consideration of potential downstream adverse impacts or other responsible AI issues. Based on our findings, we make recommendations on concrete practices and research directions.
Ask Lune
Know this paper in minutes, not an afternoon.
Lune can read every page of it and the top-tier papers that cite it, and gives you its claims, its evidence and its limits, each one cited to the paper it comes from. No skimming, no AI slop.
4 top-tier papers have cited it since. Know what they found before your reviewers do.
Lune can draw on
the full paper · the top-tier papers that cite it · related work
Free to start. No credit card required.
Lineage
- Metrics for What, Metrics for Whom: Assessing Actionability of Bias Evaluation Metrics in NLP cites Language (Technology) is Power: A Critical Survey of "Bias" in NLP
- Summarizing Speech: A Comprehensive Survey cites Reducing the Frequency of Hallucinated Quantities in Abstractive Summaries
Cited by top-tier papers4
- Metrics for What, Metrics for Whom: Assessing Actionability of Bias Evaluation Metrics in NLPPieter Delobelle, Giuseppe Attanasio, Debora Nozza, Su Lin Blodgett et al.EMNLP 2024 · 4 citations
- Summarizing Speech: A Comprehensive SurveyFabian Retkowski, Maike Züfle, Andreas Sudmann, Dinah Pfau et al.EMNLP 2025 · 3 citations
- Narrative License and Model Sycophancy in LLM Summaries of Scientific WorkCalvin Isch, Grace JenningsACL 2026
- Social Good or Scientific Curiosity? Uncovering the Research Framing Behind NLP ArtefactsEric Chamoun, Nedjma Ousidhoum, Michael Sejr Schlichtkrull, Andreas VlachosEMNLP 2025
Builds on21
- PRIMERA: Pyramid-based Masked Sentence Pre-training for Multi-document SummarizationWen Xiao, Iz Beltagy, Giuseppe Carenini, Arman CohanACL 2022 · 147 citations
- Reducing the Frequency of Hallucinated Quantities in Abstractive SummariesZheng Zhao, Shay B. Cohen, Bonnie WebberEMNLP Findings 2020 · 99 citations
- Multi-Fact Correction in Abstractive Text SummarizationYue Dong, Shuohang Wang, Zhe Gan, Yu Cheng et al.EMNLP 2020 · 99 citations
- Coarse-to-Fine Query Focused Multi-Document SummarizationYumo Xu, Mirella LapataEMNLP 2020 · 76 citations
- Language (Technology) is Power: A Critical Survey of "Bias" in NLPSu Lin Blodgett, Solon Barocas, Hal Daumé III, Hanna M. WallachACL 2020 · 68 citations
Related papers
- A Scoping Study of Evaluation Practices for Responsible AI Tools: Steps Towards Effectiveness EvaluationsGlen Berman, Nitesh Goyal, Michael MadaioCHI 2024 · 40 citations
- 'I'm Categorizing LLM as a Productivity Tool': Examining Ethics of LLM Use in HCI Research PracticesShivani Kapania, Ruiyi Wang, Toby Jia-Jun Li, Tianshi Li et al.CSCW 2025 · 31 citations
- Reporting and Reviewing LLM-Integrated Systems in HCI: Challenges and ConsiderationsKarla Felix Navarro, Eugene Syriani, Ian ArawjoCHI 2026 · 1 citation
- The State of Multilingual LLM Safety Research: From Measuring The Language Gap To Mitigating ItZheng Xin Yong, Beyza Ermis, Marzieh Fadaee, Stephen H. Bach et al.EMNLP 2025 · 2 citations
- COVER: Context-Driven Over-Refusal Verification in LLMsGiovanni Sullutrone, Riccardo Amerigo Vigliermo, Sonia Bergamaschi, Luca SalaACL Findings 2025 · 1 citation
