Lune

EMNLP Findings2025

Identifying Aspects in Peer Reviews

Sheng Lu, Ilia Kuznetsov, Iryna Gurevych

Abstract

Peer review is central to academic publishing, but the growing volume of submissions is straining the process. This motivates the development of computational approaches to support peer review. While each review is tailored to a specific paper, reviewers often make assessments according to certain aspects such as Novelty, which reflect the values of the research community. This alignment creates opportunities for standardizing the reviewing process, improving quality control, and enabling computational support. While prior work has demonstrated the potential of aspect analysis for peer review assistance, the notion of aspect remains poorly formalized. Existing approaches often derive aspects from review forms and guidelines, yet data-driven methods for aspect identification are underexplored. To address this gap, our work takes a bottom-up approach: we propose an operational definition of aspect and develop a data-driven schema for deriving aspects from a corpus of peer reviews. We introduce a dataset of peer reviews augmented with aspects and show how it can be used for communitylevel review analysis. We further show how the choice of aspects can impact downstream applications, such as LLM-generated review detection. Our results lay a foundation for a principled and data-driven investigation of review aspects, and pave the path for new applications of NLP to support peer review. 1

Ask Lune

Know this paper in minutes, not an afternoon.

Lune can read every page of it and the top-tier papers that cite it, and gives you its claims, its evidence and its limits, each one cited to the paper it comes from. No skimming, no AI slop.

2 top-tier papers have cited it since. Know what they found before your reviewers do.

Questions to start from

Lune can draw on

the full paper · the top-tier papers that cite it · related work

Get the answer

Free to start. No credit card required.

Lineage

  • Can AI Be a Good Peer Reviewer? A Survey of Peer Review Process, Evaluation, and the Future cites Mind the Blind Spots: A Focus-Level Evaluation Framework for LLM Reviews
  • Mind the Blind Spots: A Focus-Level Evaluation Framework for LLM Reviews cites MetaWriter: Exploring the Potential and Perils of AI Writing Support in Scientific Peer Review
Builds onCited byLeft to right by year; the more cited, the higher.

Cited by top-tier papers2

  1. Can AI Be a Good Peer Reviewer? A Survey of Peer Review Process, Evaluation, and the FutureSihong Wu, Owen Jiang, Yilun Zhao, Tiansheng Hu et al.ACL 2026 · 2 citations
  2. Mind the Blind Spots: A Focus-Level Evaluation Framework for LLM ReviewsHyungyu Shin, Jingyu Tang, Yoonjoo Lee, Nayoung Kim et al.EMNLP 2025 · 2 citations

Builds on12

  1. The Unlocking Spell on Base LLMs: Rethinking Alignment via In-Context LearningBill Yuchen Lin, Abhilasha Ravichander, Ximing Lu, Nouha Dziri et al.ICLR 2024 · 299 citations
  2. A Novice-Reviewer Experiment to Address Scarcity of Qualified Reviewers in Large ConferencesIvan Stelmakh, Nihar B. Shah, Aarti Singh, Hal Daumé IIIAAAI 2021 · 37 citations
  3. MetaWriter: Exploring the Potential and Perils of AI Writing Support in Scientific Peer ReviewLu Sun, Stone Tao, Junjie Hu, Steven P. DowCSCW 2024 · 36 citations
  4. On the Relation between Sensitivity and Accuracy in In-Context LearningYanda Chen, Chen Zhao, Zhou Yu, Kathleen R. McKeown et al.EMNLP Findings 2023 · 29 citations
  5. Debiasing Evaluations That Are Biased by EvaluationsJingyan Wang, Ivan Stelmakh, Yuting Wei, Nihar B. ShahAAAI 2021 · 24 citations

Related papers

  1. NovBench: Evaluating Large Language Models on Academic Paper Novelty AssessmentWenqing Wu, Yi Zhao, Yuzhuo Wang, Siyou Li et al.ACL Findings 2026 · 4 citations
  2. NLPeer: A Unified Resource for the Computational Study of Peer ReviewNils Dycke, Ilia Kuznetsov, Iryna GurevychACL 2023 · 17 citations
  3. The Good, the Bad and the Constructive: Automatically Measuring Peer Review's Utility for AuthorsAbdelrahman Sadallah, Tim Baumgärtner, Iryna Gurevych, Ted BriscoeEMNLP 2025
  4. Argument Mining Driven Analysis of Peer-ReviewsMichael Fromm, Evgeniy Faerman, Max Berrendorf, Siddharth Bhargava et al.AAAI 2021 · 36 citations
  5. Is Your Paper Being Reviewed by an LLM? Benchmarking AI Text Detection in Peer ReviewSungduk Yu, Man Luo, Avinash Madasu, Vasudev Lal et al.ICLR 2026 · 24 citations
Dusk over the sea between two cliffs drawn in fine vertical lines