Lune

EMNLP Findings2025

Re-FRAME the Meeting Summarization SCOPE: Fact-Based Summarization and Personalization via Questions

Frederic Kirstein, Sonu Kumar, Terry Ruas, Bela Gipp

Abstract

Meeting summarization with large language models (LLMs) remains error-prone, often producing outputs with hallucinations, omissions, and irrelevancies.We present FRAME, a modular pipeline that reframes summarization as a semantic enrichment task.FRAME extracts and scores salient facts, organizes them thematically, and uses these to enrich an outline into an abstractive summary.To personalize summaries, we introduce SCOPE, a reasonout-loud protocol that has the model build a reasoning trace by answering nine questions before content selection.For evaluation, we propose P-MESA, a multi-dimensional, referencefree evaluation framework to assess if a summary fits a target reader.P-MESA reliably identifies error instances, achieving ≥ 89% balanced accuracy against human annotations and strongly aligns with human severity ratings (ρ ≥ 0.70).On QMSum and FAME, FRAME reduces hallucination and omission by 2 out of 5 points (measured with MESA), while SCOPE improves knowledge fit and goal alignment over prompt-only baselines.Our findings advocate for rethinking summarization to improve control, faithfulness, and personalization 1 .QMSum FAME [EN] GPT 4o Gemini 1.5 pro FRAME GPT-4o GPT 4o Gemini 1.5 pro FRAME GPT-4o MESA (lower is better) Coreference 0 1.22 3 1.58 0 1.64 0 1.45 3 1.570 1.36 Hallucination 3 1.22 4 2.04 1 1.75 4 0.98 4 1.40 1 0.72 Incoherence 4 1.50 4 1.09 3 1.88 4 0.94 4 0.72 3 1.24 Irrelevance 2 1.70 3 1.32 1 1.45 3 1.14 3 1.07 1 1.51 Language 1 1.30 2 1.44 1 1.40 1 1.17 1 1.20 1 1.11 Omission 3 0.40 3 0.38 1 0.16 4 0.16 4 0.31 1 0.00 Repetition 4 1.05 3 0.98 1 1.23 4 0.74 4 0.44 2 0.53 Structure 4 0.90 3 1.70 3 1.24 3 1.57 3 1.53 3 1.46

Ask Lune

Ask anything. The answer quotes the paper.

Lune reads all of it and the papers around it, then shows you the passages behind every answer.

Questions to start from

Lune answers from

the full paper · related work

Get the answer

Free to start. No credit card required.

Builds on8

  1. BERTScore: Evaluating Text Generation with BERTTianyi Zhang, Varsha Kishore, Felix Wu, Kilian Q. Weinberger et al.ICLR 2020 · 8,443 citations
  2. Benchmarking Retrieval-Augmented Generation for MedicineGuangzhi Xiong, Qiao Jin, Zhiyong Lu, Aidong ZhangACL Findings 2024 · 312 citations
  3. Unsupervised Opinion Summarization with Content PlanningReinald Kim Amplayo, Stefanos Angelidis, Mirella LapataAAAI 2021 · 51 citations
  4. Finding a Balanced Degree of Automation for Summary EvaluationShiyue Zhang, Mohit BansalEMNLP 2021 · 16 citations
  5. Human-in-the-loop Abstractive Dialogue SummarizationJiaao Chen, Mohan Dodda, Diyi YangACL Findings 2023 · 6 citations

Related papers

  1. ACUEval: Fine-grained Hallucination Evaluation and Correction for Abstractive SummarizationDavid Wan, Koustuv Sinha, Srini Iyer, Asli Celikyilmaz et al.ACL Findings 2024 · 11 citations
  2. ModalSyncSum: Synchronizing Image and Text for Reliable Summary GenerationXuanqi Chen, Ziying Rong, Xinfeng Liao, Yiqian Wu et al.AAAI 2026
  3. SummIt: Iterative Text Summarization via ChatGPTHaopeng Zhang, Xiao Liu, Jiawei ZhangEMNLP Findings 2023 · 58 citations
  4. FRAME: Feedback-Refined Agent Methodology for Enhancing Medical Research InsightsChengzhang Yu, Yiming Zhang, Zhixin Liu, Zenghui Ding et al.ACL Findings 2025 · 3 citations
  5. Large Language Models are Not Yet Human-Level Evaluators for Abstractive SummarizationChenhui Shen, Liying Cheng, Xuan-Phi Nguyen, Yang You et al.EMNLP Findings 2023 · 52 citations
Dusk over the sea between two cliffs drawn in fine vertical lines