Lune

EMNLP Findings2023

Is ChatGPT a Good Multi-Party Conversation Solver?

Chao-Hong Tan, Jia-Chen Gu, Zhen-Hua Ling

Abstract

Large Language Models (LLMs) have emerged as influential instruments within the realm of natural language processing; nevertheless, their capacity to handle multi-party conversations (MPCs) -a scenario marked by the presence of multiple interlocutors involved in intricate information exchanges -remains uncharted. In this paper, we delve into the potential of generative LLMs such as ChatGPT and GPT-4 within the context of MPCs. An empirical analysis is conducted to assess the zero-shot learning capabilities of ChatGPT and GPT-4 by subjecting them to evaluation across three MPC datasets that encompass five representative tasks. The findings reveal that ChatGPT's performance on a number of evaluated MPC tasks leaves much to be desired, whilst GPT-4's results portend a promising future. Additionally, we endeavor to bolster performance through the incorporation of MPC structures, encompassing both speaker and addressee architecture. This study provides an exhaustive evaluation and analysis of applying generative LLMs to MPCs, casting a light upon the conception and creation of increasingly effective and robust MPC agents. Concurrently, this work underscores the challenges implicit in the utilization of LLMs for MPCs, such as deciphering graphical information flows and generating stylistically consistent responses.

Ask Lune

Know this paper in minutes, not an afternoon.

Lune can read every page of it and the top-tier papers that cite it, and gives you its claims, its evidence and its limits, each one cited to the paper it comes from. No skimming, no AI slop.

4 top-tier papers have cited it since. Know what they found before your reviewers do.

Questions to start from

Lune can draw on

the full paper · the top-tier papers that cite it · related work

Get the answer

Free to start. No credit card required.

Lineage

  • Don't Stop the Multi-Party! On Generating Synthetic Written Multi-Party Conversations with Constraints cites Do LLMs suffer from Multi-Party Hangover? A Diagnostic Approach to Addressee Recognition and Response Selection in Conversations
  • G-Eval: NLG Evaluation using Gpt-4 with Better Human Alignment cites BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehension
  • Improving Multi-party Dialogue Generation via Topic and Rhetorical Coherence cites BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehension
Builds onCited byLeft to right by year; the more cited, the higher.

Cited by top-tier papers4

  1. Don't Stop the Multi-Party! On Generating Synthetic Written Multi-Party Conversations with ConstraintsNicolò Penzo, Marco Guerini, Bruno Lepri, Goran Glavas et al.AAAI 2026 · 3 citations
  2. Omni-MMSI: Toward Identity-attributed Social Interaction UnderstandingXinpeng Li, Bolin Lai, Hardy Chen, Shijian Deng et al.CVPR 2026 · 3 citations
  3. Do LLMs suffer from Multi-Party Hangover? A Diagnostic Approach to Addressee Recognition and Response Selection in ConversationsNicolò Penzo, Maryam Sajedinia, Bruno Lepri, Sara Tonelli et al.EMNLP 2024 · 2 citations
  4. Improving Multi-party Dialogue Generation via Topic and Rhetorical CoherenceYaxin Fan, Peifeng Li, Qiaoming ZhuEMNLP 2024 · 1 citation

Builds on10

  1. Language Models are Few-Shot LearnersTom B. Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah et al.NeurIPS 2020 · 64,255 citations
  2. BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and ComprehensionMike Lewis, Yinhan Liu, Naman Goyal, Marjan Ghazvininejad et al.ACL 2020 · 1,224 citations
  3. G-Eval: NLG Evaluation using Gpt-4 with Better Human AlignmentYang Liu, Dan Iter, Yichong Xu, Shuohang Wang et al.EMNLP 2023 · 549 citations
  4. Is ChatGPT a General-Purpose Natural Language Processing Task Solver?Chengwei Qin, Aston Zhang, Zhuosheng Zhang, Jiaao Chen et al.EMNLP 2023 · 449 citations
  5. Supervised Prototypical Contrastive Learning for Emotion Recognition in ConversationXiaohui Song, Longtao Huang, Hui Xue, Songlin HuEMNLP 2022 · 84 citations

Related papers

  1. An Empirical Study of Many-to-Many Summarization with Large Language ModelsJiaan Wang, Fandong Meng, Zengkui Sun, Yunlong Liang et al.ACL 2025
  2. Evaluating Large Language Models on Academic Literature Understanding and Review: An Empirical Study among Early-stage ScholarsJiyao Wang, Haolong Hu, Zuyuan Wang, Song Yan et al.CHI 2024 · 21 citations
  3. MEGA: Multilingual Evaluation of Generative AIKabir Ahuja, Harshita Diddee, Rishav Hada, Millicent Ochieng et al.EMNLP 2023 · 91 citations
  4. PPTC Benchmark: Evaluating Large Language Models for PowerPoint Task CompletionYiduo Guo, Zekai Zhang, Yaobo Liang, Dongyan Zhao et al.ACL Findings 2024 · 6 citations
  5. A Systematic Study and Comprehensive Evaluation of ChatGPT on Benchmark DatasetsMd. Tahmid Rahman Laskar, M. Saiful Bari, Mizanur Rahman, Md Amran Hossen Bhuiyan et al.ACL Findings 2023 · 97 citations
Dusk over the sea between two cliffs drawn in fine vertical lines