Lune

ACL Findings2024

MODDP: A Multi-modal Open-domain Chinese Dataset for Dialogue Discourse Parsing

Chen Gong, Dexin Kong, Suxian Zhao, Xingyu Li, Guohong Fu

Abstract

Dialogue discourse parsing (DDP) aims to capture the relations between utterances in the dialogue. In everyday real-world scenarios, dialogues are typically multi-modal and cover open-domain topics. However, most existing widely used benchmark datasets for DDP contain only textual modality and are domainspecific. This makes it challenging to accurately and comprehensively understand the dialogue without multi-modal clues, and prevents them from capturing the discourse structures of the more prevalent daily conversations. This paper proposes MODDP, the first multi-modal Chinese discourse parsing dataset derived from open-domain daily dialogues, consisting 864 dialogues and 18,114 utterances, accompanied by 12.7 hours of video clips. We present a simple yet effective benchmark approach for multi-modal DDP. Through extensive experiments, we present several benchmark results based on MODDP. The significant improvement in performance from introducing multimodalities into the original textual unimodal DDP model demonstrates the necessity of integrating multi-modalities into DDP.

Ask Lune

Know this paper in minutes, not an afternoon.

Lune can read every page of it and the top-tier papers that cite it, and gives you its claims, its evidence and its limits, each one cited to the paper it comes from. No skimming, no AI slop.

Questions to start from

Lune can draw on

the full paper · related work

Get the answer

Free to start. No credit card required.

Lineage

  • M3ED: Multi-modal Multi-scene Multi-label Emotional Dialogue Database cites wav2vec 2.0: A Framework for Self-Supervised Learning of Speech Representations
  • DualGATs: Dual Graph Attention Networks for Emotion Recognition in Conversations cites Multi-turn Response Selection using Dialogue Dependency Relations
Builds onCited byLeft to right by year; the more cited, the higher.

Builds on8

  1. An Image is Worth 16x16 Words: Transformers for Image Recognition at ScaleAlexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn et al.ICLR 2021 · 21,477 citations
  2. wav2vec 2.0: A Framework for Self-Supervised Learning of Speech RepresentationsAlexei Baevski, Yuhao Zhou, Abdelrahman Mohamed, Michael AuliNeurIPS 2020 · 9,451 citations
  3. M3ED: Multi-modal Multi-scene Multi-label Emotional Dialogue DatabaseJinming Zhao, Tenggan Zhang, Jingwen Hu, Yuchen Liu et al.ACL 2022 · 88 citations
  4. DualGATs: Dual Graph Attention Networks for Emotion Recognition in ConversationsDuzhen Zhang, Feilong Chen, Xiuyi ChenACL 2023 · 54 citations
  5. Multi-turn Response Selection using Dialogue Dependency RelationsQi Jia, Yizhu Liu, Siyu Ren, Kenny Q. Zhu et al.EMNLP 2020 · 31 citations

Related papers

  1. DraDDP: A Multimodal Multi-Party Dialogue Discourse Parsing DatasetShannan Liu, Peifeng Li, Yaxin Fan, Qiaoming ZhuACL Findings 2026 · 1 citation
  2. VSTAR: A Video-grounded Dialogue Dataset for Situated Semantic Understanding with Scene and Topic TransitionsYuxuan Wang, Zilong Zheng, Xueliang Zhao, Jinpeng Li et al.ACL 2023 · 4 citations
  3. CoMuMDR: Code-mixed Multi-modal Multi-domain corpus for Discourse paRsing in conversationsDivyaksh Shukla, Ritesh Baviskar, Dwijesh Gohil, Aniket Tiwari et al.ACL Findings 2025 · 1 citation
  4. JDDC 2.1: A Multimodal Chinese Dialogue Dataset with Joint Tasks of Query Rewriting, Response Generation, Discourse Parsing, and SummarizationNan Zhao, Haoran Li, Youzheng Wu, Xiaodong HeEMNLP 2022 · 6 citations
  5. RealTalk-CN: A Realistic Chinese Speech Task-Oriented Dialogue Benchmark with Cross-Modal AnalysisEnzhi Wang, Jiaming Zhou, Yuhang Jia, Aobo Kong et al.ACL 2026
Dusk over the sea between two cliffs drawn in fine vertical lines