Lune

ACL Findings2023

Explanation Graph Generation via Generative Pre-training over Synthetic Graphs

Han Cui, Shangzhan Li, Yu Zhang, Qi Shi

Abstract

The generation of explanation graphs is a significant task that aims to produce explanation graphs in response to user input, revealing the internal reasoning process. This task is challenging due to the significant discrepancy between unstructured user queries and structured explanation graphs. Current research commonly fine-tunes a text-based pre-trained language model on a small downstream dataset that is annotated with labeled graphs. However, due to the limited scale of available datasets, this approach may prove to be insufficient in bridging the gap between natural language text and structured graphs. In this paper, to alleviate the above limitations, we propose a novel pre-trained framework EG 3 P(for Explanation Graph Generation via Generative Pre-training over synthetic graphs) for the explanation graph generation task. Specifically, we first propose a text-to-graph generative task to pre-train the model with the goal of bridging the text-graph gap. Additionally, we propose an automatic corpus synthesis strategy for synthesizing a large scale of high-quality corpus, reducing the reliance on costly manual annotation methods. Experimental results on ExplaGraphs show the effectiveness of EG 3 P that our model surpasses all baseline systems with remarkable margins. Besides, further analysis demonstrates that EG 3 P is able to generate better explanation graphs on actual reasoning tasks such as CommonsenseQA and OpenbookQA. 1 1 Our code, checkpoints and corpus is released in https://github.com/cccccent/EG3P

Ask Lune

Know this paper in minutes, not an afternoon.

Lune can read every page of it and the top-tier papers that cite it, and gives you its claims, its evidence and its limits, each one cited to the paper it comes from. No skimming, no AI slop.

Questions to start from

Lune can draw on

the full paper · related work

Get the answer

Free to start. No credit card required.

Lineage

  • TAPEX: Table Pre-training via Learning a Neural SQL Executor cites BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehension
Builds onCited byLeft to right by year; the more cited, the higher.

Builds on18

  1. BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and ComprehensionMike Lewis, Yinhan Liu, Naman Goyal, Marjan Ghazvininejad et al.ACL 2020 · 1,224 citations
  2. GraphAF: a Flow-based Autoregressive Model for Molecular Graph GenerationChence Shi, Minkai Xu, Zhaocheng Zhu, Weinan Zhang et al.ICLR 2020 · 532 citations
  3. QASC: A Dataset for Question Answering via Sentence CompositionTushar Khot, Peter Clark, Michal Guerquin, Peter Jansen et al.AAAI 2020 · 387 citations
  4. TAPEX: Table Pre-training via Learning a Neural SQL ExecutorQian Liu, Bei Chen, Jiaqi Guo, Morteza Ziyadi et al.ICLR 2022 · 347 citations
  5. Sequence-to-Sequence Knowledge Graph Completion and Question AnsweringApoorv Saxena, Adrian Kochsiek, Rainer GemullaACL 2022 · 183 citations

Related papers

  1. ExplaGraphs: An Explanation Graph Generation Task for Structured Commonsense ReasoningSwarnadeep Saha, Prateek Yadav, Lisa Bauer, Mohit BansalEMNLP 2021 · 27 citations
  2. Data-Efficient Concept Extraction from Pre-trained Language Models for Commonsense Explanation GenerationYanbo Fang, Yongfeng ZhangEMNLP Findings 2022 · 5 citations
  3. Connecting the Dots: A Knowledgeable Path Generator for Commonsense Question AnsweringPeifeng Wang, Nanyun Peng, Filip Ilievski, Pedro A. Szekely et al.EMNLP Findings 2020 · 69 citations
  4. Language Models of Code are Few-Shot Commonsense LearnersAman Madaan, Shuyan Zhou, Uri Alon, Yiming Yang et al.EMNLP 2022 · 103 citations
  5. Unsupervised Explanation Generation via Correct InstantiationsSijie Cheng, Zhiyong Wu, Jiangjie Chen, Zhixing Li et al.AAAI 2023 · 6 citations
Dusk over the sea between two cliffs drawn in fine vertical lines