Lune

ACL Findings2021

Learning Contextualized Knowledge Structures for Commonsense Reasoning

Jun Yan, Mrigank Raman, Aaron Chan, Tianyu Zhang, Ryan A. Rossi, Handong Zhao, Sungchul Kim, Nedim Lipka, Xiang Ren

Abstract

Recently, knowledge graph (KG) augmented models have achieved noteworthy success on various commonsense reasoning tasks. However, KG edge (fact) sparsity and noisy edge extraction/generation often hinder models from obtaining useful knowledge to reason over. To address these issues, we propose a new KG-augmented model: Hybrid Graph Network (HGN). Unlike prior methods, HGN learns to jointly contextualize extracted and generated knowledge by reasoning over both within a unified graph structure. Given the task input context and an extracted KG subgraph, HGN is trained to generate embeddings for the subgraph's missing edges to form a "hybrid" graph, then reason over the hybrid graph while filtering out context-irrelevant edges. We demonstrate HGN's effectiveness through considerable performance gains across four commonsense reasoning benchmarks, plus a user study on edge validness and helpfulness. 1

Ask Lune

Know this paper in minutes, not an afternoon.

Lune can read every page of it and the top-tier papers that cite it, and gives you its claims, its evidence and its limits, each one cited to the paper it comes from. No skimming, no AI slop.

3 top-tier papers have cited it since. Know what they found before your reviewers do.

Questions to start from

Lune can draw on

the full paper · the top-tier papers that cite it · related work

Get the answer

Free to start. No credit card required.

Lineage

  • PINTO: Faithful Language Reasoning Using Prompt-Generated Rationales cites Graph-Based Reasoning over Heterogeneous External Knowledge for Commonsense Question Answering
  • ACENet: Attention Guided Commonsense Reasoning on Hybrid Knowledge Graph cites On the Variance of the Adaptive Learning Rate and Beyond
  • UnifiedQA: Crossing Format Boundaries With a Single QA System cites ALBERT: A Lite BERT for Self-supervised Learning of Language Representations
  • UnifiedQA: Crossing Format Boundaries With a Single QA System cites QASC: A Dataset for Question Answering via Sentence Composition
Builds onCited byLeft to right by year; the more cited, the higher.

Cited by top-tier papers3

  1. AdaLoGN: Adaptive Logic Graph Network for Reasoning-Based Machine Reading ComprehensionXiao Li, Gong Cheng, Ziheng Chen, Yawei Sun et al.ACL 2022 · 29 citations
  2. PINTO: Faithful Language Reasoning Using Prompt-Generated RationalesPeifeng Wang, Aaron Chan, Filip Ilievski, Muhao Chen et al.ICLR 2023 · 21 citations
  3. ACENet: Attention Guided Commonsense Reasoning on Hybrid Knowledge GraphChuzhan Hao, Minghui Xie, Peng ZhangEMNLP 2022 · 5 citations

Builds on10

  1. ALBERT: A Lite BERT for Self-supervised Learning of Language RepresentationsZhenzhong Lan, Mingda Chen, Sebastian Goodman, Kevin Gimpel et al.ICLR 2020 · 7,418 citations
  2. On the Variance of the Adaptive Learning Rate and BeyondLiyuan Liu, Haoming Jiang, Pengcheng He, Weizhu Chen et al.ICLR 2020 · 2,210 citations
  3. UnifiedQA: Crossing Format Boundaries With a Single QA SystemDaniel Khashabi, Sewon Min, Tushar Khot, Ashish Sabharwal et al.EMNLP Findings 2020 · 514 citations
  4. QASC: A Dataset for Question Answering via Sentence CompositionTushar Khot, Peter Clark, Michal Guerquin, Peter Jansen et al.AAAI 2020 · 387 citations
  5. Graph-Based Reasoning over Heterogeneous External Knowledge for Commonsense Question AnsweringShangwen Lv, Daya Guo, Jingjing Xu, Duyu Tang et al.AAAI 2020 · 224 citations

Related papers

  1. KG-BART: Knowledge Graph-Augmented BART for Generative Commonsense ReasoningYe Liu, Yao Wan, Lifang He, Hao Peng et al.AAAI 2021 · 220 citations
  2. Logic-Aware Knowledge Graph Reasoning for Structural Sparsity under Large Language Model SupervisionYudai Pan, Jiajie Hong, Tianzhe Zhao, Lingyun Song et al.WWW 2025 · 1 citation
  3. CAKE: A Scalable Commonsense-Aware Framework For Multi-View Knowledge Graph CompletionGuanglin Niu, Bo Li, Yongfei Zhang, Shiliang PuACL 2022 · 56 citations
  4. Connecting the Dots: A Knowledgeable Path Generator for Commonsense Question AnsweringPeifeng Wang, Nanyun Peng, Filip Ilievski, Pedro A. Szekely et al.EMNLP Findings 2020 · 69 citations
  5. KE-GCL: Knowledge Enhanced Graph Contrastive Learning for Commonsense Question AnsweringLihui Zhang, Ruifan LiEMNLP Findings 2022 · 7 citations
Dusk over the sea between two cliffs drawn in fine vertical lines