Lune

AAAI2021Top-tier venue

(Comet-) Atomic 2020: On Symbolic and Neural Commonsense Knowledge Graphs

Jena D. Hwang, Chandra Bhagavatula, Ronan Le Bras, Jeff Da, Keisuke Sakaguchi, Antoine Bosselut, Yejin Choi

Abstract

Recent years have brought about a renewed interest in commonsense representation and reasoning in the field of natural language understanding. The development of new commonsense knowledge graphs (CSKG) has been central to these advances as their diverse facts can be used and referenced by machine learning models for tackling new and challenging tasks. At the same time, there remain questions about the quality and coverage of these resources due to the massive scale required to comprehensively encompass general commonsense knowledge. In this work, we posit that manually constructed CSKGs will never achieve the coverage necessary to be applicable in all situations encountered by NLP agents. Therefore, we propose a new evaluation framework for testing the utility of KGs based on how effectively implicit knowledge representations can be learned from them. With this new goal, we propose ATOMIC 20 20 , a new CSKG of general-purpose commonsense knowledge containing knowledge that is not readily available in pretrained language models. We evaluate its properties in comparison with other leading CSKGs, performing the first large-scale pairwise study of commonsense knowledge resources. Next, we show that ATOMIC 20 20 is better suited for training knowledge models that can generate accurate, representative knowledge for new, unseen entities and events. Finally, through human evaluation, we show that the few-shot performance of GPT-3 (175B parameters), while impressive, remains ∼12 absolute points lower than a BART-based knowledge model trained on ATOMIC 20 20 despite using over 430x fewer parameters.

Ask Lune

Know this paper in minutes, not an afternoon.

Lune can read every page of it and the top-tier papers that cite it, and gives you its claims, its evidence and its limits, each one cited to the paper it comes from. No skimming, no AI slop.

82 top-tier papers have cited it since. Know what they found before your reviewers do.

Questions to start from

Lune can draw on

the full paper · the top-tier papers that cite it · related work

Get the answer

Free to start. No credit card required.

Lineage

  • IntentQA: Context-aware Video Intent Reasoning cites Neural Theory-of-Mind? On the Limits of Social Intelligence in Large LMs
  • ClarET: Pre-training a Correlation-Aware Context-To-Event Transformer for Event-Centric Generation and Classification cites BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehension
  • ClarET: Pre-training a Correlation-Aware Context-To-Event Transformer for Event-Centric Generation and Classification cites BERTScore: Evaluating Text Generation with BERT
  • GreaseLM: Graph REASoning Enhanced Language Models cites Scalable Multi-Hop Relational Reasoning for Knowledge-Aware Question Answering
Builds onCited byLeft to right by year; the more cited, the higher.

Cited by top-tier papers82

  1. GreaseLM: Graph REASoning Enhanced Language ModelsXikun Zhang, Antoine Bosselut, Michihiro Yasunaga, Hongyu Ren et al.ICLR 2022 · 285 citations
  2. IntentQA: Context-aware Video Intent ReasoningJiapeng Li, Ping Wei, Wenjuan Han, Lifeng FanICCV 2023 · 97 citations
  3. Neural Theory-of-Mind? On the Limits of Social Intelligence in Large LMsMaarten Sap, Ronan Le Bras, Daniel Fried, Yejin ChoiEMNLP 2022 · 92 citations
  4. Moral Stories: Situated Reasoning about Norms, Intents, Actions, and their ConsequencesDenis Emelin, Ronan Le Bras, Jena D. Hwang, Maxwell Forbes et al.EMNLP 2021 · 83 citations
  5. ClarET: Pre-training a Correlation-Aware Context-To-Event Transformer for Event-Centric Generation and ClassificationYucheng Zhou, Tao Shen, Xiubo Geng, Guodong Long et al.ACL 2022 · 76 citations

Builds on9

  1. Language Models are Few-Shot LearnersTom B. Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah et al.NeurIPS 2020 · 64,255 citations
  2. BERTScore: Evaluating Text Generation with BERTTianyi Zhang, Varsha Kishore, Felix Wu, Kilian Q. Weinberger et al.ICLR 2020 · 8,443 citations
  3. PIQA: Reasoning about Physical Commonsense in Natural LanguageYonatan Bisk, Rowan Zellers, Ronan Le Bras, Jianfeng Gao et al.AAAI 2020 · 2,916 citations
  4. BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and ComprehensionMike Lewis, Yinhan Liu, Naman Goyal, Marjan Ghazvininejad et al.ACL 2020 · 1,224 citations
  5. Scalable Multi-Hop Relational Reasoning for Knowledge-Aware Question AnsweringYanlin Feng, Xinyue Chen, Bill Yuchen Lin, Peifeng Wang et al.EMNLP 2020 · 207 citations

Related papers

  1. Dense-ATOMIC: Towards Densely-connected ATOMIC with High Knowledge Coverage and Massive Multi-hop PathsXiangqing Shen, Siwei Wu, Rui XiaACL 2023 · 3 citations
  2. On the Consistency of Commonsense in Large Language ModelsGuozheng Li, Peng Wang, Wenjun Ke, Zijie Xu et al.ACL Findings 2025
  3. DISCOS: Bridging the Gap between Discourse Knowledge and Commonsense KnowledgeTianqing Fang, Hongming Zhang, Weiqi Wang, Yangqiu Song et al.WWW 2021 · 48 citations
  4. Benchmarking Commonsense Knowledge Base Population with an Effective Evaluation DatasetTianqing Fang, Weiqi Wang, Sehyun Choi, Shibo Hao et al.EMNLP 2021 · 17 citations
  5. Evaluating Commonsense in Pre-Trained Language ModelsXuhui Zhou, Yue Zhang, Leyang Cui, Dandan HuangAAAI 2020 · 198 citations
Dusk over the sea between two cliffs drawn in fine vertical lines