EMNLP Findings2023
Beneath Surface Similarity: Large Language Models Make Reasonable Scientific Analogies after Structure Abduction
Siyu Yuan, Jiangjie Chen, Xuyang Ge, Yanghua Xiao, Deqing Yang
Abstract
The vital role of analogical reasoning in human cognition allows us to grasp novel concepts by linking them with familiar ones through shared relational structures. Despite the attention previous research has given to word analogies, this work suggests that Large Language Models (LLMs) often overlook the structures that underpin these analogies, raising questions about the efficacy of word analogies as a measure of analogical reasoning skills akin to human cognition. In response to this, our paper introduces a task of analogical structure abduction, grounded in cognitive psychology, designed to abduce structures that form an analogy between two systems. In support of this task, we establish a benchmark called SCAR, containing 400 scientific analogies from 13 distinct fields, tailored for evaluating analogical reasoning with structure abduction. The empirical evidence underlines the continued challenges faced by LLMs, including ChatGPT and GPT-4, in mastering this task, signifying the need for future exploration to enhance their abilities. 1
Ask Lune
Know this paper in minutes, not an afternoon.
Lune can read every page of it and the top-tier papers that cite it, and gives you its claims, its evidence and its limits, each one cited to the paper it comes from. No skimming, no AI slop.
3 top-tier papers have cited it since. Know what they found before your reviewers do.
Lune can draw on
the full paper · the top-tier papers that cite it · related work
Free to start. No credit card required.
Lineage
- Unlocking Scientific Concepts: How Effective Are LLM-Generated Analogies for Student Understanding and Classroom Practice? cites E-KAR: A Benchmark for Rationalizing Natural Language Analogical Reasoning
- Can Large Language Models Generalize Procedures Across Representations? cites Chain-of-Thought Prompting Elicits Reasoning in Large Language Models
- Large Language Models are Zero-Shot Reasoners cites Chain-of-Thought Prompting Elicits Reasoning in Large Language Models
- Past Meets Present: Creating Historical Analogy with Large Language Models cites E-KAR: A Benchmark for Rationalizing Natural Language Analogical Reasoning
Cited by top-tier papers3
- Unlocking Scientific Concepts: How Effective Are LLM-Generated Analogies for Student Understanding and Classroom Practice?Zekai Shao, Siyu Yuan, Lin Gao, Yixuan He et al.CHI 2025 · 12 citations
- Can Large Language Models Generalize Procedures Across Representations?Fangru Lin, Valentin Hofmann, Xingchen Wan, Weixing Wang et al.ICML 2026 · 2 citations
- Past Meets Present: Creating Historical Analogy with Large Language ModelsNianqi Li, Siyu Yuan, Jiangjie Chen, Jiaqing Liang et al.ACL 2025
Builds on12
- Chain-of-Thought Prompting Elicits Reasoning in Large Language ModelsJason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma et al.NeurIPS 2022 · 22,562 citations
- Large Language Models are Zero-Shot ReasonersTakeshi Kojima, Shixiang Shane Gu, Machel Reid, Yutaka Matsuo et al.NeurIPS 2022 · 8,168 citations
- SimCSE: Simple Contrastive Learning of Sentence EmbeddingsTianyu Gao, Xingcheng Yao, Danqi ChenEMNLP 2021 · 2,496 citations
- Can ChatGPT Assess Human Personalities? A General Evaluation FrameworkHaocong Rao, Cyril Leung, Chunyan MiaoEMNLP Findings 2023 · 45 citations
- E-KAR: A Benchmark for Rationalizing Natural Language Analogical ReasoningJiangjie Chen, Rui Xu, Ziquan Fu, Wei Shi et al.ACL Findings 2022 · 21 citations
Related papers
- AnaloBench: Benchmarking the Identification of Abstract and Long-context AnalogiesXiao Ye, Andrew Wang, Jacob Choi, Yining Lu et al.EMNLP 2024 · 3 citations
- KiVA: Kid-inspired Visual Analogies for Testing Large Multimodal ModelsEunice Yiu, Maan Qraitem, Anisa Noor Majhi, Charlie Wong et al.ICLR 2025
- StoryAnalogy: Deriving Story-level Analogies from Large Language Models to Unlock Analogical UnderstandingCheng Jiayang, Lin Qiu, Tsz Ho Chan, Tianqing Fang et al.EMNLP 2023 · 8 citations
- Scientific and Creative Analogies in Pretrained Language ModelsTamara Czinczoll, Helen Yannakoudakis, Pushkar Mishra, Ekaterina ShutovaEMNLP Findings 2022 · 14 citations
- VOILA: Evaluation of MLLMs For Perceptual Understanding and Analogical ReasoningNilay Yilmaz, Maitreya Patel, Yiran Lawrence Luo, Tejas Gokhale et al.ICLR 2025
