Lune

ACL Findings2024

Direct Evaluation of Chain-of-Thought in Multi-hop Reasoning with Knowledge Graphs

Thi Nguyen, Linhao Luo, Fatemeh Shiri, Dinh Phung, Yuan-Fang Li, Thuy-Trang Vu, Gholamreza Haffari

Abstract

Large language models (LLMs) have demonstrated strong reasoning abilities when prompted to generate chain-of-thought (CoT) explanations alongside answers. However, previous research on evaluating LLMs has solely focused on answer accuracy, neglecting the correctness of the generated CoT. In this paper, we delve deeper into the CoT reasoning capabilities of LLMs in multi-hop question answering by utilizing knowledge graphs (KGs). We propose a novel discriminative and generative CoT evaluation paradigm to assess LLMs' knowledge of reasoning and the accuracy of the generated CoT. Through experiments conducted on 5 different families of LLMs across 2 multi-hop question-answering datasets, we find that LLMs possess sufficient knowledge to perform reasoning. However, there exists a significant disparity between answer accuracy and faithfulness of the CoT generated by LLMs, indicating that they often arrive at correct answers through incorrect reasoning. 1

Ask Lune

Know this paper in minutes, not an afternoon.

Lune can read every page of it and the top-tier papers that cite it, and gives you its claims, its evidence and its limits, each one cited to the paper it comes from. No skimming, no AI slop.

16 top-tier papers have cited it since. Know what they found before your reviewers do.

Questions to start from

Lune can draw on

the full paper · the top-tier papers that cite it · related work

Get the answer

Free to start. No credit card required.

Lineage

  • Tree of Thoughts: Deliberate Problem Solving with Large Language Models cites Self-Consistency Improves Chain of Thought Reasoning in Language Models
  • Tree of Thoughts: Deliberate Problem Solving with Large Language Models cites Language Models are Few-Shot Learners
  • Tree of Thoughts: Deliberate Problem Solving with Large Language Models cites Chain-of-Thought Prompting Elicits Reasoning in Large Language Models
  • Self-Consistency Improves Chain of Thought Reasoning in Language Models cites Large Language Models are Zero-Shot Reasoners
  • Large Language Models are Zero-Shot Reasoners cites Self-Consistency Improves Chain of Thought Reasoning in Language Models
  • Self-Consistency Improves Chain of Thought Reasoning in Language Models cites Language Models are Few-Shot Learners
  • Self-Consistency Improves Chain of Thought Reasoning in Language Models cites Chain-of-Thought Prompting Elicits Reasoning in Large Language Models
  • Chain-of-Thought Prompting Elicits Reasoning in Large Language Models cites Self-Consistency Improves Chain of Thought Reasoning in Language Models
  • Do LLM Agents Know How to Ground, Recover, and Assess? Evaluating Epistemic Competence in Information-Seeking Agents cites Chain-of-Thought Prompting Elicits Reasoning in Large Language Models
  • Large Language Models are Zero-Shot Reasoners cites Language Models are Few-Shot Learners
  • Large Language Models are Zero-Shot Reasoners cites Chain-of-Thought Prompting Elicits Reasoning in Large Language Models
  • Are Language Models Efficient Reasoners? A Perspective from Logic Programming cites Large Language Models are Zero-Shot Reasoners
  • Are Language Models Efficient Reasoners? A Perspective from Logic Programming cites Language Models are Few-Shot Learners
  • Compress the Easy, Explore the Hard: Difficulty-Aware Entropy Regularization for Efficient LLM Reasoning cites Chain-of-Thought Prompting Elicits Reasoning in Large Language Models
  • An Empirical Analysis on Spatial Reasoning Capabilities of Large Multimodal Models cites Language Models are Few-Shot Learners
  • An Empirical Analysis on Spatial Reasoning Capabilities of Large Multimodal Models cites Chain-of-Thought Prompting Elicits Reasoning in Large Language Models
Builds onCited byLeft to right by year; the more cited, the higher.

Cited by top-tier papers16

  1. An Empirical Analysis on Spatial Reasoning Capabilities of Large Multimodal ModelsFatemeh Shiri, Xiao-Yu Guo, Mona Far, Xin Yu et al.EMNLP 2024 · 7 citations
  2. TraPO: A Semi-Supervised Reinforcement Learning Framework for Boosting LLM ReasoningShenzhi Yang, Guangcheng Zhu, Haobo Wang, Xing Zheng et al.ICLR 2026 · 6 citations
  3. Do LLM Agents Know How to Ground, Recover, and Assess? Evaluating Epistemic Competence in Information-Seeking AgentsJiaqi Shao, Yuxiang Lin, Munish Prasad Lohani, Yufeng Miao et al.ICLR 2026 · 4 citations
  4. Are Language Models Efficient Reasoners? A Perspective from Logic ProgrammingAndreas Opedal, Yanick Zengaffinen, Haruki Shirakami, Clemente Pasti et al.NeurIPS 2025 · 4 citations
  5. Compress the Easy, Explore the Hard: Difficulty-Aware Entropy Regularization for Efficient LLM ReasoningQin-Wen Luo, Sheng Ren, Xiang Chen, Rui Liu et al.KDD 2026 · 4 citations

Builds on13

  1. Language Models are Few-Shot LearnersTom B. Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah et al.NeurIPS 2020 · 64,255 citations
  2. Chain-of-Thought Prompting Elicits Reasoning in Large Language ModelsJason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma et al.NeurIPS 2022 · 22,562 citations
  3. Large Language Models are Zero-Shot ReasonersTakeshi Kojima, Shixiang Shane Gu, Machel Reid, Yutaka Matsuo et al.NeurIPS 2022 · 8,168 citations
  4. Tree of Thoughts: Deliberate Problem Solving with Large Language ModelsShunyu Yao, Dian Yu, Jeffrey Zhao, Izhak Shafran et al.NeurIPS 2023 · 5,068 citations
  5. Self-Consistency Improves Chain of Thought Reasoning in Language ModelsXuezhi Wang, Jason Wei, Dale Schuurmans, Quoc V. Le et al.ICLR 2023 · 681 citations

Related papers

  1. KAM-CoT: Knowledge Augmented Multimodal Chain-of-Thoughts ReasoningDebjyoti Mondal, Suraj Modi, Subhadarshi Panda, Rituraj Singh et al.AAAI 2024 · 96 citations
  2. Understanding Chain-of-Thought in LLMs through Information TheoryJean-Francois Ton, Muhammad Faaiz Taufiq, Yang LiuICML 2025
  3. DCMKC: A Dual Consistency Matching Approach for Multi-hop Question Answering in LLMsXinyi Wang, Yiping Song, Chang Liu, Tingjin Luo et al.EMNLP Findings 2025
  4. Mapping the Minds of LLMs: A Graph-Based Analysis of Reasoning LLMsZhen Xiong, Yujun Cai, Zhecheng Li, Yiwei WangEMNLP 2025
  5. Threading the Needle: Reweaving Chain-of-Thought Reasoning to Explain Human Label VariationBeiduo Chen, Yang Janet Liu, Anna Korhonen, Barbara PlankEMNLP 2025
Dusk over the sea between two cliffs drawn in fine vertical lines