Lune

EMNLP Findings2025

Following Occam's Razor: Dynamic Combination of Structured Knowledge for Multi-Hop Question Answering using LLMs

Wei Chen, Zhi Zheng, Lili Zhao, Huijun Hou, Tong Xu

Abstract

Multi-hop question answering is a challenging task that requires capturing information from different positions in multiple documents. Recently, several methods propose to enhance Large Language Models (LLMs) by incorporating structured knowledge, aiming to grasp key information for solving this task. Despite certain achievements, they still face the following challenges: 1) The neglect of text-based reasoning capabilities. 2) Information redundancy between text and triples. 3) Information loss during structured knowledge extraction. To solve the above challenges, in this paper, we propose Dynamic Combination of Structured Knowledge (DCSK), a novel framework for integrating text-based and triple-based paradigms. Following Occam's Razor, DCSK dynamically determine the necessity of structured knowledge by the designed multi-faceted evaluation, which systematically assess the correctness, clarity, and informativeness of text-based prediction. For questions that require structured knowledge, we develop an iterative fact refiner that screens for question-relevant triples, verifies their factual adequacy, and thereby effectively excludes irrelevant and redundant information. Furthermore, based on the verification, we construct an adaptive knowledge reasoner that dynamically adjusts the need for text supplementation, thus mitigating the information deficiency in selected triples. Extensive experiments on three MHQA datasets demonstrate the efficiency and effectiveness of DCSK 1 .

Ask Lune

Know this paper in minutes, not an afternoon.

Lune can read every page of it and the top-tier papers that cite it, and gives you its claims, its evidence and its limits, each one cited to the paper it comes from. No skimming, no AI slop.

1 top-tier paper has cited it since. Know what they found before your reviewers do.

Questions to start from

Lune can draw on

the full paper · the top-tier papers that cite it · related work

Get the answer

Free to start. No credit card required.

Lineage

  • Measuring and Narrowing the Compositionality Gap in Language Models cites Chain-of-Thought Prompting Elicits Reasoning in Large Language Models
  • Self-Consistency Improves Chain of Thought Reasoning in Language Models cites Learning to Retrieve Reasoning Paths over Wikipedia Graph for Question Answering
  • Self-Consistency Improves Chain of Thought Reasoning in Language Models cites Chain-of-Thought Prompting Elicits Reasoning in Large Language Models
  • Chain-of-Thought Prompting Elicits Reasoning in Large Language Models cites Self-Consistency Improves Chain of Thought Reasoning in Language Models
  • Hierarchical Graph Network for Multi-hop Question Answering cites Learning to Retrieve Reasoning Paths over Wikipedia Graph for Question Answering
  • STRIDE: Strategic Iterative Decision-Making for Retrieval-Augmented Multi-Hop Question Answering cites Measuring and Narrowing the Compositionality Gap in Language Models
  • STRIDE: Strategic Iterative Decision-Making for Retrieval-Augmented Multi-Hop Question Answering cites Hierarchical Graph Network for Multi-hop Question Answering
  • STRIDE: Strategic Iterative Decision-Making for Retrieval-Augmented Multi-Hop Question Answering cites Chain-of-Thought Prompting Elicits Reasoning in Large Language Models
Builds onCited byLeft to right by year; the more cited, the higher.

Cited by top-tier papers1

  1. STRIDE: Strategic Iterative Decision-Making for Retrieval-Augmented Multi-Hop Question AnsweringWei Chen, Lili Zhao, Zhi Zheng, Huijun Hou et al.SIGIR 2026

Builds on17

  1. Chain-of-Thought Prompting Elicits Reasoning in Large Language ModelsJason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma et al.NeurIPS 2022 · 22,562 citations
  2. Self-Consistency Improves Chain of Thought Reasoning in Language ModelsXuezhi Wang, Jason Wei, Dale Schuurmans, Quoc V. Le et al.ICLR 2023 · 681 citations
  3. Learning to Retrieve Reasoning Paths over Wikipedia Graph for Question AnsweringAkari Asai, Kazuma Hashimoto, Hannaneh Hajishirzi, Richard Socher et al.ICLR 2020 · 322 citations
  4. Measuring and Narrowing the Compositionality Gap in Language ModelsOfir Press, Muru Zhang, Sewon Min, Ludwig Schmidt et al.EMNLP Findings 2023 · 296 citations
  5. Hierarchical Graph Network for Multi-hop Question AnsweringYuwei Fang, Siqi Sun, Zhe Gan, Rohit Pillai et al.EMNLP 2020 · 157 citations

Related papers

  1. DAMR: Efficient and Adaptive Context-Aware Knowledge Graph Question Answering with LLM-Guided MCTSYingxu Wang, Shiqi Fan, Mengzhu Wang, Siyang Gao et al.ICLR 2026 · 7 citations
  2. Resource-Friendly Dynamic Enhancement Chain for Multi-Hop Question AnsweringBinquan Ji, Haibo Luo, Yifei Lu, Lei Hei et al.ACL Findings 2025 · 1 citation
  3. SEARCH-R: Structured Entity-Aware Retrieval with Chain-of-Reasoning Navigator for Multi-hop Question AnsweringYuqing Fu, Yimin Deng, Wanyu Wang, Yuhao Wang et al.ACL Findings 2026
  4. DTKG: Dual-Track Knowledge Graph-Verified Reasoning Framework for Multi-Hop QAChanghao Wang, Yanfang Liu, Xinxin Fan, Ao Tian et al.ICML 2026
  5. Reflection on Knowledge Graph for Large Language Models ReasoningYigeng Zhou, Wu Li, Yifan Lu, Jing Li et al.ACL Findings 2025 · 8 citations
Dusk over the sea between two cliffs drawn in fine vertical lines