CoTKR: Chain-of-Thought Enhanced Knowledge Rewriting for Complex Knowledge Graph Question Answering

Recent studies have explored the use of Large Language Models (LLMs) with Retrieval Augmented Generation (RAG) for Knowledge Graph Question Answering (KGQA). They typically require rewriting retrieved subgraphs into natural language formats comprehensible to LLMs. However, when tackling complex questions, the knowledge rewritten by existing methods may include irrelevant information, omit crucial details, or fail to align with the question's semantics. To address them, we propose a novel rewriting method CoTKR, Chain-of-Thought Enhanced Knowledge Rewriting, for generating reasoning traces and corresponding knowledge in an interleaved manner, thereby mitigating the limitations of single-step knowledge rewriting. Additionally, to bridge the preference gap between the knowledge rewriter and the question answering (QA) model, we propose a training strategy PAQAF, Preference Alignment from Question Answering Feedback, for leveraging feedback from the QA model to further optimize the knowledge rewriter. We conduct experiments using various LLMs across several KGQA benchmarks. Experimental results demonstrate that, compared with previous knowledge rewriting methods, CoTKR generates the most beneficial knowledge representation for QA models, which significantly improves the performance of LLMs in KGQA 1 . * Equal contribution. †Corresponding author. 1 Our code is available at https://github.com/wuyike2000/CoTKR . Question: what is the unit of area that the measurement system that have an electric field strength units of volt per metre have? Answer: Square kilometer, Square meter KR Methods KR Results Summary The International System of Units includes various measurement systems for different physical quantities such as length, area, volume, conductance, resistance, current, force, and pressure. The measurement system with an electric field strength unit of Volt per meter does not directly relate to a specific area unit within the International System of Units. CoTKR Reason: I need to know the unit of area associated with the measurement system that has an electric field strength unit of Volt per meter. Summarize: The International System of Units includes various units for different physical quantities such as length, area, volume, etc. Reason: I need to find the specific unit of area in the International System of Units that corresponds to the electric field strength unit of Volt per meter. Summarize: The International System of Units includes units like Square kilometer, Square meter, etc. for measuring area. CoTKR+PA Reason: I need to know which measurement system includes the unit for electric field strength in Volt per meter. Knowledge: The International System of Units contains the unit for electric field strength in Volt per meter. Reason: I need to identify the unit of area within the International System of Units. Knowledge: The International System of Units includes the units Square kilometer and Square meter as area units.