EMNLP Findings2024
CoTAR: Chain-of-Thought Attribution Reasoning with Multi-level Granularity
Moshe Berchansky, Daniel Fleischer, Moshe Wasserblat, Peter Izsak
Abstract
State-of-the-art performance in QA tasks is currently achieved by systems employing Large Language Models (LLMs), however these models tend to hallucinate information in their responses.One approach focuses on enhancing the generation process by incorporating attribution from the given input to the output.However, the challenge of identifying appropriate attributions and verifying their accuracy against a source is a complex task that requires significant improvements in assessing such systems.We introduce an attribution-oriented Chain-of-Thought reasoning method to enhance the accuracy of attributions.This approach focuses the reasoning process on generating an attributioncentric output.Evaluations on two contextenhanced question-answering datasets using GPT-4 demonstrate improved accuracy and correctness of attributions.In addition, the combination of our method with finetuning enhances the response and attribution accuracy of two smaller LLMs, showing their potential to outperform GPT-4 in some cases. 1 1 Our code is publicly available for reproduction: https: //github.com/mosheber/cotar.git what is the title of tears for fears song?[1] Johnny Panic and the Bible of Dreams...[3] ... Songs from the Big Chair: ...[2] Human eyes produce tears... QuestionLets analyze the relevant spans: From passage [1]: * Johnny Panic and the ... * is a song by the British band ... From passage [3]: * international hit singles * "Mothers Talk", "Shout", .... Thus, the final answer is:
Ask Lune
Know this paper in minutes, not an afternoon.
Lune can read every page of it and the top-tier papers that cite it, and gives you its claims, its evidence and its limits, each one cited to the paper it comes from. No skimming, no AI slop.
4 top-tier papers have cited it since. Know what they found before your reviewers do.
Lune can draw on
the full paper · the top-tier papers that cite it · related work
Free to start. No credit card required.
Lineage
- Attribute or Abstain: Large Language Models as Long Document Assistants cites Enabling Large Language Models to Generate Text with Citations
- Attribute or Abstain: Large Language Models as Long Document Assistants cites Evaluating Verifiability in Generative Search Engines
- MAVIS: A Benchmark for Multimodal Source Attribution in Long-form Visual Question Answering cites Enabling Large Language Models to Generate Text with Citations
- MAVIS: A Benchmark for Multimodal Source Attribution in Long-form Visual Question Answering cites Evaluating Verifiability in Generative Search Engines
- MAVIS: A Benchmark for Multimodal Source Attribution in Long-form Visual Question Answering cites Chain-of-Thought Prompting Elicits Reasoning in Large Language Models
- MAVIS: A Benchmark for Multimodal Source Attribution in Long-form Visual Question Answering cites Attribute First, then Generate: Locally-attributable Grounded Text Generation
- On Synthesizing Data for Context Attribution in Question Answering cites Attribute or Abstain: Large Language Models as Long Document Assistants
- On Synthesizing Data for Context Attribution in Question Answering cites Attribute First, then Generate: Locally-attributable Grounded Text Generation
- The Potential of CoT for Reasoning: A Closer Look at Trace Dynamics cites Chain-of-Thought Prompting Elicits Reasoning in Large Language Models
- Enabling Large Language Models to Generate Text with Citations cites Evaluating Verifiability in Generative Search Engines
- Attribute First, then Generate: Locally-attributable Grounded Text Generation cites The Dawn After the Dark: An Empirical Study on Factuality Hallucination in Large Language Models
- Attribute First, then Generate: Locally-attributable Grounded Text Generation cites Evaluating Verifiability in Generative Search Engines
- Attribute First, then Generate: Locally-attributable Grounded Text Generation cites Chain-of-Thought Prompting Elicits Reasoning in Large Language Models
Cited by top-tier papers4
- The Potential of CoT for Reasoning: A Closer Look at Trace DynamicsGregor Bachmann, Yichen Jiang, Seyed-Mohsen Moosavi-Dezfooli, Moin NabiICLR 2026 · 5 citations
- On Synthesizing Data for Context Attribution in Question AnsweringGorjan Radevski, Kiril Gashteovski, Shahbaz Syed, Christopher Malon et al.ACL 2025 · 1 citation
- Attribute or Abstain: Large Language Models as Long Document AssistantsJan Buchmann, Xiao Liu, Iryna GurevychEMNLP 2024
- MAVIS: A Benchmark for Multimodal Source Attribution in Long-form Visual Question AnsweringSeokwon Song, Minsu Park, Gunhee KimAAAI 2026
Builds on5
- Chain-of-Thought Prompting Elicits Reasoning in Large Language ModelsJason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma et al.NeurIPS 2022 · 22,562 citations
- Enabling Large Language Models to Generate Text with CitationsTianyu Gao, Howard Yen, Jiatong Yu, Danqi ChenEMNLP 2023 · 152 citations
- Evaluating Verifiability in Generative Search EnginesNelson F. Liu, Tianyi Zhang, Percy LiangEMNLP Findings 2023 · 143 citations
- The Dawn After the Dark: An Empirical Study on Factuality Hallucination in Large Language ModelsJunyi Li, Jie Chen, Ruiyang Ren, Xiaoxue Cheng et al.ACL 2024 · 49 citations
- Attribute First, then Generate: Locally-attributable Grounded Text GenerationAviv Slobodkin, Eran Hirsch, Arie Cattan, Tal Schuster et al.ACL 2024
Related papers
- Interleaving Retrieval with Chain-of-Thought Reasoning for Knowledge-Intensive Multi-Step QuestionsHarsh Trivedi, Niranjan Balasubramanian, Tushar Khot, Ashish SabharwalACL 2023 · 187 citations
- Beyond Facts: Evaluating Intent Hallucination in Large Language ModelsYijie Hao, Haofei Yu, Jiaxuan YouACL 2025
- Enhancing Graph Of Thought: Enhancing Prompts with LLM Rationales and Dynamic Temperature ControlSunguk Shin, Youngjoon KimICLR 2025
- CoMAT: Chain of Mathematically Annotated Thought Improves Mathematical ReasoningJoshua Ong Jun Leang, Aryo Pradipta Gema, Shay B. CohenEMNLP 2025
- Wrong-of-Thought: An Integrated Reasoning Framework with Multi-Perspective Verification and Wrong InformationYongheng Zhang, Qiguang Chen, Jingxuan Zhou, Peng Wang et al.EMNLP Findings 2024 · 6 citations
