Lune

EMNLP Findings2024

Narrative-of-Thought: Improving Temporal Reasoning of Large Language Models via Recounted Narratives

Xinliang Frederick Zhang, Nicholas Beauchamp, Lu Wang

Abstract

Reasoning about time and temporal relations is an integral aspect of human cognition, essential for perceiving the world and navigating our experiences. Though large language models (LLMs) have demonstrated impressive performance in many reasoning tasks, temporal reasoning remains challenging due to its intrinsic complexity. In this work, we first study an essential task of temporal reasoning-temporal graph generation, to unveil LLMs' inherent, global reasoning capabilities. We show that this task presents great challenges even for the most powerful LLMs, such as GPT-3.5/4. We also notice a significant performance gap by small models (< 10B) that lag behind LLMs by 50%.

Next, we study how to close this gap with a budget constraint, e.g., not using model finetuning. We propose a new prompting technique tailored for temporal reasoning, NARRATIVE-OF-THOUGHT (NOT), that first converts the events set to a Python class, then prompts a small model to generate a temporally grounded narrative, guiding the final generation of a temporal graph. Extensive experiments showcase the efficacy of NOT in improving various metrics. Notably, NOT attains the highest F1 on the Schema-11 evaluation set, while securing an overall F1 on par with GPT-3.5. NOT also achieves the best structural similarity across the board, even compared with GPT-3.5/4. 1

Ask Lune

Know this paper in minutes, not an afternoon.

Lune can read every page of it and the top-tier papers that cite it, and gives you its claims, its evidence and its limits, each one cited to the paper it comes from. No skimming, no AI slop.

6 top-tier papers have cited it since. Know what they found before your reviewers do.

Questions to start from

Lune can draw on

the full paper · the top-tier papers that cite it · related work

Get the answer

Free to start. No credit card required.

Lineage

  • LoRA: Low-Rank Adaptation of Large Language Models cites Language Models are Few-Shot Learners
  • Tree of Thoughts: Deliberate Problem Solving with Large Language Models cites Language Models are Few-Shot Learners
  • Tree of Thoughts: Deliberate Problem Solving with Large Language Models cites Chain-of-Thought Prompting Elicits Reasoning in Large Language Models
  • Large Language Models are Zero-Shot Reasoners cites Language Models are Few-Shot Learners
  • Large Language Models are Zero-Shot Reasoners cites Chain-of-Thought Prompting Elicits Reasoning in Large Language Models
  • ODL-TempLLM: Ontology-Guided and Description Logic-Reasoned Temporal Reasoning with LLMs cites Language Models are Few-Shot Learners
  • Temporal reasoning for timeline summarisation in social media cites LoRA: Low-Rank Adaptation of Large Language Models
  • Do LLMs Really Need 10+ Thoughts for "Find the Time 1000 Days Later"? Towards Structural Understanding of LLM Overthinking cites Large Language Models are Zero-Shot Reasoners
  • Do LLMs Really Need 10+ Thoughts for "Find the Time 1000 Days Later"? Towards Structural Understanding of LLM Overthinking cites Chain-of-Thought Prompting Elicits Reasoning in Large Language Models
Builds onCited byLeft to right by year; the more cited, the higher.

Cited by top-tier papers6

  1. It's High Time: A Survey of Temporal Question AnsweringBhawna Piryani, Abdelrahman Abdallah, Jamshid Mozafari, Avishek Anand et al.ACL 2026 · 6 citations
  2. Temporal reasoning for timeline summarisation in social mediaJiayu Song, Mahmud Elahi Akhter, Dana Atzil-Slonim, Maria LiakataACL 2025 · 6 citations
  3. Do LLMs Really Need 10+ Thoughts for "Find the Time 1000 Days Later"? Towards Structural Understanding of LLM OverthinkingXinliang Frederick Zhang, Anhad Mohananey, Alexandra Chronopoulou, Pinelopi Papalampidi et al.ACL 2026 · 6 citations
  4. PRIME: Large Language Model Personalization with Cognitive Dual-Memory and Personalized Thought ProcessXinliang Frederick Zhang, Nicholas Beauchamp, Lu WangEMNLP 2025
  5. ODL-TempLLM: Ontology-Guided and Description Logic-Reasoned Temporal Reasoning with LLMsJinshuo Liu, Cheng Bi, Meng Wang, Juan Deng et al.ACL 2026

Builds on21

  1. Language Models are Few-Shot LearnersTom B. Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah et al.NeurIPS 2020 · 64,255 citations
  2. Chain-of-Thought Prompting Elicits Reasoning in Large Language ModelsJason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma et al.NeurIPS 2022 · 22,562 citations
  3. LoRA: Low-Rank Adaptation of Large Language ModelsEdward J. Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu et al.ICLR 2022 · 18,833 citations
  4. Large Language Models are Zero-Shot ReasonersTakeshi Kojima, Shixiang Shane Gu, Machel Reid, Yutaka Matsuo et al.NeurIPS 2022 · 8,168 citations
  5. Tree of Thoughts: Deliberate Problem Solving with Large Language ModelsShunyu Yao, Dian Yu, Jeffrey Zhao, Izhak Shafran et al.NeurIPS 2023 · 5,068 citations

Related papers

  1. Do Language Models Have a Common Sense regarding Time? Revisiting Temporal Commonsense Reasoning in the Era of Large Language ModelsRaghav Jain, Daivik Sojitra, Arkadeep Acharya, Sriparna Saha et al.EMNLP 2023 · 17 citations
  2. Large Language Models Can Learn Temporal ReasoningSiheng Xiong, Ali Payani, Ramana Kompella, Faramarz FekriACL 2024
  3. Towards Explainable Temporal Reasoning in Large Language Models: A Structure-Aware Generative FrameworkZihao Jiang, Ben Liu, Miao Peng, Wenjie Xu et al.ACL Findings 2025 · 7 citations
  4. Structure Guided Prompt: Instructing Large Language Model in Multi-Step Reasoning by Exploring Graph Structure of the TextKewei Cheng, Nesreen K. Ahmed, Theodore L. Willke, Yizhou SunEMNLP 2024 · 6 citations
  5. Causal Graph based Event Reasoning using Semantic Relation ExpertsMahnaz Koupaee, Xueying Bai, Mudan Chen, Greg Durrett et al.ACL 2025
Dusk over the sea between two cliffs drawn in fine vertical lines