EMNLP Findings2023
Fusing Temporal Graphs into Transformers for Time-Sensitive Question Answering
Abstract
Answering time-sensitive questions from long documents requires temporal reasoning over the times in questions and documents.An important open question is whether large language models can perform such reasoning solely using a provided text document, or whether they can benefit from additional temporal information extracted using other systems.We address this research question by applying existing temporal information extraction systems to construct temporal graphs of events, times, and temporal relations in questions and documents.We then investigate different approaches for fusing these graphs into Transformer models.Experimental results show that our proposed approach for fusing temporal graphs into input text substantially enhances the temporal reasoning capabilities of Transformer models with or without fine-tuning.Additionally, our proposed method outperforms various graph convolution-based approaches and establishes a new state-of-the-art performance on Situat-edQA and three splits of TimeQA.1 https://github.com/nchambers/caevo
Ask Lune
Know this paper in minutes, not an afternoon.
Lune can read every page of it and the top-tier papers that cite it, and gives you its claims, its evidence and its limits, each one cited to the paper it comes from. No skimming, no AI slop.
2 top-tier papers have cited it since. Know what they found before your reviewers do.
Lune can draw on
the full paper · the top-tier papers that cite it · related work
Free to start. No credit card required.
Lineage
Cited by top-tier papers2
Builds on9
- Big Bird: Transformers for Longer SequencesManzil Zaheer, Guru Guruganesh, Kumar Avinava Dubey, Joshua Ainslie et al.NeurIPS 2020 · 3,159 citations
- GreaseLM: Graph REASoning Enhanced Language ModelsXikun Zhang, Antoine Bosselut, Michihiro Yasunaga, Hongyu Ren et al.ICLR 2022 · 285 citations
- Interleaving Retrieval with Chain-of-Thought Reasoning for Knowledge-Intensive Multi-Step QuestionsHarsh Trivedi, Niranjan Balasubramanian, Tushar Khot, Ashish SabharwalACL 2023 · 187 citations
- TempoQR: Temporal Question Reasoning over Knowledge GraphsCostas Mavromatis, Prasanna Lakkur Subramanyam, Vassilis N. Ioannidis, Adesoji Adeshina et al.AAAI 2022 · 77 citations
- RAT-SQL: Relation-Aware Schema Encoding and Linking for Text-to-SQL ParsersBailin Wang, Richard Shin, Xiaodong Liu, Oleksandr Polozov et al.ACL 2020 · 39 citations
Related papers
- MTGER: Multi-view Temporal Graph Enhanced Temporal Reasoning over Time-Involved DocumentZheng Chu, Zekun Wang, Jiafeng Liang, Ming Liu et al.EMNLP Findings 2023
- Enhancing Temporal Sensitivity and Reasoning for Time-Sensitive Question AnsweringWanqi Yang, Yanda Li, Meng Fang, Ling ChenEMNLP Findings 2024 · 4 citations
- TimeR⁴ : Time-aware Retrieval-Augmented Large Language Models for Temporal Knowledge Graph Question AnsweringXinying Qian, Ying Zhang, Yu Zhao, Baohang Zhou et al.EMNLP 2024 · 11 citations
- It's High Time: A Survey of Temporal Question AnsweringBhawna Piryani, Abdelrahman Abdallah, Jamshid Mozafari, Avishek Anand et al.ACL 2026 · 6 citations
- ITFormer: Bridging Time Series and Natural Language for Multi-Modal QA with Large-Scale Multitask DatasetYilin Wang, Peixuan Lei, Jie Song, Yuzhe Hao et al.ICML 2025
