EMNLP Findings2024
Language Models Still Struggle to Zero-shot Reason about Time Series
Mike A. Merrill, Mingtian Tan, Vinayak Gupta, Thomas Hartvigsen, Tim Althoff
摘要
Time series are critical for decision making in fields like finance and healthcare. Their importance has driven a recent influx of works passing time series into language models, leading to non-trivial forecasting on some datasets. But it remains unknown whether non-trivial forecasting implies that language models can reason about time series. To address this gap, we introduce a first-of-its-kind evaluation framework for time series reasoning, including formal tasks and a corresponding dataset of multi-scale time series paired with text captions across ten domains. Using these data, we probe whether language models achieve three forms of reasoning: (1) Etiological Reasoning-given an input time series, can the language model identify the scenario that most likely created it? (2) Question Answering-can a language model answer factual questions about time series? (3) Context-Aided Forecasting-does relevant textual context improve a language model's time series forecasts? We find that otherwise highlycapable language models demonstrate surprisingly limited time series reasoning: they score marginally above random on etiological and question answering tasks (up to 30 percentage points worse than humans) and show modest success in using context to improve forecasting. These weakness showcase that time series reasoning is an impactful, yet deeply underdeveloped direction for language model research. 1
问 Lune
几分钟读透这篇 Paper,不用再花一下午。
Lune 能读完全文,也能读引用它的顶会 Paper,讲清它的结论、证据和局限,每一点都注明出自哪篇 Paper。不用硬啃,也不靠猜。
已有 29 篇顶会 Paper 引用了它。别等审稿人问起,先弄清它们发现了什么。
脉络
- UniTS: A Unified Multi-Task Time Series Model 引用了 Visual Instruction Tuning
- UniTS: A Unified Multi-Task Time Series Model 引用了 Large Language Models Are Zero-Shot Time Series Forecasters
- UniTS: A Unified Multi-Task Time Series Model 引用了 Time-LLM: Time Series Forecasting by Reprogramming Large Language Models
- UniTS: A Unified Multi-Task Time Series Model 引用了 Are Language Models Actually Useful for Time Series Forecasting?
- UniTS: A Unified Multi-Task Time Series Model 引用了 One Fits All: Power General Time Series Analysis by Pretrained LM
- ChatTime: A Unified Multimodal Time Series Foundation Model Bridging Numerical and Textual Data 引用了 Large Language Models Are Zero-Shot Time Series Forecasters
- ChatTime: A Unified Multimodal Time Series Foundation Model Bridging Numerical and Textual Data 引用了 Time-LLM: Time Series Forecasting by Reprogramming Large Language Models
- ChatTime: A Unified Multimodal Time Series Foundation Model Bridging Numerical and Textual Data 引用了 One Fits All: Power General Time Series Analysis by Pretrained LM
- Large Language Models Are Zero-Shot Time Series Forecasters 引用了 One Fits All: Power General Time Series Analysis by Pretrained LM
- Time-LLM: Time Series Forecasting by Reprogramming Large Language Models 引用了 Large Language Models Are Zero-Shot Time Series Forecasters
- Time-LLM: Time Series Forecasting by Reprogramming Large Language Models 引用了 One Fits All: Power General Time Series Analysis by Pretrained LM
- ChatTS: Aligning Time Series with LLMs via Synthetic Data for Enhanced Understanding and Reasoning 引用了 Visual Instruction Tuning
- ChatTS: Aligning Time Series with LLMs via Synthetic Data for Enhanced Understanding and Reasoning 引用了 Large Language Models Are Zero-Shot Time Series Forecasters
- ChatTS: Aligning Time Series with LLMs via Synthetic Data for Enhanced Understanding and Reasoning 引用了 Time-LLM: Time Series Forecasting by Reprogramming Large Language Models
- ChatTS: Aligning Time Series with LLMs via Synthetic Data for Enhanced Understanding and Reasoning 引用了 One Fits All: Power General Time Series Analysis by Pretrained LM
- Are Language Models Actually Useful for Time Series Forecasting? 引用了 Large Language Models Are Zero-Shot Time Series Forecasters
- Are Language Models Actually Useful for Time Series Forecasting? 引用了 Time-LLM: Time Series Forecasting by Reprogramming Large Language Models
- Are Language Models Actually Useful for Time Series Forecasting? 引用了 One Fits All: Power General Time Series Analysis by Pretrained LM
引用它的顶会 Paper29
- Are Language Models Actually Useful for Time Series Forecasting?Mingtian Tan, Mike A. Merrill, Vinayak Gupta, Tim Althoff 等NeurIPS 2024 · 326 次引用
- UniTS: A Unified Multi-Task Time Series ModelShanghua Gao, Teddy Koker, Owen Queen, Tom Hartvigsen 等NeurIPS 2024 · 159 次引用
- ChatTime: A Unified Multimodal Time Series Foundation Model Bridging Numerical and Textual DataChengsen Wang, Qi Qi, Jingyu Wang, Haifeng Sun 等AAAI 2025 · 109 次引用
- GPTCoach: Towards LLM-Based Physical Activity CoachingMatthew Jörke, Shardul Sapkota, Lyndsea Warkenthien, Niklas Vainio 等CHI 2025 · 89 次引用
- ChatTS: Aligning Time Series with LLMs via Synthetic Data for Enhanced Understanding and ReasoningZhe Xie, Zeyan Li, Xiao He, Longlong Xu 等VLDB 2025 · 87 次引用
它引用的 Paper16
- Visual Instruction TuningHaotian Liu, Chunyuan Li, Qingyang Wu, Yong Jae LeeNeurIPS 2023 · 11,349 次引用
- Robust Speech Recognition via Large-Scale Weak SupervisionAlec Radford, Jong Wook Kim, Tao Xu, Greg Brockman 等ICML 2023 · 6,966 次引用
- One Fits All: Power General Time Series Analysis by Pretrained LMTian Zhou, Peisong Niu, Xue Wang, Liang Sun 等NeurIPS 2023 · 1,178 次引用
- Time-LLM: Time Series Forecasting by Reprogramming Large Language ModelsMing Jin, Shiyu Wang, Lintao Ma, Zhixuan Chu 等ICLR 2024 · 915 次引用
- Large Language Models Are Zero-Shot Time Series ForecastersNate Gruver, Marc Finzi, Shikai Qiu, Andrew Gordon WilsonNeurIPS 2023 · 898 次引用
相关 Paper
- TimeSeriesExamAgent: Creating Time Series Reasoning Benchmarks at ScaleMalgorzata Gwiazda, Yifu Cai, Mononito Goswami, Arjun Choudhry 等ICLR 2026 · 6 次引用
- Time-MQA: Time Series Multi-Task Question Answering with Context EnhancementYaxuan Kong, Yiyuan Yang, Yoontae Hwang, Wenjie Du 等ACL 2025
- Do Language Models Have a Common Sense regarding Time? Revisiting Temporal Commonsense Reasoning in the Era of Large Language ModelsRaghav Jain, Daivik Sojitra, Arkadeep Acharya, Sriparna Saha 等EMNLP 2023 · 17 次引用
- HEARTS: Benchmarking LLM Reasoning on Health Time SeriesSirui Li, Shuhan Xiao, Mihir Joshi, Ahmed Metwally 等ICML 2026 · 7 次引用
- TsLLM: Augmenting LLMs for General Time Series Understanding and PredictionFelix Parker, Nimeesha Chan, Chi Zhang, Kimia GhobadiICML 2026 · 3 次引用
