Lune

EMNLP Findings2024

Language Models Still Struggle to Zero-shot Reason about Time Series

Mike A. Merrill, Mingtian Tan, Vinayak Gupta, Thomas Hartvigsen, Tim Althoff

Abstract

Time series are critical for decision making in fields like finance and healthcare. Their importance has driven a recent influx of works passing time series into language models, leading to non-trivial forecasting on some datasets. But it remains unknown whether non-trivial forecasting implies that language models can reason about time series. To address this gap, we introduce a first-of-its-kind evaluation framework for time series reasoning, including formal tasks and a corresponding dataset of multi-scale time series paired with text captions across ten domains. Using these data, we probe whether language models achieve three forms of reasoning: (1) Etiological Reasoning-given an input time series, can the language model identify the scenario that most likely created it? (2) Question Answering-can a language model answer factual questions about time series? (3) Context-Aided Forecasting-does relevant textual context improve a language model's time series forecasts? We find that otherwise highlycapable language models demonstrate surprisingly limited time series reasoning: they score marginally above random on etiological and question answering tasks (up to 30 percentage points worse than humans) and show modest success in using context to improve forecasting. These weakness showcase that time series reasoning is an impactful, yet deeply underdeveloped direction for language model research. 1

Ask Lune

Know this paper in minutes, not an afternoon.

Lune can read every page of it and the top-tier papers that cite it, and gives you its claims, its evidence and its limits, each one cited to the paper it comes from. No skimming, no AI slop.

29 top-tier papers have cited it since. Know what they found before your reviewers do.

Questions to start from

Lune can draw on

the full paper · the top-tier papers that cite it · related work

Get the answer

Free to start. No credit card required.

Lineage

  • UniTS: A Unified Multi-Task Time Series Model cites Visual Instruction Tuning
  • UniTS: A Unified Multi-Task Time Series Model cites Large Language Models Are Zero-Shot Time Series Forecasters
  • UniTS: A Unified Multi-Task Time Series Model cites Time-LLM: Time Series Forecasting by Reprogramming Large Language Models
  • UniTS: A Unified Multi-Task Time Series Model cites Are Language Models Actually Useful for Time Series Forecasting?
  • UniTS: A Unified Multi-Task Time Series Model cites One Fits All: Power General Time Series Analysis by Pretrained LM
  • ChatTime: A Unified Multimodal Time Series Foundation Model Bridging Numerical and Textual Data cites Large Language Models Are Zero-Shot Time Series Forecasters
  • ChatTime: A Unified Multimodal Time Series Foundation Model Bridging Numerical and Textual Data cites Time-LLM: Time Series Forecasting by Reprogramming Large Language Models
  • ChatTime: A Unified Multimodal Time Series Foundation Model Bridging Numerical and Textual Data cites One Fits All: Power General Time Series Analysis by Pretrained LM
  • Large Language Models Are Zero-Shot Time Series Forecasters cites One Fits All: Power General Time Series Analysis by Pretrained LM
  • Time-LLM: Time Series Forecasting by Reprogramming Large Language Models cites Large Language Models Are Zero-Shot Time Series Forecasters
  • Time-LLM: Time Series Forecasting by Reprogramming Large Language Models cites One Fits All: Power General Time Series Analysis by Pretrained LM
  • ChatTS: Aligning Time Series with LLMs via Synthetic Data for Enhanced Understanding and Reasoning cites Visual Instruction Tuning
  • ChatTS: Aligning Time Series with LLMs via Synthetic Data for Enhanced Understanding and Reasoning cites Large Language Models Are Zero-Shot Time Series Forecasters
  • ChatTS: Aligning Time Series with LLMs via Synthetic Data for Enhanced Understanding and Reasoning cites Time-LLM: Time Series Forecasting by Reprogramming Large Language Models
  • ChatTS: Aligning Time Series with LLMs via Synthetic Data for Enhanced Understanding and Reasoning cites One Fits All: Power General Time Series Analysis by Pretrained LM
  • Are Language Models Actually Useful for Time Series Forecasting? cites Large Language Models Are Zero-Shot Time Series Forecasters
  • Are Language Models Actually Useful for Time Series Forecasting? cites Time-LLM: Time Series Forecasting by Reprogramming Large Language Models
  • Are Language Models Actually Useful for Time Series Forecasting? cites One Fits All: Power General Time Series Analysis by Pretrained LM
Builds onCited byLeft to right by year; the more cited, the higher.

Cited by top-tier papers29

  1. Are Language Models Actually Useful for Time Series Forecasting?Mingtian Tan, Mike A. Merrill, Vinayak Gupta, Tim Althoff et al.NeurIPS 2024 · 326 citations
  2. UniTS: A Unified Multi-Task Time Series ModelShanghua Gao, Teddy Koker, Owen Queen, Tom Hartvigsen et al.NeurIPS 2024 · 159 citations
  3. ChatTime: A Unified Multimodal Time Series Foundation Model Bridging Numerical and Textual DataChengsen Wang, Qi Qi, Jingyu Wang, Haifeng Sun et al.AAAI 2025 · 109 citations
  4. GPTCoach: Towards LLM-Based Physical Activity CoachingMatthew Jörke, Shardul Sapkota, Lyndsea Warkenthien, Niklas Vainio et al.CHI 2025 · 89 citations
  5. ChatTS: Aligning Time Series with LLMs via Synthetic Data for Enhanced Understanding and ReasoningZhe Xie, Zeyan Li, Xiao He, Longlong Xu et al.VLDB 2025 · 87 citations

Builds on16

  1. Visual Instruction TuningHaotian Liu, Chunyuan Li, Qingyang Wu, Yong Jae LeeNeurIPS 2023 · 11,349 citations
  2. Robust Speech Recognition via Large-Scale Weak SupervisionAlec Radford, Jong Wook Kim, Tao Xu, Greg Brockman et al.ICML 2023 · 6,966 citations
  3. One Fits All: Power General Time Series Analysis by Pretrained LMTian Zhou, Peisong Niu, Xue Wang, Liang Sun et al.NeurIPS 2023 · 1,178 citations
  4. Time-LLM: Time Series Forecasting by Reprogramming Large Language ModelsMing Jin, Shiyu Wang, Lintao Ma, Zhixuan Chu et al.ICLR 2024 · 915 citations
  5. Large Language Models Are Zero-Shot Time Series ForecastersNate Gruver, Marc Finzi, Shikai Qiu, Andrew Gordon WilsonNeurIPS 2023 · 898 citations

Related papers

  1. TimeSeriesExamAgent: Creating Time Series Reasoning Benchmarks at ScaleMalgorzata Gwiazda, Yifu Cai, Mononito Goswami, Arjun Choudhry et al.ICLR 2026 · 6 citations
  2. Time-MQA: Time Series Multi-Task Question Answering with Context EnhancementYaxuan Kong, Yiyuan Yang, Yoontae Hwang, Wenjie Du et al.ACL 2025
  3. Do Language Models Have a Common Sense regarding Time? Revisiting Temporal Commonsense Reasoning in the Era of Large Language ModelsRaghav Jain, Daivik Sojitra, Arkadeep Acharya, Sriparna Saha et al.EMNLP 2023 · 17 citations
  4. HEARTS: Benchmarking LLM Reasoning on Health Time SeriesSirui Li, Shuhan Xiao, Mihir Joshi, Ahmed Metwally et al.ICML 2026 · 7 citations
  5. TsLLM: Augmenting LLMs for General Time Series Understanding and PredictionFelix Parker, Nimeesha Chan, Chi Zhang, Kimia GhobadiICML 2026 · 3 citations
Dusk over the sea between two cliffs drawn in fine vertical lines