Lune

EMNLP Findings2024

Financial Forecasting from Textual and Tabular Time Series

Ross Koval, Nicholas Andrews, Xifeng Yan

Abstract

There is a variety of multimodal data pertinent to public companies, spanning from accounting statements, macroeconomic statistics, earnings conference calls, and financial reports. These diverse modalities capture the state of firms from a variety of different perspectives but requires complex interactions to reconcile in the formation of accurate financial predictions. The commonality between these different modalities is that they all represent a time series, typically observed for a particular firm at a quarterly horizon, providing the ability to model trends and variations of company data over time. However, the time series of these diverse modalities contains varying temporal and cross-channel patterns that are challenging to model without the appropriate inductive biases. In this work, we design a novel multimodal time series prediction task that includes numerical financial results, macroeconomic states, and long financial documents to predict next quarter's company earnings relative to analyst expectations. We explore a variety of approaches for this novel setting, establish strong unimodal baselines, and propose a multimodal model that exhibits state-of-the-art performance on this unique task. We demonstrate that each modality contains unique information and that the best performing model requires careful fusion of the different modalities in a multi-stage training approach. To better understand model behavior, we conduct a variety of probing experiments, reveal insights into the value of different modalities, and demonstrate the practical utility of our proposed method in a simulated trading setting.

Ask Lune

Know this paper in minutes, not an afternoon.

Lune can read every page of it and the top-tier papers that cite it, and gives you its claims, its evidence and its limits, each one cited to the paper it comes from. No skimming, no AI slop.

2 top-tier papers have cited it since. Know what they found before your reviewers do.

Questions to start from

Lune can draw on

the full paper · the top-tier papers that cite it · related work

Get the answer

Free to start. No credit card required.

Lineage

  • LETS-C: Leveraging Text Embedding for Time Series Classification cites A Time Series is Worth 64 Words: Long-term Forecasting with Transformers
Builds onCited byLeft to right by year; the more cited, the higher.

Cited by top-tier papers2

  1. FinCall-Surprise: A Large Scale Multi-modal Benchmark for Earning Surprise PredictionDong Shu, Yanguang Liu, Huopu Zhang, Mengnan DuACL 2026 · 1 citation
  2. LETS-C: Leveraging Text Embedding for Time Series ClassificationRachneet Kaur, Zhen Zeng, Tucker Balch, Manuela VelosoACL 2025

Builds on11

  1. Big Bird: Transformers for Longer SequencesManzil Zaheer, Guru Guruganesh, Kumar Avinava Dubey, Joshua Ainslie et al.NeurIPS 2020 · 3,159 citations
  2. SimCSE: Simple Contrastive Learning of Sentence EmbeddingsTianyu Gao, Xingcheng Yao, Danqi ChenEMNLP 2021 · 2,496 citations
  3. A Time Series is Worth 64 Words: Long-term Forecasting with TransformersYuqi Nie, Nam H. Nguyen, Phanwadee Sinthong, Jayant KalagnanamICLR 2023 · 536 citations
  4. Don't Stop Pretraining: Adapt Language Models to Domains and TasksSuchin Gururangan, Ana Marasovic, Swabha Swayamdipta, Kyle Lo et al.ACL 2020 · 93 citations
  5. Domain Adaptive Multi-Modality Neural Attention Network for Financial ForecastingDawei Zhou, Lecheng Zheng, Yada Zhu, Jianbo Li et al.WWW 2020 · 51 citations

Related papers

  1. Forecasting Earnings Surprises from Conference Call TranscriptsRoss Koval, Nicholas Andrews, Xifeng YanACL Findings 2023 · 10 citations
  2. NumHTML: Numeric-Oriented Hierarchical Transformer Model for Multi-Task Financial ForecastingLinyi Yang, Jiazheng Li, Ruihai Dong, Yue Zhang et al.AAAI 2022 · 54 citations
  3. HTML: Hierarchical Transformer-based Multi-task Learning for Volatility PredictionLinyi Yang, Tin Lok James Ng, Barry Smyth, Ruihai DongWWW 2020 · 127 citations
  4. Guided Attention Multimodal Multitask Financial Forecasting with Inter-Company Relationships and Global and Local NewsGary Ang, Ee-Peng LimACL 2022
  5. Multimodal Multi-Task Financial Risk ForecastingRamit Sawhney, Puneet Mathur, Ayush Mangal, Piyush Khanna et al.ACM MM 2020 · 61 citations
Dusk over the sea between two cliffs drawn in fine vertical lines