Lune

CVPR Findings2026

FinChart-Multimodal: A Dataset for Context-Injected Financial Chart Understanding with Aligned OHLCV Time Series

Devansh Garg

Abstract

Understanding financial charts requires integrating visual patterns with numerical time series, but existing datasets lack pixel-level alignment between chart images and underlying data. We introduce FinChart-Multimodal, a dataset of 100,000 financial charts with pixel-aligned OHLCV time series data. The key innovation is deterministic generation providing exact pixel-to-timestamp and pixel-to-price mappings without OCR. Charts are generated from real market data using a reproducible pipeline with extensive visual diversity (23+ color schemes, 30+ backgrounds, 16 grid styles, 15-20 font families) while maintaining alignment guarantees. The dataset spans 105 tickers across 10 sectors, 5 chart types, and 8 time periods (1mo to 5y), with content-based uniqueness checking incorporating visual parameters ensuring < 0.001% collision probability. We propose three benchmark tasks: temporal-visual alignment learning, context injection effectiveness, and crossmodal retrieval. The dataset, generation pipeline, and benchmark task definitions will be publicly released.

Ask Lune

Ask anything. The answer quotes the paper.

Lune reads all of it and the papers around it, then shows you the passages behind every answer.

Questions to start from

Lune answers from

the full paper · related work

Get the answer

Free to start. No credit card required.

Builds on7

  1. Learning Transferable Visual Models From Natural Language SupervisionAlec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh et al.ICML 2021 · 47,906 citations
  2. A Simple Framework for Contrastive Learning of Visual RepresentationsTing Chen, Simon Kornblith, Mohammad Norouzi, Geoffrey E. HintonICML 2020 · 24,064 citations
  3. Visual Instruction TuningHaotian Liu, Chunyuan Li, Qingyang Wu, Yong Jae LeeNeurIPS 2023 · 11,349 citations
  4. Pix2Struct: Screenshot Parsing as Pretraining for Visual Language UnderstandingKenton Lee, Mandar Joshi, Iulia Raluca Turc, Hexiang Hu et al.ICML 2023 · 426 citations
  5. DePlot: One-shot visual language reasoning by plot-to-table translationFangyu Liu, Julian Martin Eisenschlos, Francesco Piccinno, Syrine Krichene et al.ACL Findings 2023 · 72 citations

Related papers

  1. FinChart-Bench: Benchmarking Financial Chart Comprehension in Vision-Language ModelsDong Shu, Haoyang Yuan, Yuchen Wang, Yanguang Liu et al.ACL 2026 · 11 citations
  2. ChartNet: A Million-Scale, High-Quality Multimodal Dataset for Robust Chart UnderstandingJovana Kondic, Pengyuan Li, Dhiraj Joshi, Isaac Sanchez et al.CVPR 2026 · 7 citations
  3. FinMME: Benchmark Dataset for Financial Multi-Modal Reasoning EvaluationJunyu Luo, Zhizhuo Kou, Liming Yang, Xiao Luo et al.ACL 2025
  4. FinRAGBench-V: A Benchmark for Multimodal RAG with Visual Citation in the Financial DomainSuifeng Zhao, Zhuoran Jin, Sujian Li, Jun GaoEMNLP 2025 · 1 citation
  5. NovaChart: A Large-scale Dataset towards Chart Understanding and Generation of Multimodal Large Language ModelsLinmei Hu, Duokang Wang, Yiming Pan, Jifan Yu et al.ACM MM 2024 · 5 citations
Dusk over the sea between two cliffs drawn in fine vertical lines