EMNLP Findings2025
Agent Trading Arena: A Study on Numerical Understanding in LLM-Based Agents
Tianmi Ma, Jiawei Du, Wenxin Huang, Wenjie Wang, Liang Xie, Xian Zhong, Joey Tianyi Zhou
Abstract
Large language models (LLMs) have demonstrated remarkable capabilities in natural language tasks, yet their performance in dynamic, real-world financial environments remains underexplored. Existing approaches are limited to historical backtesting, where trading actions cannot influence market prices and agents train only on static data. To address this limitation, we present the Agent Trading Arena, a virtual zero-sum stock market in which LLM-based agents engage in competitive multi-agent trading and directly impact price dynamics. By simulating realistic bid-ask interactions, our platform enables training in scenarios that closely mirror live markets, thereby narrowing the gap between training and evaluation. Experiments reveal that LLMs struggle with numerical reasoning when given plain-text data, often overfitting to local patterns and recent values. In contrast, chart-based visualizations significantly enhance both numerical reasoning and trading performance. Furthermore, incorporating a reflection module yields additional improvements, especially with visual inputs. Evaluations on NASDAQ and CSI datasets demonstrate the superiority of our method, particularly under high volatility. All code and data are available at https://github.com/ wekjsdvnm/Agent-Trading-Arena.
Ask Lune
Know this paper in minutes, not an afternoon.
Lune can read every page of it and the top-tier papers that cite it, and gives you its claims, its evidence and its limits, each one cited to the paper it comes from. No skimming, no AI slop.
1 top-tier paper has cited it since. Know what they found before your reviewers do.
Lune can draw on
the full paper · the top-tier papers that cite it · related work
Free to start. No credit card required.
Lineage
- Evolving Quantitative Reasoning through Self-Play in Digital Twin Markets cites Generative Agents: Interactive Simulacra of Human Behavior
- Evolving Quantitative Reasoning through Self-Play in Digital Twin Markets cites TimesNet: Temporal 2D-Variation Modeling for General Time Series Analysis
- Reflexion: language agents with verbal reinforcement learning cites Generative Agents: Interactive Simulacra of Human Behavior
- AgentVerse: Facilitating Multi-Agent Collaboration and Exploring Emergent Behaviors cites Generative Agents: Interactive Simulacra of Human Behavior
- AgentVerse: Facilitating Multi-Agent Collaboration and Exploring Emergent Behaviors cites Reflexion: language agents with verbal reinforcement learning
Cited by top-tier papers1
Builds on18
- Reflexion: language agents with verbal reinforcement learningNoah Shinn, Federico Cassano, Ashwin Gopinath, Karthik Narasimhan et al.NeurIPS 2023 · 5,828 citations
- Generative Agents: Interactive Simulacra of Human BehaviorJoon Sung Park, Joseph C. O'Brien, Carrie Jun Cai, Meredith Ringel Morris et al.UIST 2023 · 1,882 citations
- AgentVerse: Facilitating Multi-Agent Collaboration and Exploring Emergent BehaviorsWeize Chen, Yusheng Su, Jingwei Zuo, Cheng Yang et al.ICLR 2024 · 594 citations
- PDFormer: Propagation Delay-Aware Dynamic Long-Range Transformer for Traffic Flow PredictionJiawei Jiang, Chengkai Han, Wayne Xin Zhao, Jingyuan WangAAAI 2023 · 542 citations
- TimesNet: Temporal 2D-Variation Modeling for General Time Series AnalysisHaixu Wu, Tengge Hu, Yong Liu, Hang Zhou et al.ICLR 2023 · 423 citations
Related papers
- AlphaQuanter: An End-to-End Tool-Augmented Agentic Reinforcement Learning Framework for Stock TradingZheye Deng, Weixiang Yan, Changlong Yu, Jiashu WangACL Findings 2026 · 1 citation
- VisualWebArena: Evaluating Multimodal Agents on Realistic Visual Web TasksJing Yu Koh, Robert Lo, Lawrence Jang, Vikram Duvvur et al.ACL 2024 · 25 citations
- FinCon: A Synthesized LLM Multi-Agent System with Conceptual Verbal Reinforcement for Enhanced Financial Decision MakingYangyang Yu, Zhiyuan Yao, Haohang Li, Zhiyang Deng et al.NeurIPS 2024 · 197 citations
- INVESTORBENCH: A Benchmark for Financial Decision-Making Tasks with LLM-based AgentHaohang Li, Yupeng Cao, Yangyang Yu, Shashidhar Reddy Javaji et al.ACL 2025
- Market-Bench: Benchmarking Large Language Models on Economic and Trade CompetitionYushuo Zheng, Huiyu Duan, Zicheng Zhang, Yucheng Zhu et al.ACL 2026 · 1 citation
