Lune

ACL Findings2026

Are Large Language Models Reliable Reviewers? A Benchmark for Error Detection in Financial Documents

Ying He, Zhouhong Gu, Zhecheng Hu, Yubo Zhou, Hao Shen, Jiaqing Liang, Zhaoqian Dai, Ma Shuguang, Fei Yu, Yanghua Xiao, Zhixu Li

Abstract

Ensuring the accuracy of financial documents is critical for economic analysis, regulatory compliance, and corporate decision-making. Several studies have shown that Large Language Models (LLMs) perform well in many financial tasks, such as stock price movements and financial analytics. However, a critical task remains unexplored: the ability of LLMs to identify errors in financial documents. In this paper, we introduce FinED-Bench, the first publicly Benchmark for Financial Error Detection across three levels of cognitive complexity. FinED-Bench covers nine real-world financial scenarios, and includes over 900 documents reported in 2025 that are unseen by existing language models. We detail the benchmark construction process and evaluate several advanced LLMs (e.g., GPT-4o, Qwen3-14B) on this tasks, which requires both financial domain knowledge and reasoning capabilities. Experimental results show that current LLMs still struggle with this task, especially in high-complexity cases. Besides, supervised fine-tuning can significantly improve the performance of weaker LLMs on this task. Our data and code are available at https://anonymous.4open.science/r/FinED-Bench-406F.

Ask Lune

Ask anything. The answer quotes the paper.

Lune reads all of it and the papers around it, then shows you the passages behind every answer.

Questions to start from

Lune answers from

the full paper · related work

Get the answer

Free to start. No credit card required.

Builds on10

  1. Using an LLM to Help With Code UnderstandingDaye Nam, Andrew Macvean, Vincent J. Hellendoorn, Bogdan Vasilescu et al.ICSE 2024 · 264 citations
  2. When FLUE Meets FLANG: Benchmarks and Large Pretrained Language Model for Financial DomainRaj Sanjay Shah, Kunal Chawla, Dheeraj Eidnani, Agam Shah et al.EMNLP 2022 · 63 citations
  3. MEDEC: A Benchmark for Medical Error Detection and Correction in Clinical NotesAsma Ben Abacha, Wen-Wai Yim, Yujuan Fu, Zhaoyi Sun et al.ACL Findings 2025 · 29 citations
  4. Trillion Dollar Words: A New Financial Dataset, Task & Market AnalysisAgam Shah, Suvan Paturi, Sudheer ChavaACL 2023 · 27 citations
  5. BizBench: A Quantitative Reasoning Benchmark for Business and FinanceMichael Krumdick, Rik Koncel-Kedziorski, Viet Dac Lai, Varshini Reddy et al.ACL 2024 · 10 citations

Related papers

  1. EDINET-Bench: Evaluating LLMs on Complex Financial Tasks using Japanese Financial StatementsIssa Sugiura, Takashi Ishida, Taro Makino, Chieko Tazuke et al.ICLR 2026 · 9 citations
  2. XFinBench: Benchmarking LLMs in Complex Financial Problem Solving and ReasoningZhihan Zhang, Yixin Cao, Lizi LiaoACL Findings 2025 · 4 citations
  3. FinMaster: A Holistic Benchmark for Full-Pipeline Financial Management with Large Language ModelsJunzhe Jiang, Chang Yang, Aixin Cui, Sihan Jin et al.ACL Findings 2026
  4. MultiFinBen: Benchmarking Large Language Models for Multilingual and Multimodal Financial ApplicationXueqing Peng, Lingfei Qian, Yan Wang, Ruoyu Xiang et al.ACL 2026 · 6 citations
  5. FinChart-Bench: Benchmarking Financial Chart Comprehension in Vision-Language ModelsDong Shu, Haoyang Yuan, Yuchen Wang, Yanguang Liu et al.ACL 2026 · 11 citations
Dusk over the sea between two cliffs drawn in fine vertical lines