FinMRAGBench: A Realistic and Complex Benchmark for Multi-Modal RAG in Financial Document Analysis

Retrieval-augmented generation (RAG) has become a widely adopted paradigm for realistic financial analysis over financial documents. However, existing benchmarks fail to capture realistic financial analysis settings that involve cross-document retrieval, multi-page evidence integration, and diverse analytical tasks. To address this gap, we introduce FinMRAG-Bench, a comprehensive multi-modal financial RAG benchmark in which most questions require retrieving evidence scattered across multiple pages and documents, constructed from large-scale real-world annual reports and comprising 887 expert-verified QA pairs spanning five representative financial analysis tasks. Moreover, we introduce FinMRAGAgent, an agent trained on high-quality agentic trajectories following the reasoning-andacting (ReAct) paradigm, capable of dynamic tool invocation and multi-step financial analysis. Our extensive experiments show that current multi-modal RAG systems still struggle with incomplete retrieval and complex financial reasoning. In contrast, FinMRAGAgent achieves the strongest overall performance across all models, demonstrating that our structured reasoning approach significantly enhances multi-modal RAG in realistic financial scenarios. The code and data are available at https://github.com/sqyangit/FinMRAGBench . Question: (Knowledge-Intensive Reasoning) Analyze McDonald's capital expenditure efficiency in 2024, focusing on total CapEx, ... Corpus: Multiple Financial Documents In 2024, the Company opened 2,116 restaurants and closed 461 restaurants. In 2023, the Company opened 2,067 restaurants and closed 520 restaurants. The increase in openings in 2024 is a result of ...