Lune

CVPR Findings2026

MARS-RL: Enhancing Multi-Agent RAG Systems for Multi-Modal Documents via Strategic Reasoning with Reinforcement Learning

Zhongyu Wang, Pengbo Liu

Abstract

Although Retrieval-Augmented Generation (RAG) enhances Large Language Models (LLMs) by incorporating external knowledge, conventional single-agent RAG frameworks remain limited in handling tasks that demand complex multi-turn reasoning. Multi-agent systems enhance multi-modal retrieval through collaboration, but their reliance on predefined communication workflows often results in inefficient retrieval and limited strategic reasoning capability. To address these limitations, we propose MARS-RL, a Multi-Agent RAG System that integrates a strategic reasoning model capable of generating real-time policies in an open action space into interactive agents to guide their collaborative workflow during generation. To further improve adaptability and policy transferability, we train the strategic reasoning model with multi-turn reinforcement learning (RL), incorporating process rewards and iterative self-play. Extensive experiments on diverse and challenging benchmarks demonstrate the effectiveness of our approach and establishing a new state-of-the-art on the ViDoSeek benchmark.

Ask Lune

Know this paper in minutes, not an afternoon.

Lune can read every page of it and the top-tier papers that cite it, and gives you its claims, its evidence and its limits, each one cited to the paper it comes from. No skimming, no AI slop.

Questions to start from

Lune can draw on

the full paper · related work

Get the answer

Free to start. No credit card required.

Lineage

Builds onCited byLeft to right by year; the more cited, the higher.

Builds on32

  1. Sigmoid Loss for Language Image Pre-TrainingXiaohua Zhai, Basil Mustafa, Alexander Kolesnikov, Lucas BeyerICCV 2023 · 2,932 citations
  2. WebShop: Towards Scalable Real-World Web Interaction with Grounded Language AgentsShunyu Yao, Howard Chen, John Yang, Karthik NarasimhanNeurIPS 2022 · 1,477 citations
  3. ALFWorld: Aligning Text and Embodied Environments for Interactive LearningMohit Shridhar, Xingdi Yuan, Marc-Alexandre Côté, Yonatan Bisk et al.ICLR 2021 · 819 citations
  4. Visual-RFT: Visual Reinforcement Fine-TuningZiyu Liu, Zeyi Sun, Yuhang Zang, Xiaoyi Dong et al.ICCV 2025 · 563 citations
  5. SWE-RL: Advancing LLM Reasoning via Reinforcement Learning on Open Software EvolutionYuxiang Wei, Olivier Duchenne, Jade Copet, Quentin Carbonneaux et al.NeurIPS 2025 · 291 citations

Related papers

  1. MDocRAG-RL: Empowering Multi-Modal Document RAG via Complex Visual Reasoning with Reinforcement LearningZhongyu WangACL Findings 2026
  2. ViDoRAG: Visual Document Retrieval-Augmented Generation via Dynamic Iterative Reasoning AgentsQiuchen Wang, Ruixue Ding, Zehui Chen, Weiqi Wu et al.EMNLP 2025 · 6 citations
  3. R3-RAG: Learning Step-by-Step Reasoning and Retrieval for LLMs via Reinforcement LearningYuan Li, Qi Luo, Xiaonan Li, Bufan Li et al.EMNLP Findings 2025 · 4 citations
  4. Towards Open-World Retrieval-Augmented Generation on Knowledge Graph: A Multi-Agent Collaboration FrameworkJiasheng Xu, Mingda Li, Yongqiang Tang, Peijie Wang et al.WWW 2026
  5. InstructRAG: Leveraging Retrieval-Augmented Generation on Instruction Graphs for LLM-Based Task PlanningZheng Wang, Shu Xian Teo, Jun Jie Chew, Wei ShiSIGIR 2025 · 4 citations
Dusk over the sea between two cliffs drawn in fine vertical lines