Lune

CVPR Findings2026

CarePilot: A Multi-Agent Framework for Long-Horizon Computer Task Automation in Healthcare

Akash Ghosh, Tajamul Ashraf, Rishu Kumar Singh, Numan Saeed, Sriparna Saha, Xiuying Chen, Salman Khan

Abstract

Multimodal agentic pipelines are advancing human-computer interaction by enabling efficient automation of complex real-world tasks, yet current work largely focuses on short-horizon or general-purpose settings, leaving long-horizon, domain-specific automation, especially in healthcare, underexplored. To address this, we introduce CareFlow, a human-annotated benchmark of complex, long-horizon workflows across medical annotation tools, DICOM viewers, EHR systems, and laboratory information systems, where existing VLMs struggle with multi-step reasoning. We propose CarePilot, an actor-critic multi-agent framework where the Actor leverages tool grounding and dual-memory (long-and short-term) to predict actions from visual and system context, and the Critic evaluates actions, updates memory, and provides corrective feedback. Through iterative agentic simulation, the Actor learns more robust, reasoning-aware behavior. Experiments show that CarePilot achieves state-of-the-art performance, outperforming strong closed-and open-source multimodal baselines by 15.26% and 3.38% on the benchmark and OOD dataset, respectively. The code and dataset for this project are available at: Carepilot.

Ask Lune

Know this paper in minutes, not an afternoon.

Lune can read every page of it and the top-tier papers that cite it, and gives you its claims, its evidence and its limits, each one cited to the paper it comes from. No skimming, no AI slop.

2 top-tier papers have cited it since. Know what they found before your reviewers do.

Questions to start from

Lune can draw on

the full paper · the top-tier papers that cite it · related work

Get the answer

Free to start. No credit card required.

Lineage

  • When Background Matters: Breaking Medical Vision Language Models by Transferable Attack cites CURE-Med: Curriculum-Informed Reinforcement Learning for Multilingual Medical Reasoning
Builds onCited byLeft to right by year; the more cited, the higher.

Cited by top-tier papers2

  1. CURE-Med: Curriculum-Informed Reinforcement Learning for Multilingual Medical ReasoningEric Onyame, Akash Ghosh, Subhadip Baidya, Sriparna Saha et al.ACL 2026 · 7 citations
  2. When Background Matters: Breaking Medical Vision Language Models by Transferable AttackAkash Ghosh, Subhadip Baidya, Sriparna Saha, Xiuying ChenACL 2026

Builds on15

  1. Reflexion: language agents with verbal reinforcement learningNoah Shinn, Federico Cassano, Ashwin Gopinath, Karthik Narasimhan et al.NeurIPS 2023 · 5,828 citations
  2. WebArena: A Realistic Web Environment for Building Autonomous AgentsShuyan Zhou, Frank F. Xu, Hao Zhu, Xuhui Zhou et al.ICLR 2024 · 1,197 citations
  3. GPT-4V(ision) is a Generalist Web Agent, if GroundedBoyuan Zheng, Boyu Gou, Jihyung Kil, Huan Sun et al.ICML 2024 · 496 citations
  4. AppAgent: Multimodal Agents as Smartphone UsersChi Zhang, Zhao Yang, Jiaxuan Liu, Yanda Li et al.CHI 2025 · 57 citations
  5. Chiron-o1: Igniting Multimodal Large Language Models towards Generalizable Medical Reasoning via Mentor-Intern Collaborative SearchHaoran Sun, Yankai Jiang, Wenjie Lou, Yujie Zhang et al.NeurIPS 2025 · 16 citations

Related papers

  1. CARE: Towards Clinical Accountability in Multi-Modal Medical Reasoning with an Evidence-Grounded Agentic FrameworkYuexi Du, Jinglu Wang, Shujie Liu, Nicha C. Dvornek et al.ICLR 2026 · 4 citations
  2. ReflecTool: Towards Reflection-Aware Tool-Augmented Clinical AgentsYusheng Liao, Shuyang Jiang, Yanfeng Wang, Yu WangACL 2025 · 14 citations
  3. MedDCR: Learning to Design Agentic Workflows for Medical CodingJiyang Zheng, Islam Nassar, Thanh Vu, Xu Zhong et al.ACL Findings 2026 · 1 citation
  4. PASS: Probabilistic Agentic Supernet Sampling for Interpretable and Adaptive Chest X-Ray ReasoningYushi Feng, Junye Du, Yingying Hong, Qifan Wang et al.AAAI 2026 · 1 citation
  5. MMedAgent-RL: Optimizing Multi-Agent Collaboration for Multimodal Medical ReasoningPeng Xia, Jinglu Wang, Yibo Peng, Kaide Zeng et al.ICLR 2026 · 47 citations
Dusk over the sea between two cliffs drawn in fine vertical lines