ACL Findings2026
EchoMLLM: Incentivizing Echocardiographic Video Understanding with Keyframe Grounding and Report Generation
Heyu Huang, Wanran Sun, Chi Chen, Bo Chen, Zonghao Guo, Yuhua Li, Ruixuan Li, Kunlun He, Maosong Sun
Abstract
Echocardiography analysis demands a dual capability: rigorous quantitative keyframe localization for evidence verification and comprehensive qualitative synthesis for diagnostic reporting. However, current Multi-Modal Large Language Models (MLLMs) struggle to meet these clinical requirements due to a misalignment with diagnostic workflows, a scarcity of video instruction data, and the critical challenge of cyclic temporal ambiguity-where the repetitive nature of cardiac cycles renders standard single-frame supervision ill-posed. To bridge this gap, we introduce EchoMLLM, a unified framework designed for real-world echocardiography video understanding. First, we align model capabilities with clinical needs by defining two fine-grained tasks: cycleand pathology-conditioned keyframe grounding and video report generation. To facilitate this, we curate EchoMM-120k, a large-scale instruction dataset specifically constructed to support temporal localization and professional reporting. Furthermore, to resolve the cyclic ambiguity, we propose a multi-stage training paradigm incorporating a novel cycle-aware Reinforcement Learning (RL) strategy. By prioritizing logical consistency over rigid index matching, our approach moves beyond rote memorization to elicit invariant reasoning. Extensive experiments demonstrate that EchoM-LLM reduces temporal grounding errors by up to 76% and improves report generation quality by 65% over its backbone, achieving state-ofthe-art performance against both generalist and medical baselines.
Ask Lune
Know this paper in minutes, not an afternoon.
Lune can read every page of it and the top-tier papers that cite it, and gives you its claims, its evidence and its limits, each one cited to the paper it comes from. No skimming, no AI slop.
Related papers
- HeartLLM: Discretized ECG Tokenization for LLM-Based Diagnostic ReasoningJinning Yang, Wenjie Sun, Wen ShiAAAI 2026
- ECG-R1: Protocol-Guided and Modality-Agnostic MLLM for Reliable ECG InterpretationJiarui Jin, Haoyu Wang, Xingliang Wu, Xiaocheng Fang et al.ICML 2026 · 11 citations
- MedReasoner: Reinforcement Learning Drives Reasoning Grounding from Clinical Thought to Pixel-Level PrecisionZhonghao Yan, Muxi Diao, Yuxuan Yang, Ruoyan Jing et al.AAAI 2026 · 4 citations
- MEIT: Multimodal Electrocardiogram Instruction Tuning on Large Language Models for Report GenerationZhongwei Wan, Che Liu, Xin Wang, Chaofan Tao et al.ACL Findings 2025 · 16 citations
- anyECG-chat: A Generalist ECG-MLLM for Flexible ECG Input and Multi-Task UnderstandingHaitao Li, Ziyu Li, Yiheng Mao, Ziyi Liu et al.AAAI 2026 · 4 citations
