Lune

ACL Findings2026

EchoMLLM: Incentivizing Echocardiographic Video Understanding with Keyframe Grounding and Report Generation

Heyu Huang, Wanran Sun, Chi Chen, Bo Chen, Zonghao Guo, Yuhua Li, Ruixuan Li, Kunlun He, Maosong Sun

Abstract

Echocardiography analysis demands a dual capability: rigorous quantitative keyframe localization for evidence verification and comprehensive qualitative synthesis for diagnostic reporting. However, current Multi-Modal Large Language Models (MLLMs) struggle to meet these clinical requirements due to a misalignment with diagnostic workflows, a scarcity of video instruction data, and the critical challenge of cyclic temporal ambiguity-where the repetitive nature of cardiac cycles renders standard single-frame supervision ill-posed. To bridge this gap, we introduce EchoMLLM, a unified framework designed for real-world echocardiography video understanding. First, we align model capabilities with clinical needs by defining two fine-grained tasks: cycleand pathology-conditioned keyframe grounding and video report generation. To facilitate this, we curate EchoMM-120k, a large-scale instruction dataset specifically constructed to support temporal localization and professional reporting. Furthermore, to resolve the cyclic ambiguity, we propose a multi-stage training paradigm incorporating a novel cycle-aware Reinforcement Learning (RL) strategy. By prioritizing logical consistency over rigid index matching, our approach moves beyond rote memorization to elicit invariant reasoning. Extensive experiments demonstrate that EchoM-LLM reduces temporal grounding errors by up to 76% and improves report generation quality by 65% over its backbone, achieving state-ofthe-art performance against both generalist and medical baselines.

Ask Lune

Know this paper in minutes, not an afternoon.

Lune can read every page of it and the top-tier papers that cite it, and gives you its claims, its evidence and its limits, each one cited to the paper it comes from. No skimming, no AI slop.

Questions to start from

Lune can draw on

the full paper · related work

Get the answer

Free to start. No credit card required.

Related papers

  1. HeartLLM: Discretized ECG Tokenization for LLM-Based Diagnostic ReasoningJinning Yang, Wenjie Sun, Wen ShiAAAI 2026
  2. ECG-R1: Protocol-Guided and Modality-Agnostic MLLM for Reliable ECG InterpretationJiarui Jin, Haoyu Wang, Xingliang Wu, Xiaocheng Fang et al.ICML 2026 · 11 citations
  3. MedReasoner: Reinforcement Learning Drives Reasoning Grounding from Clinical Thought to Pixel-Level PrecisionZhonghao Yan, Muxi Diao, Yuxuan Yang, Ruoyan Jing et al.AAAI 2026 · 4 citations
  4. MEIT: Multimodal Electrocardiogram Instruction Tuning on Large Language Models for Report GenerationZhongwei Wan, Che Liu, Xin Wang, Chaofan Tao et al.ACL Findings 2025 · 16 citations
  5. anyECG-chat: A Generalist ECG-MLLM for Flexible ECG Input and Multi-Task UnderstandingHaitao Li, Ziyu Li, Yiheng Mao, Ziyi Liu et al.AAAI 2026 · 4 citations
Dusk over the sea between two cliffs drawn in fine vertical lines