Clinically-Grounded Counterfactual Reasoning for Medical Video Diagnosis
Jianzhe Gao, Churan Wang, Weiyi Zhang, Jianghua Li, Lian Li, Wenguan Wang, Yixin Zhu, Yizhou Wang
Abstract
Clinical video diagnosis, in which physicians assess dynamic tissue responses across procedural stages, is critical for detecting diseases such as cervical and colorectal cancers. Recent spatiotemporal models map visual progressions directly to diagnostic outputs, yet overlook two hallmarks of expert reasoning: clinically grounded diagnostic principles and hypothesis-driven counterfactual thinking. This gap leads existing methods to conflate causal pathological cues with non-pathological variations, thereby limiting their reliability in data-scarce settings. Here we show that explicitly modeling counterfactual tissue evolution, guided by clinical rules that encode expert diagnostic principles, substantially improves diagnostic robustness-emulating the hypothesis-driven reasoning clinicians employ in practice. We introduce MEDVCR, comprising a Counterfactual Generator (CG) that synthesizes hypothetical tissue transitions via diffusion modeling, a Counterfactual Representation Learning (CRL) module that enforces temporal consistency, pathological separability, and counterfactual alignment, and a Dual Diagnostic Prediction (DDP) strategy that combines video-level context with frame-level counterfactual contrast. On two representative tasks, MEDVCR achieves 93.0% Recall@1 on colposcopy (+10.2%) and 94.8% Average Precision (AP) on colonoscopy (+2.6%), outperforming all state-of-theart (SOTA) baselines. We anticipate that this counterfactual reasoning paradigm will open new avenues for vision-based clinical diagnostic tasks toward more transparent and interpretable decision support.
Ask about this paper
Your agent reads all of it.
Lune indexed this paper to the last equation, along with the top-tier papers that cite it. Ask a question and the answer quotes them.
Your agent calls
Luneget_paper_fulltext
Free to start. No credit card required.
Terminal
Install the CLIlune papers fulltext 38dafeca-bb46-493a-8ad2-83138eeb9401Builds on29
- Learning Transferable Visual Models From Natural Language SupervisionAlec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh et al.ICML 2021 · 47,906 citations
- SlowFast Networks for Video RecognitionChristoph Feichtenhofer, Haoqi Fan, Jitendra Malik, Kaiming HeICCV 2019 · 4,104 citations
- ViViT: A Video Vision TransformerAnurag Arnab, Mostafa Dehghani, Georg Heigold, Chen Sun et al.ICCV 2021 · 2,947 citations
- Is Space-Time Attention All You Need for Video Understanding?Gedas Bertasius, Heng Wang, Lorenzo TorresaniICML 2021 · 2,927 citations
- Video Swin TransformerZe Liu, Jia Ning, Yue Cao, Yixuan Wei et al.CVPR 2022 · 1,847 citations
Related papers
- MedEyes: Learning Dynamic Visual Focus for Medical Progressive DiagnosisChunzheng Zhu, Yangfang Lin, Shen Chen, Yijun Wang et al.AAAI 2026 · 8 citations
- Beyond Accuracy: Latent Perturbations for Cognitive-Aware DiagnosisYuting Yan, Yinghao Fu, Wendi Ren, Haozhou Gao et al.ICML 2026
- What Changed and What Could Have Changed? State-Change Counterfactuals for Procedure-Aware Video Representation LearningChi-Hsi Kung, Frangil Ramirez, Juhyung Ha, Yi-Ting Chen et al.ICCV 2025 · 3 citations
- Multi-Level Counterfactual Contrast for Visual Commonsense ReasoningXi Zhang, Feifei Zhang, Changsheng XuACM MM 2021 · 22 citations
- MedVR: Annotation-Free Medical Visual Reasoning via Agentic Reinforcement LearningZheng Jiang, Heng Guo, Chengyu Fang, Changchen Xiao et al.ICLR 2026 · 8 citations
