EMNLP Findings2025
PathoHR: Hierarchical Reasoning for Vision-Language Models in Pathology
Yating Huang, Ziyan Huang, Lintao Xiang, Qijun Yang, Hujun Yin
Abstract
Accurate analysis of pathological images is essential for automated tumor diagnosis but remains challenging due to high structural similarity and subtle morphological variations in tissue images. Current vision-language (VL) models often struggle to capture the complex reasoning required for interpreting structured pathological reports. To address these limitations, we propose PathoHR-Bench, a novel benchmark designed to evaluate VL models' abilities in hierarchical semantic understanding and compositional reasoning within the pathology domain. Results of this benchmark reveal that existing VL models fail to effectively model intricate cross-modal relationships, hence limiting their applicability in clinical setting. To overcome this, we further introduce a pathology-specific VL training scheme that generates enhanced and perturbed samples for multimodal contrastive learning. Experimental evaluations demonstrate that our approach achieves state-of-the-art performance on PathoHR-Bench and six additional pathology datasets, highlighting its effectiveness in fine-grained pathology representation.
Ask Lune
Know this paper in minutes, not an afternoon.
Lune can read every page of it and the top-tier papers that cite it, and gives you its claims, its evidence and its limits, each one cited to the paper it comes from. No skimming, no AI slop.
Lineage
- BLIP: Bootstrapping Language-Image Pre-training for Unified Vision-Language Understanding and Generation cites Learning Transferable Visual Models From Natural Language Supervision
- CyCLIP: Cyclic Contrastive Language-Image Pretraining cites Learning Transferable Visual Models From Natural Language Supervision
- When and Why Vision-Language Models Behave like Bags-Of-Words, and What to Do About It? cites BLIP: Bootstrapping Language-Image Pre-training for Unified Vision-Language Understanding and Generation
- When and Why Vision-Language Models Behave like Bags-Of-Words, and What to Do About It? cites Learning Transferable Visual Models From Natural Language Supervision
Builds on13
- Learning Transferable Visual Models From Natural Language SupervisionAlec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh et al.ICML 2021 · 47,906 citations
- High-Resolution Image Synthesis with Latent Diffusion ModelsRobin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser et al.CVPR 2022 · 13,123 citations
- BLIP: Bootstrapping Language-Image Pre-training for Unified Vision-Language Understanding and GenerationJunnan Li, Dongxu Li, Caiming Xiong, Steven C. H. HoiICML 2022 · 6,549 citations
- CyCLIP: Cyclic Contrastive Language-Image PretrainingShashank Goel, Hritik Bansal, Sumit Bhatia, Ryan A. Rossi et al.NeurIPS 2022 · 192 citations
- When and Why Vision-Language Models Behave like Bags-Of-Words, and What to Do About It?Mert Yüksekgönül, Federico Bianchi, Pratyusha Kalluri, Dan Jurafsky et al.ICLR 2023 · 37 citations
Related papers
- Beyond Single View: A Comprehensive Benchmark for Medical Multimodal Large Language Models on Multi-Image UnderstandingDexuan Xu, Jiayin Yuan, Jianing Wang, Yanyuan Chen et al.ACL 2026
- WSI-LLaVA: A Multimodal Large Language Model for Whole Slide ImageYuci Liang, Xinheng Lyu, Wenting Chen, Meidan Ding et al.ICCV 2025 · 10 citations
- Logic Unseen: Revealing the Logical Blindspots of Vision-Language ModelsYuchen Zhou, Jiayu Tang, Shuo Yang, Xiaoyan Xiao et al.AAAI 2026 · 2 citations
- Patho-R1: A Multimodal Reinforcement Learning-Based Pathology Expert ReasonerWenchuan Zhang, Penghao Zhang, Jingru Guo, Tao Cheng et al.AAAI 2026 · 17 citations
- Transitive Representation Learning Enhances Histopathology AnnotationMoritz Schaefer, Zoe Piran, Nils Philipp Walter, Animesh Awasthi et al.ICML 2026
