MLLM-HWSI: A Multimodal Large Language Model for Hierarchical Whole Slide Image Understanding

Whole Slide Images (WSIs) exhibit hierarchical structure, where diagnostic cues arise from cellular morphology, regional tissue organization, and global context. Existing Computational Pathology (CPath) Multimodal Large Language Models (MLLMs) typically compress an entire WSI into a single embedding, which hinders fine-grained grounding and ignores how pathologists synthesize evidence across different scales. We introduce MLLM-HWSI, a Hierarchical WSI-level MLLM that aligns visual features with pathology language at four distinct scales—cell as word, patch as phrase, region as sentence, and WSI as paragraph—to support interpretable, evidence-grounded reasoning. MLLM-HWSI decomposes each WSI into multi-scale embeddings with scale-specific V$\rightarrow$L projectors and jointly enforces (i) a hierarchical contrastive objective and (ii) a cross-scale consistency loss, preserving semantic coherence from cells to the WSI. To make gigapixel processing tractable and clinically meaningful, we compute diagnostically relevant tokens and aggregate segmented cell embeddings into a compact cellular token per-patch using a lightweight Cell–Cell Attention Fusion (CCAF) transformer. The projected multi-scale tokens are fused with text tokens and fed to an instruction-tuned LLM for open-ended reasoning, VQA, report, and caption generation tasks. Trained in three stages, MLLM-HWSI achieves new SOTA results on 13 WSI-level benchmarks across six CPath tasks. By grounding language in calibrated, multi-scale visual evidence, HMLLM provides accurate, interpretable outputs that mirror expert diagnostic workflows and advance holistic WSI understanding. Code will be released upon the publication.