OMHBench: Benchmarking Balanced and Grounded Omni-Modal Multi-Hop Reasoning

Multimodal Large Language Models (MLLMs) have increasingly supported omni-modal processing across text, vision, and speech.However, existing evaluation frameworks for such models suffer from critical limitations, including modality shortcuts and biased reasoning paths.To address these challenges, we propose OMHBench, a novel benchmark designed to rigorously evaluate omni-modal multi-hop reasoning.It consists of 6,144 questions with balanced reasoning paths that are jointly grounded across all three modalities.Extensive evaluation of 13 state-of-the-art models reveals that (1) a large performance gap exists between proprietary and open-source MLLMs and (2) even proprietary models exhibit high sensitivity to reasoning path variations, resulting in asymmetric omni-modal grounding.Notably, models struggle when processing the speech modality, underscoring the need for balanced, multihop evaluation of omni-modal intelligence.* This work was conducted at Sony AI.† Corresponding author 1 Omni-modal refers to the text-vision-audio setting, while multi-modal is a general term for more than one modality.