Lune

CVPR Findings2026

R²MoE: Representation and Expert Selection Dual-Regularized Mixture-of-Experts for Multimodal Clinical Data

Wajih Hassan Raza, Mya Schiess, Juan Martinez Lemus, Timothy Michael Ellmore, Charles Green, Claudio Soto, Xin Fu, Renjie Hu

Abstract

Accurate prediction of complex diseases and patient outcomes depends on effectively combining heterogeneous clinical modalities-such as neuroimaging, genomics, and free-text notes. Although multimodal fusion methods, especially Mixture-of-Experts (MoE) architectures have shown strong performance on large clinical cohorts, naively applying standard MoE models to clinical multimodal inputs exposes two fundamental limitations. First, the strong variability in different modalities introduce highly non-uniform input patterns, which make it difficult for the gating network to learn stable and reliable expert assignments. Second, the resulting latent embeddings often lack coherent semantic structure and adequate inter-class separation, hindering robust clinical prediction. We propose R 2 MoE, a dual-regularized MoE framework that addresses these challenges. A specialized expert regularization anchors expert behavior to modality-driven patterns, preventing routing collapse and promoting stable expert coalitions. A complementary representation regularization shapes the fused latent embedding into a semantically structured and classdiscriminative space. Furthermore, we aggregate layerwise routing weights into a gating-weight fingerprint, capturing patient-specific computational pathways that reveal clinically meaningful subtype structure and enhance downstream prediction. Across ADNI and MIMIC-IV, R 2 MoE outperforms the best existing multimodal models, achieving relative improvements of 6.1% and 3.3% in classification accuracy, respectively, while also yielding more interpretable structure in the learned representation space.

Ask Lune

Know this paper in minutes, not an afternoon.

Lune can read every page of it and the top-tier papers that cite it, and gives you its claims, its evidence and its limits, each one cited to the paper it comes from. No skimming, no AI slop.

Questions to start from

Lune can draw on

the full paper · related work

Get the answer

Free to start. No credit card required.

Lineage

  • Multimodal Contrastive Learning with LIMoE: the Language-Image Mixture of Experts cites VATT: Transformers for Multimodal Self-Supervised Learning from Raw Video, Audio and Text
  • Multimodal Contrastive Learning with LIMoE: the Language-Image Mixture of Experts cites Scaling Vision with Sparse Mixture of Experts
  • VATT: Transformers for Multimodal Self-Supervised Learning from Raw Video, Audio and Text cites Scaling Vision with Sparse Mixture of Experts
  • FuseMoE: Mixture-of-Experts Transformers for Fleximodal Fusion cites Multimodal Contrastive Learning with LIMoE: the Language-Image Mixture of Experts
  • FuseMoE: Mixture-of-Experts Transformers for Fleximodal Fusion cites Scaling Vision with Sparse Mixture of Experts
  • FuseMoE: Mixture-of-Experts Transformers for Fleximodal Fusion cites Integrating Multimodal Information in Large Pretrained Transformers
Builds onCited byLeft to right by year; the more cited, the higher.

Builds on13

  1. Scaling Vision with Sparse Mixture of ExpertsCarlos Riquelme, Joan Puigcerver, Basil Mustafa, Maxim Neumann et al.NeurIPS 2021 · 1,213 citations
  2. VATT: Transformers for Multimodal Self-Supervised Learning from Raw Video, Audio and TextHassan Akbari, Liangzhe Yuan, Rui Qian, Wei-Hong Chuang et al.NeurIPS 2021 · 782 citations
  3. Integrating Multimodal Information in Large Pretrained TransformersWasifur Rahman, Md. Kamrul Hasan, Sangwu Lee, AmirAli Bagher Zadeh et al.ACL 2020 · 584 citations
  4. Multimodal Contrastive Learning with LIMoE: the Language-Image Mixture of ExpertsBasil Mustafa, Carlos Riquelme, Joan Puigcerver, Rodolphe Jenatton et al.NeurIPS 2022 · 359 citations
  5. FuseMoE: Mixture-of-Experts Transformers for Fleximodal FusionXing Han, Huy Nguyen, Carl Harris, Nhat Ho et al.NeurIPS 2024 · 129 citations

Related papers

  1. Flex-MoE: Modeling Arbitrary Modality Combination via the Flexible Mixture-of-ExpertsSukwon Yun, Inyoung Choi, Jie Peng, Yangfan Wu et al.NeurIPS 2024 · 98 citations
  2. Soft Modality-Guided Expert Specialization in MoE-VLMsZi-Hao Bo, Yaqian Li, Anzhou Hou, Rinyoichi Takezoe et al.CVPR 2026
  3. I2MoE: Interpretable Multimodal Interaction-aware Mixture-of-ExpertsJiayi Xin, Sukwon Yun, Jie Peng, Inyoung Choi et al.ICML 2025
  4. MeMix: Multi-Encoder Mixture Framework for Medical Report GenerationYiming Cao, Lizhen Cui, Zhiqi ShenCVPR Findings 2026
  5. Rethinking Gating Mechanism in Sparse MoE: Handling Arbitrary Modality Inputs with Confidence-Guided GateLiangwei Zheng, Wei Emma Zhang, Mingyu Guo, Olaf Maennel et al.ICML 2026 · 7 citations
Dusk over the sea between two cliffs drawn in fine vertical lines