Lune

CVPR Findings2026

ReCliFF: Adaptive Orthogonal Decoupling for Federated Fine-tuning of Medical MLLMs

Yuncheng Jiang, Chun-Mei Feng, Rui Sun, Le Zhang

Abstract

Federated fine-tuning of medical multi-modal large language models (MLLMs) holds immense potential for leveraging decentralized healthcare data while preserving patient privacy. However, the efficacy of federated learning in this domain is severely undermined by client drift, a phenomenon that arises from the complex multi-dimensional data heterogeneity in real-world medical data. Existing approaches designed to mitigate this drift are often static and unsuitable for the unique training dynamics of parameterefficient adapters. To address this, we introduce ReCliFF, a novel framework that actively Reverses Client drift in Federated Fine-tuning. It actively decouples shared knowledge from client-specific biases by decomposing the Low-Rank Adaptation (LoRA) adapter into common and specific components. A dynamic controller modulates the decoupling regularization strength by monitoring the conflict between local and global update directions. This design enables the global model to aggregate generalizable information effectively. Extensive experiments on four challenging benchmarks simulating institutional, modality, organ, and task heterogeneity demonstrate that ReCliFF achieves substantial improvements over twelve state-of-the-art baselines. It consistently achieves superior accuracy and faster convergence, without introducing additional uplink communication overhead. These attributes make ReCliFF a robust and practical solution for deploying federated MLLMs in real-world medical settings.

Ask Lune

Know this paper in minutes, not an afternoon.

Lune can read every page of it and the top-tier papers that cite it, and gives you its claims, its evidence and its limits, each one cited to the paper it comes from. No skimming, no AI slop.

Questions to start from

Lune can draw on

the full paper · related work

Get the answer

Free to start. No credit card required.

Lineage

  • FedBN: Federated Learning on Non-IID Features via Local Batch Normalization cites Adaptive Federated Optimization
Builds onCited byLeft to right by year; the more cited, the higher.

Builds on18

  1. LoRA: Low-Rank Adaptation of Large Language ModelsEdward J. Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu et al.ICLR 2022 · 18,833 citations
  2. Visual Instruction TuningHaotian Liu, Chunyuan Li, Qingyang Wu, Yong Jae LeeNeurIPS 2023 · 11,349 citations
  3. SCAFFOLD: Stochastic Controlled Averaging for Federated LearningSai Praneeth Karimireddy, Satyen Kale, Mehryar Mohri, Sashank J. Reddi et al.ICML 2020 · 3,875 citations
  4. Adaptive Federated OptimizationSashank J. Reddi, Zachary Charles, Manzil Zaheer, Zachary Garrett et al.ICLR 2021 · 1,917 citations
  5. FedBN: Federated Learning on Non-IID Features via Local Batch NormalizationXiaoxiao Li, Meirui Jiang, Xiaofei Zhang, Michael Kamp et al.ICLR 2021 · 1,166 citations

Related papers

  1. Federated Residual Low-Rank Adaptation of Large Language ModelsYunlu Yan, Chun-Mei Feng, Wangmeng Zuo, Rick Siow Mong Goh et al.ICLR 2025
  2. FedSRD: Sparsify-Reconstruct-Decompose for Communication-Efficient Federated Large Language Models Fine-TuningGuochen Yan, Luyuan Xie, Qingni Shen, Yuejian Fang et al.WWW 2026 · 1 citation
  3. Personalized Federated Fine-Tuning for LLMs via Data-Driven Heterogeneous Model ArchitecturesYicheng Zhang, Zhen Qin, Zhaomin Wu, Jian Hou et al.WWW 2026 · 9 citations
  4. Adaptive LoRA Experts Allocation and Selection for Federated Fine-TuningLei Wang, Jieming Bian, Letian Zhang, Jie XuNeurIPS 2025 · 14 citations
  5. UniFLoW: Universal Multi-Modal Federated LoRA Fine-Tuning Framework with Analytical Aggregationhaoyuan liang, Zhiyu Ye, Jielong Tang, Yang Yang et al.ICML 2026
Dusk over the sea between two cliffs drawn in fine vertical lines