Weaving in the Clouds: Achieving Synergistic Collaboration among LLM Agents via Federated Learning

Multi-Agent Systems (MAS) powered by Large Language Models (LLMs) have become a promising paradigm for solving workflow-structured tasks through expert collaboration. However, the data required for such collaboration are often distributed across organizations and cannot be centrally pooled due to privacy, intellectual property, and compliance constraints. Federated Learning preserves data locality, but existing federated paradigms typically treat clients as independent and fail to model workflow dependencies that are crucial for coherent multi-stage collaboration. We introduce FedWave , a federated and workflow-aware framework that enables LLM-based experts to solve sequential tasks under data-locality constraints. FedWave combines a Value Chain Layer to model inter-stage dependencies, federated LoRA adaptation to preserve role-specific expertise, a shared Mixture-of-Experts (MoE) router for input-conditioned expert fusion, and a Direct Preference Optimization (DPO) stage based on router-induced preferences. Experiments across workflow datasets and LLM backbones show that FedWave outperforms federated baselines and remains competitive with centralized multi-agent systems while preserving data locality.