FedScar: Correcting Geometric Bias for Flatness-Consistent Federated Learning
Jianfeng Lu, YuZhao Xiang, Yue Chen, Gang Li, Shuqin Cao, Guanghui Wen
Abstract
Federated Learning (FL) often suffers from degraded generalization under statistical heterogeneity, where client updates systematically deviate from the global objective. While recent Sharpness-Aware Minimization (SAM) methods promote locally flat solutions, they implicitly assume that local flatness transfers to the global model, which generally does not hold under heterogeneous data distributions. This mismatch gives rise to a flatness discrepancy induced by misaligned loss landscapes. To address this issue, we propose FedScar, a federated optimization framework that explicitly corrects heterogeneityinduced geometric inconsistency. FedScar maintains a history-accumulated geometric bias to capture persistent curvature skew across clients, and employs a variance-aware injection mechanism to steer local updates toward regions that are flat with respect to the global objective. We provide a theoretical interpretation of FedScar as a Split-Dual ADMM formulation, which jointly enforces parameter consensus and geometric alignment. Extensive experiments under severe heterogeneity demonstrate that FedScar consistently reduces flatness discrepancy and improves generalization over state-of-the-art methods, without incurring additional communication overhead. The source code is available at https://github.com/ Ultraman6/FedScar.
Ask about this paper
Your agent reads all of it.
Lune indexed this paper to the last equation, along with the top-tier papers that cite it. Ask a question and the answer quotes them.
Your agent calls
Luneget_paper_fulltext
Free to start. No credit card required.
Terminal
Install the CLIlune papers fulltext aef2447c-a183-4c60-b176-0693c4dd1b78Builds on14
- SCAFFOLD: Stochastic Controlled Averaging for Federated LearningSai Praneeth Karimireddy, Satyen Kale, Mehryar Mohri, Sashank J. Reddi et al.ICML 2020 · 3,875 citations
- Sharpness-aware Minimization for Efficiently Improving GeneralizationPierre Foret, Ariel Kleiner, Hossein Mobahi, Behnam NeyshaburICLR 2021 · 1,861 citations
- The Non-IID Data Quagmire of Decentralized Machine LearningKevin Hsieh, Amar Phanishayee, Onur Mutlu, Phillip B. GibbonsICML 2020 · 672 citations
- No Fear of Heterogeneity: Classifier Calibration for Federated Learning with Non-IID DataMi Luo, Fei Chen, Dapeng Hu, Yifan Zhang et al.NeurIPS 2021 · 510 citations
- Preservation of the Global Knowledge by Not-True Distillation in Federated LearningGihun Lee, Minchan Jeong, Yongjin Shin, Sangmin Bae et al.NeurIPS 2022 · 235 citations
Related papers
- Rethinking the Flat Minima Searching in Federated LearningTaehwan Lee, Sung Whan YoonICML 2024 · 10 citations
- Flexible Sharpness-Aware Personalized Federated LearningXinda Xing, Qiugang Zhan, Xiurui Xie, Yuning Yang et al.AAAI 2025 · 5 citations
- One Arrow, Two Hawks: Sharpness-aware Minimization for Federated Learning via Global Model TrajectoryYuhang Li, Tong Liu, Yangguang Cui, Ming Hu et al.ICML 2025
- Dynamic Regularized Sharpness Aware Minimization in Federated Learning: Approaching Global Consistency and Smooth LandscapeYan Sun, Li Shen, Shixiang Chen, Liang Ding et al.ICML 2023 · 69 citations
- Locally Estimated Global Perturbations are Better than Local Perturbations for Federated Sharpness-aware MinimizationZiqing Fan, Shengchao Hu, Jiangchao Yao, Gang Niu et al.ICML 2024 · 35 citations
