CVPR Findings2026
Bi-Level Optimization for Single Domain Generalization
Marzi Heidari, Hanping Zhang, Hao Yan, Yuhong Guo
Abstract
Generalizing from a single labeled source domain to unseen target domains, without access to any target data during training, remains a fundamental challenge in robust machine learning. We address this underexplored setting, known as Single Domain Generalization (SDG), by proposing BiSDG, a bi-level optimization framework that explicitly decouples task learning from domain modeling. BiSDG simulates distribution shifts through surrogate domains constructed via label-preserving transformations of the source data. To capture domain-specific context, we propose a domain prompt encoder that generates lightweight modulation signals to produce augmenting features via feature-wise linear modulation. The learning process is formulated as a bi-level optimization problem: the inner objective optimizes task performance under fixed prompts, while the outer objective maximizes generalization across the surrogate domains by updating the domain prompt encoder. We further develop a practical gradient approximation scheme that enables efficient bi-level training without second-order derivatives. Extensive experiments on various SGD benchmarks demonstrate that BiSDG consistently outperforms prior methods, setting new state-of-theart performance in the SDG setting.
Ask Lune
Know this paper in minutes, not an afternoon.
Lune can read every page of it and the top-tier papers that cite it, and gives you its claims, its evidence and its limits, each one cited to the paper it comes from. No skimming, no AI slop.
Lineage
- SWAD: Domain Generalization by Seeking Flat Minima cites CutMix: Regularization Strategy to Train Strong Classifiers With Localizable Features
- SWAD: Domain Generalization by Seeking Flat Minima cites Moment Matching for Multi-Source Domain Adaptation
Builds on15
- CutMix: Regularization Strategy to Train Strong Classifiers With Localizable FeaturesSangdoo Yun, Dongyoon Han, Sanghyuk Chun, Seong Joon Oh et al.ICCV 2019 · 5,843 citations
- RandAugment: Practical Automated Data Augmentation with a Reduced Search SpaceEkin Dogus Cubuk, Barret Zoph, Jonathon Shlens, Quoc LeNeurIPS 2020 · 4,453 citations
- Moment Matching for Multi-Source Domain AdaptationXingchao Peng, Qinxun Bai, Xide Xia, Zijun Huang et al.ICCV 2019 · 2,239 citations
- AugMix: A Simple Data Processing Method to Improve Robustness and UncertaintyDan Hendrycks, Norman Mu, Ekin Dogus Cubuk, Barret Zoph et al.ICLR 2020 · 1,572 citations
- SWAD: Domain Generalization by Seeking Flat MinimaJunbum Cha, Sanghyuk Chun, Kyungjae Lee, Han-Cheol Cho et al.NeurIPS 2021 · 630 citations
Related papers
- Practical Single Domain Generalization via Training-time and Test-time LearningShuai Yang, Zhen Zhang, Lichuan GuKDD 2024 · 3 citations
- CLIP the Gap: A Single Domain Generalization Approach for Object DetectionVidit Vidit, Martin Engilberge, Mathieu SalzmannCVPR 2023
- Adversarial Domain Prompt Tuning and Generation for Single Domain GeneralizationZhipeng Xu, De Cheng, Xinyang Jiang, Nannan Wang et al.CVPR 2025
- CrossMatch: Cross-Classifier Consistency Regularization for Open-Set Single Domain GeneralizationRonghang Zhu, Sheng LiICLR 2022 · 49 citations
- Multi-Modal Style Transfer-based Prompt Tuning for Efficient Federated Domain GeneralizationYuliang Chen, Xi Lin, Jun Wu, Xiangrui Cai et al.AAAI 2026
