Domain Expansion: A Latent Space Construction Framework for Multi-Task Learning
Chi-Yao Huang, Khoa Vo, Aayush Atul Verma, Duo Lu, Yezhou Yang
Abstract
Training a single network with multiple objectives often leads to conflicting gradients that degrade shared representations, forcing them into a compromised state that is suboptimal for any single task-a problem we term latent representation collapse. We introduce Domain Expansion, a framework that prevents these conflicts by restructuring the latent space itself. Our framework uses a novel orthogonal pooling mechanism to construct a latent space where each objective is assigned to a mutually orthogonal subspace. We validate our approach across diverse benchmarks-including ShapeNet, MPIIGaze, and Rotated MNIST-on challenging multi-objective problems combining classification with pose and gaze estimation. Our experiments demonstrate that this structure not only prevents collapse but also yields an explicit, interpretable, and compositional latent space where concepts can be directly manipulated.
Ask Lune
Know this paper in minutes, not an afternoon.
Lune can read every page of it and the top-tier papers that cite it, and gives you its claims, its evidence and its limits, each one cited to the paper it comes from. No skimming, no AI slop.
Lineage
- Multi-Task Learning as a Bargaining Game cites Gradient Surgery for Multi-Task Learning
- Multi-Task Learning as a Bargaining Game cites Conflict-Averse Gradient Descent for Multi-task learning
- Learning Transferable Visual Models From Natural Language Supervision cites A Simple Framework for Contrastive Learning of Visual Representations
- Conflict-Averse Gradient Descent for Multi-task learning cites Gradient Surgery for Multi-Task Learning
Builds on9
- Learning Transferable Visual Models From Natural Language SupervisionAlec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh et al.ICML 2021 · 47,906 citations
- A Simple Framework for Contrastive Learning of Visual RepresentationsTing Chen, Simon Kornblith, Mohammad Norouzi, Geoffrey E. HintonICML 2020 · 24,064 citations
- Gradient Surgery for Multi-Task LearningTianhe Yu, Saurabh Kumar, Abhishek Gupta, Sergey Levine et al.NeurIPS 2020 · 2,261 citations
- Conflict-Averse Gradient Descent for Multi-task learningBo Liu, Xingchao Liu, Xiaojie Jin, Peter Stone et al.NeurIPS 2021 · 686 citations
- Multi-Task Learning as a Bargaining GameAviv Navon, Aviv Shamsian, Idan Achituve, Haggai Maron et al.ICML 2022 · 243 citations
Related papers
- Mitigating Modality Collapse in Multimodal VAEs via Impartial OptimizationAdrián Javaloy, Maryam Meghdadi, Isabel ValeraICML 2022 · 49 citations
- CORE-MTL: Rethinking Gradient Balancing via Causal Orthogonal RepresentationsChengfeng Wu, Tao Zou, Yanru Wu, Jingge WangICML 2026
- Domain Expansion of Image GeneratorsYotam Nitzan, Michaël Gharbi, Richard Zhang, Taesung Park et al.CVPR 2023
- Visual Representation Learning over Latent DomainsLucas Deecke, Timothy M. Hospedales, Hakan BilenICLR 2022 · 14 citations
- Aligned Multi Objective OptimizationYonathan Efroni, Ben Kretzu, Daniel Jiang, Jalaj Bhandari et al.ICML 2025
