Lune

ICLR2025Top-tier venue

Skill Expansion and Composition in Parameter Space

Tenglong Liu, Jianxiong Li, Yinan Zheng, Haoyi Niu, Yixing Lan, Xin Xu, Xianyuan Zhan

Abstract

Humans excel at reusing prior knowledge to address new challenges and developing skills while solving problems. This paradigm becomes increasingly popular in the development of autonomous agents, as it develops systems that can self-evolve in response to new challenges like human beings. However, previous methods suffer from limited training efficiency when expanding new skills and fail to fully leverage prior knowledge to facilitate new task learning. In this paper, we propose Parametric Skill Expansion and Composition (PSEC), a new framework designed to iteratively evolve the agents' capabilities and efficiently address new challenges by maintaining a manageable skill library. This library can progressively integrate skill primitives as plug-and-play Low-Rank Adaptation (LoRA) modules in parameter-efficient finetuning, facilitating efficient and flexible skill expansion. This structure also enables the direct skill compositions in parameter space by merging LoRA modules that encode different skills, leveraging shared information across skills to effectively program new skills. Based on this, we propose a context-aware module to dynamically activate different skills to collaboratively handle new tasks. Empowering diverse applications including multi-objective composition, dynamics shift, and continual policy shift, the results on D4RL, DSRL benchmarks, and the DeepMind Control Suite show that PSEC exhibits superior capacity to leverage prior knowledge to efficiently tackle new challenges, as well as expand its skill libraries to evolve the capabilities. Project website: https://ltlhuuu.github.io/PSEC/.

Ask Lune

Know this paper in minutes, not an afternoon.

Lune can read every page of it and the top-tier papers that cite it, and gives you its claims, its evidence and its limits, each one cited to the paper it comes from. No skimming, no AI slop.

9 top-tier papers have cited it since. Know what they found before your reviewers do.

Questions to start from

Lune can draw on

the full paper · the top-tier papers that cite it · related work

Get the answer

Free to start. No credit card required.

Lineage

  • DADP: Domain Adaptive Diffusion Policy cites Denoising Diffusion Probabilistic Models
  • Diffusion-Based Planning for Autonomous Driving with Flexible Guidance cites Denoising Diffusion Probabilistic Models
  • Decoupled Diffusion Sparks Adaptive Scene Generation cites Diffusion-Based Planning for Autonomous Driving with Flexible Guidance
  • Learning Diffusion Policy from Primitive Skills for Robot Manipulation cites LoRA: Low-Rank Adaptation of Large Language Models
  • Offline Reinforcement Learning with Implicit Q-Learning cites Conservative Q-Learning for Offline Reinforcement Learning
  • X-VLA: Soft-Prompted Transformer as Scalable Cross-Embodiment Vision-Language-Action Model cites LoRA: Low-Rank Adaptation of Large Language Models
Builds onCited byLeft to right by year; the more cited, the higher.

Cited by top-tier papers9

  1. X-VLA: Soft-Prompted Transformer as Scalable Cross-Embodiment Vision-Language-Action ModelJinliang Zheng, Jianxiong Li, Zhihao Wang, Dongxiu Liu et al.ICLR 2026 · 335 citations
  2. Decoupled Diffusion Sparks Adaptive Scene GenerationYunsong Zhou, Naisheng Ye, William Ljungbergh, Tianyu Li et al.ICCV 2025 · 2 citations
  3. DADP: Domain Adaptive Diffusion PolicyPengcheng Wang, Qinghang Liu, Haotian Lin, Yiheng Li et al.ICML 2026 · 1 citation
  4. Diffusion-Based Planning for Autonomous Driving with Flexible GuidanceYinan Zheng, Ruiming Liang, Kexin Zheng, Jinliang Zheng et al.ICLR 2025
  5. Learning Diffusion Policy from Primitive Skills for Robot ManipulationZhihao Gu, Ming Yang, Difan Zou, Dong XuAAAI 2026

Builds on48

  1. Denoising Diffusion Probabilistic ModelsJonathan Ho, Ajay Jain, Pieter AbbeelNeurIPS 2020 · 35,902 citations
  2. LoRA: Low-Rank Adaptation of Large Language ModelsEdward J. Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu et al.ICLR 2022 · 18,833 citations
  3. Conservative Q-Learning for Offline Reinforcement LearningAviral Kumar, Aurick Zhou, George Tucker, Sergey LevineNeurIPS 2020 · 2,881 citations
  4. Gradient Surgery for Multi-Task LearningTianhe Yu, Saurabh Kumar, Abhishek Gupta, Sergey Levine et al.NeurIPS 2020 · 2,261 citations
  5. Offline Reinforcement Learning with Implicit Q-LearningIlya Kostrikov, Ashvin Nair, Sergey LevineICLR 2022 · 1,402 citations

Related papers

  1. Composing Task-Agnostic Policies with Deep Reinforcement LearningAhmed Hussain Qureshi, Jacob J. Johnson, Yuzhe Qin, Taylor Henderson et al.ICLR 2020 · 35 citations
  2. Meta-learning Parameterized SkillsHaotian Fu, Shangqun Yu, Saket Tiwari, Michael Littman et al.ICML 2023 · 8 citations
  3. Understanding LoRA as Knowledge Memory: An Empirical AnalysisSeungju Back, Dongwoo Lee, Naun Kang, Taehee Lee et al.ICML 2026 · 10 citations
  4. LoRACoE: Improving Large Language Model via Composition-based LoRA ExpertGuanyu Li, Zhiheng Xi, Zhihao Zhang, Boyang Hong et al.EMNLP 2025
  5. Think Small, Act Big: Primitive Prompt Learning for Lifelong Robot ManipulationYuanqi Yao, Siao Liu, Haoming Song, Delin Qu et al.CVPR 2025
Dusk over the sea between two cliffs drawn in fine vertical lines