Lune

ACL Findings2026

A Learnable Skill Combination Strategy for Multi-task Learning in Natural Language Understanding

Zhe Yang, Yi Huang, Yaqin Chen, Mengfei Guo, Xiaoting Wu, Junlan Feng

Abstract

In the realm of domain-specific natural language understanding (NLU) tasks, acquiring high-quality labeled data is often arduous, thereby posing significant challenges for effective model training. Multi-task learning (MTL) addresses these limitations by jointly optimizing multiple tasks within a unified framework. In this paper, we introduce a novel sparse NLU multi-task learning framework that decomposes the language model into modular skill components and employs a dynamic, learnable skillcombination mechanism to adaptively handle diverse tasks. Extensive experiments on benchmark NLU datasets demonstrate that our proposed method surpasses conventional multitask learning approaches in performance.

Ask Lune

Know this paper in minutes, not an afternoon.

Lune can read every page of it and the top-tier papers that cite it, and gives you its claims, its evidence and its limits, each one cited to the paper it comes from. No skimming, no AI slop.

Questions to start from

Lune can draw on

the full paper · related work

Get the answer

Free to start. No credit card required.

Lineage

Builds onCited byLeft to right by year; the more cited, the higher.

Builds on4

  1. Revisiting Pre-Trained Models for Chinese Natural Language ProcessingYiming Cui, Wanxiang Che, Ting Liu, Bing Qin et al.EMNLP Findings 2020 · 866 citations
  2. Conditionally Adaptive Multi-Task Learning: Improving Transfer Learning in NLP Using Fewer Parameters & Less DataJonathan Pilault, Amine Elhattami, Christopher J. PalICLR 2021 · 105 citations
  3. OCNLI: Original Chinese Natural Language InferenceHai Hu, Kyle Richardson, Liang Xu, Lu Li et al.EMNLP Findings 2020 · 82 citations
  4. GradTS: A Gradient-Based Automatic Auxiliary Task Selection Method Based on Transformer NetworksWeicheng Ma, Renze Lou, Kai Zhang, Lili Wang et al.EMNLP 2021 · 4 citations

Related papers

  1. AdapMTL: Adaptive Pruning Framework for Multitask Learning ModelMingcan Xiang, Jiaxun Tang, Qizheng Yang, Hui Guan et al.ACM MM 2024 · 4 citations
  2. Low-Rank Updates of pre-trained Weights for Multi-Task LearningAlexandre Audibert, Massih-Reza Amini, Konstantin Usevich, Marianne ClauselACL Findings 2023 · 2 citations
  3. Towards Unsupervised Language Understanding and Generation by Joint Dual LearningShang-Yu Su, Chao-Wei Huang, Yun-Nung ChenACL 2020 · 19 citations
  4. Skill Weaving: Efficient LLM Improvement via Modular SkillpacksZhuo Li, Guodong Du, Zesheng Shi, Weiyang Guo et al.ACL Findings 2026
  5. Customizable Combination of Parameter-Efficient Modules for Multi-Task LearningHaowen Wang, Tao Sun, Congyun Jin, Yingbo Wang et al.ICLR 2024 · 8 citations
Dusk over the sea between two cliffs drawn in fine vertical lines