ACL Findings2025
Synergistic Augmentation: Enhancing Cross-Domain Zero-Shot Slot Filling with Small Model-Assisted Large Language Models
Weizhen Li, Junbao Huang, Peijie Huang, Yuhong Xu, Jiekun Fan
Abstract
In real-world scenarios, cross-domain slot filling in spoken language understanding remains a significant challenge due to data scarcity. Previous works exhibit limited generalization ability in the target domain, demonstrating effective knowledge transfer only on seen slots while performing poorly on unseen slots. Although large language models (LLMs) can alleviate this issue to some extent, they underperform on seen slots compared to small models. To address these challenges, we introduce a novel framework that harnesses the power of a small model to augment the inferential capabilities of LLMs without additional training. Initially, we utilize target domain samples synthesized by LLMs as pre-calculated demonstrations, which are curated and chosen using confidence metrics derived from a small model. We further extract slot predictions from the small model to fully exploit its robust learning of familiar slots. Finally, during the inference process for test inputs, we integrate these demonstrations and slot prediction insights as references to enhance the slot filling performance of LLMs. Experiments on a slot filling dataset and a NER dataset including eight cross-domain settings show our framework achieves the best results. Our codes are publicly available at https://github.com/SIGSDSscau/SLSF.
Ask Lune
Know this paper in minutes, not an afternoon.
Lune can read every page of it and the top-tier papers that cite it, and gives you its claims, its evidence and its limits, each one cited to the paper it comes from. No skimming, no AI slop.
Lineage
- Self-Consistency Improves Chain of Thought Reasoning in Language Models cites Language Models are Few-Shot Learners
- Solving Quantitative Reasoning Problems with Language Models cites Self-Consistency Improves Chain of Thought Reasoning in Language Models
- Solving Quantitative Reasoning Problems with Language Models cites Language Models are Few-Shot Learners
- Efficient Memory Management for Large Language Model Serving with PagedAttention cites Language Models are Few-Shot Learners
Builds on17
- Language Models are Few-Shot LearnersTom B. Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah et al.NeurIPS 2020 · 64,255 citations
- Solving Quantitative Reasoning Problems with Language ModelsAitor Lewkowycz, Anders Andreassen, David Dohan, Ethan Dyer et al.NeurIPS 2022 · 2,039 citations
- Efficient Memory Management for Large Language Model Serving with PagedAttentionWoosuk Kwon, Zhuohan Li, Siyuan Zhuang, Ying Sheng et al.SOSP 2023 · 1,016 citations
- Self-Consistency Improves Chain of Thought Reasoning in Language ModelsXuezhi Wang, Jason Wei, Dale Schuurmans, Quoc V. Le et al.ICLR 2023 · 681 citations
- FlexKBQA: A Flexible LLM-Powered Framework for Few-Shot Knowledge Base Question AnsweringZhenyu Li, Sunqi Fan, Yu Gu, Xiuxing Li et al.AAAI 2024 · 143 citations
Related papers
- HierarchicalContrast: A Coarse-to-Fine Contrastive Learning Framework for Cross-Domain Zero-Shot Slot FillingJunwen Zhang, Yin ZhangEMNLP Findings 2023 · 5 citations
- Zero-Shot Slot Filling with Slot-Prefix Prompting and Attention Relationship DescriptorQiaoyang Luo, Lingqiao LiuAAAI 2023 · 10 citations
- Slot Transferability for Cross-domain Slot FillingHengtong Lu, Zhuoxin Han, Caixia Yuan, Xiaojie Wang et al.ACL Findings 2021 · 2 citations
- Generative Zero-Shot Prompt Learning for Cross-Domain Slot Filling with Inverse PromptingXuefeng Li, Liwen Wang, Guanting Dong, Keqing He et al.ACL Findings 2023 · 11 citations
- Speech-based Slot Filling using Large Language ModelsGuangzhi Sun, Shutong Feng, Dongcheng Jiang, Chao Zhang et al.ACL Findings 2024 · 2 citations
