EMNLP Findings2024
CSLM: A Framework for Question Answering Dataset Generation through Collaborative Small Language Models
Abstract
Collecting high-quality question-answer (QA) pairs is vital for the training of large language models (LLMs), yet this process is traditionally laborious and time-intensive. With the rapid evolution of LLMs, the potential for leveraging these models to autonomously generate QA pairs has become apparent, particularly through the use of large-scale models like GPT-4. However, the computational demands and associated costs often render such approaches prohibitive for the average researcher. Addressing this gap, we introduce the Collaborative Small Language Model Framework (CSLM), an innovative solution that combines a group of small-scaled, open-source LLMs to collaboratively produce QA pairs. Experiments on datasets of various domains show that CSLM unleashes the full potential of diverse small models to generate high-quality QA pairs, making it accessible to a broader range of researchers.
Ask Lune
Know this paper in minutes, not an afternoon.
Lune can read every page of it and the top-tier papers that cite it, and gives you its claims, its evidence and its limits, each one cited to the paper it comes from. No skimming, no AI slop.
Lineage
Builds on4
- Self-Instruct: Aligning Language Models with Self-Generated InstructionsYizhong Wang, Yeganeh Kordi, Swaroop Mishra, Alisa Liu et al.ACL 2023 · 540 citations
- Publicly Shareable Clinical Large Language Model Built on Synthetic Clinical NotesSunjun Kweon, Junu Kim, Jiyoun Kim, Sujeong Im et al.ACL Findings 2024 · 32 citations
- LIQUID: A Framework for List Question Answering Dataset GenerationSeongyun Lee, Hyunjae Kim, Jaewoo KangAAAI 2023 · 31 citations
- An Empirical Comparison of LM-based Question and Answer Generation MethodsAsahi Ushio, Fernando Alva-Manchego, José Camacho-ColladosACL Findings 2023 · 25 citations
Related papers
- FreeAL: Towards Human-Free Active Learning in the Era of Large Language ModelsRuixuan Xiao, Yiwen Dong, Junbo Zhao, Runze Wu et al.EMNLP 2023 · 9 citations
- Collaborative Training of Tiny-Large Vision Language ModelsShichen Lu, Longteng Guo, Wenxuan Wang, Zijia Zhao et al.ACM MM 2024 · 3 citations
- Hide and Seek in Noise Labels: Noise-Robust Collaborative Active Learning with LLMs-Powered AssistanceBo Yuan, Yulin Chen, Yin Zhang, Wei JiangACL 2024
- A Strategic Coordination Framework of Small LMs Matches Large LMs in Data SynthesisXin Gao, Qizhi Pei, Zinan Tang, Yu Li et al.ACL 2025
- Collaborative Enhancement of Large and Small Models for Question Answering via Dual Knowledge TransferShaofei Wang, Yunan Liu, Xiaolan Tang, Wenlong ChenAAAI 2026
