Lune

EMNLP Findings2024

CSLM: A Framework for Question Answering Dataset Generation through Collaborative Small Language Models

Yiming Wang, Yang Liu, Lingchen Wang, An Xiao

Abstract

Collecting high-quality question-answer (QA) pairs is vital for the training of large language models (LLMs), yet this process is traditionally laborious and time-intensive. With the rapid evolution of LLMs, the potential for leveraging these models to autonomously generate QA pairs has become apparent, particularly through the use of large-scale models like GPT-4. However, the computational demands and associated costs often render such approaches prohibitive for the average researcher. Addressing this gap, we introduce the Collaborative Small Language Model Framework (CSLM), an innovative solution that combines a group of small-scaled, open-source LLMs to collaboratively produce QA pairs. Experiments on datasets of various domains show that CSLM unleashes the full potential of diverse small models to generate high-quality QA pairs, making it accessible to a broader range of researchers.

Ask Lune

Know this paper in minutes, not an afternoon.

Lune can read every page of it and the top-tier papers that cite it, and gives you its claims, its evidence and its limits, each one cited to the paper it comes from. No skimming, no AI slop.

Questions to start from

Lune can draw on

the full paper · related work

Get the answer

Free to start. No credit card required.

Lineage

  • Publicly Shareable Clinical Large Language Model Built on Synthetic Clinical Notes cites Self-Instruct: Aligning Language Models with Self-Generated Instructions
Builds onCited byLeft to right by year; the more cited, the higher.

Builds on4

  1. Self-Instruct: Aligning Language Models with Self-Generated InstructionsYizhong Wang, Yeganeh Kordi, Swaroop Mishra, Alisa Liu et al.ACL 2023 · 540 citations
  2. Publicly Shareable Clinical Large Language Model Built on Synthetic Clinical NotesSunjun Kweon, Junu Kim, Jiyoun Kim, Sujeong Im et al.ACL Findings 2024 · 32 citations
  3. LIQUID: A Framework for List Question Answering Dataset GenerationSeongyun Lee, Hyunjae Kim, Jaewoo KangAAAI 2023 · 31 citations
  4. An Empirical Comparison of LM-based Question and Answer Generation MethodsAsahi Ushio, Fernando Alva-Manchego, José Camacho-ColladosACL Findings 2023 · 25 citations

Related papers

  1. FreeAL: Towards Human-Free Active Learning in the Era of Large Language ModelsRuixuan Xiao, Yiwen Dong, Junbo Zhao, Runze Wu et al.EMNLP 2023 · 9 citations
  2. Collaborative Training of Tiny-Large Vision Language ModelsShichen Lu, Longteng Guo, Wenxuan Wang, Zijia Zhao et al.ACM MM 2024 · 3 citations
  3. Hide and Seek in Noise Labels: Noise-Robust Collaborative Active Learning with LLMs-Powered AssistanceBo Yuan, Yulin Chen, Yin Zhang, Wei JiangACL 2024
  4. A Strategic Coordination Framework of Small LMs Matches Large LMs in Data SynthesisXin Gao, Qizhi Pei, Zinan Tang, Yu Li et al.ACL 2025
  5. Collaborative Enhancement of Large and Small Models for Question Answering via Dual Knowledge TransferShaofei Wang, Yunan Liu, Xiaolan Tang, Wenlong ChenAAAI 2026
Dusk over the sea between two cliffs drawn in fine vertical lines