Lune

EMNLP Findings2025

Weak2Wise: An Automated, Lightweight Framework for Weak-LLM-Friendly Reasoning Synthesis

Jianing Lin, Yuanfang Guo, Shunning Liu, Zeming Liu, Yunhong Wang

Abstract

Recent advances in large language model (LLM) fine-tuning have shown that training data augmented with high-quality reasoning traces can remarkably improve downstream performance. However, existing approaches usually rely on expensive manual annotations or auxiliary models, and fail to address the unique constraints of smaller "weak" LLMs. To bridge these gaps, we introduce Weak2Wise, a fully automated, lightweight framework for synthesizing high-quality, weak-LLM-friendly reasoning traces. Starting from a QA dataset, Weak2Wise filters out the samples that can already be correctly answered by the weak LLM, gathers diverse candidate reasoning traces from multiple strong LLMs, and leverages our Step-Mask scoring to rank and truncate the most guidance-effective traces. These reasoning traces are then used for fine-tuning, yielding substantial improvements in the weak LLM's reasoning abilities. The name Weak2Wise has two meanings: using a "weak" LLM to select the "wisest" reasoning traces generated by stronger LLMs, and fine-tuning the same weak LLM on these reasoning traces to become "wiser". We further use Weak2Wise to build GR-1K, a 1,000-sample math and science QA-reasoning dataset optimized for weak LLMs, and fine-tune Qwen2.5-7B on it to create GR-7B, which achieves superior performance on AIME2024, MATH-500, and GPQA Diamond benchmarks. Our codes are publicly released to facilitate further research 1 .

Ask Lune

Know this paper in minutes, not an afternoon.

Lune can read every page of it and the top-tier papers that cite it, and gives you its claims, its evidence and its limits, each one cited to the paper it comes from. No skimming, no AI slop.

Questions to start from

Lune can draw on

the full paper · related work

Get the answer

Free to start. No credit card required.

Lineage

Builds onCited byLeft to right by year; the more cited, the higher.

Builds on14

  1. Tree of Thoughts: Deliberate Problem Solving with Large Language ModelsShunyu Yao, Dian Yu, Jeffrey Zhao, Izhak Shafran et al.NeurIPS 2023 · 5,068 citations
  2. Let's Verify Step by StepHunter Lightman, Vineet Kosaraju, Yuri Burda, Harrison Edwards et al.ICLR 2024 · 3,045 citations
  3. Distilling Step-by-Step! Outperforming Larger Language Models with Less Training Data and Smaller Model SizesCheng-Yu Hsieh, Chun-Liang Li, Chih-Kuan Yeh, Hootan Nakhost et al.ACL Findings 2023 · 317 citations
  4. When Scaling Meets LLM Finetuning: The Effect of Data, Model and Finetuning MethodBiao Zhang, Zhongtao Liu, Colin Cherry, Orhan FiratICLR 2024 · 271 citations
  5. Automatic Chain of Thought Prompting in Large Language ModelsZhuosheng Zhang, Aston Zhang, Mu Li, Alex SmolaICLR 2023 · 234 citations

Related papers

  1. Weak-to-Strong ReasoningYuqing Yang, Yan Ma, Pengfei LiuEMNLP Findings 2024 · 3 citations
  2. Prompting Test-Time Scaling Is A Strong LLM Reasoning Data AugmentationSondos Mahmoud Bsharat, Zhiqiang ShenACL Findings 2026
  3. Masked Thought: Simply Masking Partial Reasoning Steps Can Improve Mathematical Reasoning Learning of Language ModelsChangyu Chen, Xiting Wang, Ting-En Lin, Ang Lv et al.ACL 2024 · 3 citations
  4. The Self-Improvement Paradox: Can Language Models Bootstrap Reasoning Capabilities without External Scaffolding?Yutao Sun, Mingshuai Chen, Tiancheng Zhao, Ruochen Xu et al.ACL Findings 2025 · 1 citation
  5. Learning from Synthetic Data Improves Multi-hop ReasoningAnmol Kabra, Yilun Yin, Albert Gong, Kamilė Stankevičiūtė et al.ICLR 2026 · 6 citations
Dusk over the sea between two cliffs drawn in fine vertical lines