Lune

ACL Findings2026

Pru-CoT: Towards Efficient Reasoning Distillation via Pruning Chain-of-Thought

Han Liu, Shuotian Ma, Hui Li, Xiaotong Zhang, Fenglong Ma, Hong Yu

Abstract

Knowledge distillation has emerged as a pivotal paradigm for transferring the superior reasoning capabilities of Large Reasoning Models (LRMs) to efficient student models. However, the raw Chain-of-Thought (CoT) trajectories are often verbose and redundant, which dilutes the underlying logic and hinders effective knowledge distillation for student models.

Although recent work has focused on pruning CoT to streamline these reasoning paths, existing local heuristic methods often fail to capture global causal logic due to rigid rules and limited search spaces, while global heuristic approaches incur substantial computational costs. To address these issues, we propose Pru-CoT (Pruning Chain-of-Thought), a framework that aims to extract the essential logical structure from reasoning chains. Pru-CoT implements a step-level importance assessment via global optimization on a frozen student large language model (LLM), quantifying the gradient-based causal contribution of each component. Guided by these important signals, the framework performs fidelity-constrained pruning, utilizing an LLM-driven process to synthesize concise, logically coherent narratives. Extensive experiments on mathematical reasoning benchmarks demonstrate that models trained with Pru-CoT not only achieve superior accuracy but also generate significantly more compact reasoning paths compared to those trained on raw verbose data.

Ask Lune

Know this paper in minutes, not an afternoon.

Lune can read every page of it and the top-tier papers that cite it, and gives you its claims, its evidence and its limits, each one cited to the paper it comes from. No skimming, no AI slop.

Questions to start from

Lune can draw on

the full paper · related work

Get the answer

Free to start. No credit card required.

Lineage

  • Let's Verify Step by Step cites Chain-of-Thought Prompting Elicits Reasoning in Large Language Models
Builds onCited byLeft to right by year; the more cited, the higher.

Builds on10

  1. Chain-of-Thought Prompting Elicits Reasoning in Large Language ModelsJason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma et al.NeurIPS 2022 · 22,562 citations
  2. Let's Verify Step by StepHunter Lightman, Vineet Kosaraju, Yuri Burda, Harrison Edwards et al.ICLR 2024 · 3,045 citations
  3. FlashAttention-2: Faster Attention with Better Parallelism and Work PartitioningTri DaoICLR 2024 · 2,600 citations
  4. Efficient Memory Management for Large Language Model Serving with PagedAttentionWoosuk Kwon, Zhuohan Li, Siyuan Zhuang, Ying Sheng et al.SOSP 2023 · 1,016 citations
  5. OpenThoughts: Data Recipes for Reasoning ModelsEtash Kumar Guha, Ryan Marten, Sedrick Keh, Negin Raoof et al.ICLR 2026 · 235 citations

Related papers

  1. DRP: Distilled Reasoning Pruning with Mathematical Skill-aware Step Decomposition for Efficient Large Reasoning ModelsYuxuan Jiang, Dawei Li, Francis FerraroACL Findings 2026 · 2 citations
  2. Do LLMs Encode Functional Importance of Reasoning Tokens ?Janvijay Singh, Dilek Hakkani-TürACL 2026 · 3 citations
  3. Your Reasoning Model Knows What Counts: Self-Guided Chain-of-Thought Pruning for Efficient ReasoningZi-Ao Ma, Xian-Ling Mao, Tian Lan, Chen Xu et al.ACL 2026
  4. UniCoTT: A Unified Framework for Structural Chain-of-Thought DistillationXianwei Zhuang, Zhihong Zhu, Zhichang Wang, Xuxin Cheng et al.ICLR 2025
  5. Learning Graph Rationales to Compress Long Chains of Thought in Multimodal ReasoningYizhi Wang, Linan Yue, Deng-Bao Wang, Tong Wei et al.KDD 2026
Dusk over the sea between two cliffs drawn in fine vertical lines