DRP: Distilled Reasoning Pruning with Mathematical Skill-aware Step Decomposition for Efficient Large Reasoning Models

While Large Reasoning Models (LRMs) excel at complex tasks via long Chain-of-Thought (CoT) reasoning, their outputs are often excessively verbose, leading to inefficiency. This problem is amplified when the student's longform reasoning mismatches the concise outputs of smaller teacher models-common in LLM distillation to avoid using costly large teachers. To address this issue, we propose Distilled Reasoning Pruning (DRP), a hybrid framework that combines inference-time pruning with tuning-based distillation. DRP adopts a trajectory-aware distillation paradigm, in which the teacher operates directly on the student's reasoning paths by performing skillaware step decomposition and pruning. The resulting refined trajectories are then distilled back into the student model, providing targeted supervision that aligns closely with the student's policy and facilitates more effective learning. This trajectory-aware design anticipates recent advances in On-Policy Distillation, as it grounds supervision in the student's own reasoning trajectories while retaining the efficiency of offline distillation. Across challenging math datasets, DRP significantly reduces token usage without sacrificing accuracy-for instance, cutting tokens on GSM8K from 917 to 328 while improving accuracy from 91.7% to 94.1%, and reducing AIME tokens by 43% with no performance drop. Further analysis shows that aligning training CoT structure with the student's capacity is key to effective knowledge transfer.