Think Less, Act Warranted: Efficient Tool-Integrated Reasoning via Dual-Efficiency Regularization
Yichen Xiao, Siyu Gong, Linan Yue
摘要
Recent methods using Reinforcement Learning (RL) have improved Tool-Integrated Reasoning (TIR) by training large language models to learn end-to-end policies for multi-step tool usage, enabling them to solve complex tasks more effectively. Despite these advances, existing methods often suffer from overthinking at both the action and reasoning levels: models tend to invoke tools redundantly and generate excessively long reasoning trajectories, resulting in high computational cost. To address this, in this paper, we propose LightTIR, a dual-penalty reward framework, to achieve efficient TIR. For action efficiency, LightTIR estimates the marginal utility of each tool call through prefix-aligned counterfactual trajectories, encouraging calls that contribute meaningful information while penalizing low-utility or redundant invocations. For reasoning efficiency, LightTIR introduces a length-aware regularization term, adaptively penalizing intermediate reasoning steps that exceed the minimal effective trajectory required for correct prediction. Extensive experiments demonstrate that LightTIR can reduce redundancy and trajectory expansion while maintaining answer correctness, achieving more efficient RL-based TIR. Code is available at https://github.com/ekventitas/LightTIR.
问问这篇 Paper
问问你的智能体。
Lune 读过与它相关的顶会 Paper,每个回答都会注明依据哪几篇。
相关 Paper
- Toward Effective Tool-Integrated Reasoning via Self-Evolved Preference LearningYifei Chen, Guanting Dong, Zhicheng DouICLR 2026 · 被引用 18 次
- Rethinking LLM Reasoning: From Explicit Trajectories to Latent RepresentationsCong Jiang, Xiaofeng Zhang, Fangzhi Zhu, XiaoWei Chen 等ICLR 2026
- DEPO: Dual-Efficiency Preference Optimization for LLM AgentsSirui Chen, Mengshi Zhao, Lei Xu, Yuying Zhao 等AAAI 2026 · 被引用 2 次
- MatchTIR: Fine-Grained Supervision for Tool-Integrated Reasoning via Bipartite MatchingChangle Qu, Sunhao Dai, Hengyi Cai, Jun Xu 等ACL 2026 · 被引用 3 次
- TInR: Exploring Tool-Internalized Reasoning in Large Language ModelsQiancheng Xu, Yongqi Li, Fan Liu, Hongru Wang 等ACL 2026
