Token-Budget-Aware LLM Reasoning

Reasoning is critical for large language models (LLMs) to excel in a wide range of tasks. While methods like Chain-of-Thought (CoT) reasoning and enhance LLM performance by decomposing problems into intermediate steps, they also incur significant overhead in token usage, leading to increased costs. We find that the reasoning process of current LLMs is unnecessarily lengthy and it can be compressed by including a reasonable token budget in the prompt, but the choice of token budget plays a crucial role in the actual compression effectiveness. We then propose a tokenbudget-aware LLM reasoning framework that dynamically adjusts the number of reasoning tokens based on the reasoning complexity of each problem. Experiments show that our method effectively reduces token costs in CoT reasoning with only a slight performance reduction, offering a practical solution to balance efficiency and accuracy in LLM reasoning. Code: https://github.com/GeniusHTX/TALE 1 . "It is not enough to have a good mind; the main thing is to use it well." -René Descartes * Equal Contribution. † Start the project and propose the idea. ‡ Corresponding Author. 1 Also available at https://www.gitlink.org.cn/ txhan/TALE Peyton's after-work activities will take 12.5 hours. Q: Peyton scheduled after-work activities of a onehour yoga class on Monday, a cooking class that lasts 9times as long as Monday' s yoga on Tuesday, a half-hour cheese-tasting event on Wednesday, a museum tour that takes half as long as the cooking class on Thursday, and two hours of errands on Friday. How many hours will all Peyton's afterwork activities take? (a) Direct answering (15 output tokens). Let's break down activities step by step: 1. Monday: Yoga class lasts 1 hour. -Total for Monday: 1 hour 2. Tuesday: Cooking class lasts 9 times as long as Monday's yoga class. -Cooking class duration = 9 * 1 hour = 9 hours -Total for Tuesday: 9 hours 3. Wednesday: Cheese-tasting event lasts half an hour. -Total for Wednesday: 0.5 hours 4. Thursday: Museum tour is half the cooking class. -Museum tour duration = 0.5 * 9 hours = 4.5 hours -Total for Thursday: 4.5 hours 5. Friday: Errands take 2 hours. -Total for Friday: 2 hours Now, let's add up all the hours from each day: -Monday: 1 hour -Tuesday: 9 hours -Wednesday: 0.5 hours -Thursday: 4.5 hours -Friday: 2 hours Now, we sum these amounts: 1 + 9 + 0.5 + 4.5 + 2 = 17 hours. Therefore, the total time is 17 hours.