Lune

ACL Findings2026

HIPO: A Hierarchical Prompt Optimization Framework with Task Awareness and Fine-Grained Debugging

Lu Qi, Lei Chai, Hongrui Yu, Binhang Qi, Hailong Sun

Abstract

Large Language Models (LLMs) have demonstrated remarkable capabilities across diverse natural language processing tasks. However, their performance often hinges on carefully designed prompts, whose creation requires substantial human effort. While numerous automatic prompt optimization techniques have been proposed, existing methods typically apply the same prompt across all samples within a dataset, ignoring variation in sample difficulty. To address these limitations, we propose HIPO, a HIerarchical Prompt Optimization framework that shifts the paradigm from dataset-level to sample-level optimization. Our framework first employs a lightweight router model, trained offline, to predict the difficulty of each sample at test time. Based on this prediction, HIPO dynamically selects a prompt from a five-tiered hierarchy, tailoring complexity to sample difficulty. Furthermore, two refinement stages-Task Description Prompt Refine and Attribution-Based Prompt Refine-enhance generalizability and fine-grained optimization. Extensive experiments on 27 tasks demonstrate that HIPO outperforms all baselines, achieving state-ofthe-art performance on 25% more tasks than the strongest baseline. Cost analysis further demonstrates substantial efficiency gains, reducing API calls, token consumption, and overall cost by 1.2× to 80×. Our implementation

Ask Lune

Know this paper in minutes, not an afternoon.

Lune can read every page of it and the top-tier papers that cite it, and gives you its claims, its evidence and its limits, each one cited to the paper it comes from. No skimming, no AI slop.

Questions to start from

Lune can draw on

the full paper · related work

Get the answer

Free to start. No credit card required.

Lineage

  • Promptbreeder: Self-Referential Self-Improvement via Prompt Evolution cites Self-Consistency Improves Chain of Thought Reasoning in Language Models
  • Promptbreeder: Self-Referential Self-Improvement via Prompt Evolution cites Large Language Models as Optimizers
  • Large Language Models as Optimizers cites Promptbreeder: Self-Referential Self-Improvement via Prompt Evolution
  • Promptbreeder: Self-Referential Self-Improvement via Prompt Evolution cites Large Language Models are Human-Level Prompt Engineers
  • Large Language Models as Optimizers cites Self-Consistency Improves Chain of Thought Reasoning in Language Models
  • Large Language Models as Optimizers cites AutoPrompt: Eliciting Knowledge from Language Models with Automatically Generated Prompts
  • Large Language Models as Optimizers cites Large Language Models are Human-Level Prompt Engineers
  • Large Language Models are Human-Level Prompt Engineers cites AutoPrompt: Eliciting Knowledge from Language Models with Automatically Generated Prompts
Builds onCited byLeft to right by year; the more cited, the higher.

Builds on21

  1. AutoPrompt: Eliciting Knowledge from Language Models with Automatically Generated PromptsTaylor Shin, Yasaman Razeghi, Robert L. Logan IV, Eric Wallace et al.EMNLP 2020 · 1,162 citations
  2. Large Language Models as OptimizersChengrun Yang, Xuezhi Wang, Yifeng Lu, Hanxiao Liu et al.ICLR 2024 · 817 citations
  3. Self-Consistency Improves Chain of Thought Reasoning in Language ModelsXuezhi Wang, Jason Wei, Dale Schuurmans, Quoc V. Le et al.ICLR 2023 · 681 citations
  4. Promptbreeder: Self-Referential Self-Improvement via Prompt EvolutionChrisantha Fernando, Dylan Banarse, Henryk Michalewski, Simon Osindero et al.ICML 2024 · 432 citations
  5. Large Language Models are Human-Level Prompt EngineersYongchao Zhou, Andrei Ioan Muresanu, Ziwen Han, Keiran Paster et al.ICLR 2023 · 297 citations

Related papers

  1. Unleashing the Potential of Large Language Models as Prompt Optimizers: Analogical Analysis with Gradient-based Model OptimizersXinyu Tang, Xiaolei Wang, Wayne Xin Zhao, Siyuan Lu et al.AAAI 2025 · 36 citations
  2. SEE: Strategic Exploration and Exploitation for Cohesive In-Context Prompt OptimizationWendi Cui, Jiaxin Zhang, Zhuohang Li, Hao Sun et al.ACL 2025
  3. TAMPO: Task- and Model-Aware Automatic Prompt Optimization for Auto-Routing in LLM-based SystemsYuejun Jiao, Jun Xia, Yanxin Yang, Yonghao Yang et al.ICML 2026
  4. PRompt Optimization in Multi-Step Tasks (PROMST): Integrating Human Feedback and Heuristic-based SamplingYongchao Chen, Jacob Arkin, Yilun Hao, Yang Zhang et al.EMNLP 2024 · 6 citations
  5. MAPO: Boosting Large Language Model Performance with Model-Adaptive Prompt OptimizationYuyan Chen, Zhihao Wen, Ge Fan, Zhengyu Chen et al.EMNLP Findings 2023 · 18 citations
Dusk over the sea between two cliffs drawn in fine vertical lines