Direct Prompt Optimization with Continuous Representations
Yangkun Wang, Zihan Wang, Jingbo Shang
摘要
Prompt optimization for language models faces challenges due to the large discrete search space, the reliance on continuous gradient updates, and the need to round continuous representations into discrete prompts, which causes inflexibility and instability. Existing methods attempt to address these by constraining the search space and adopting greedy, incremental improvements, but they often fail to fully leverage historical gradient information. In this paper, we model the prompt optimization problem by the probability distribution of the prompt and present a novel approach that integrates greedy strategies into optimization with continuous representations. This approach can exploit historical gradient information to address the instability caused by rounding in existing methods. Our study indicates that using continuous representations can improve prompt optimization performance on both text classification and attack tasks, as well as models, including GPT-2, OPT, Vicuna, and LLaMA-2, and also be adaptable to models of different sizes.
问 Lune
几分钟读透这篇 Paper,不用再花一下午。
Lune 能读完全文,也能读引用它的顶会 Paper,讲清它的结论、证据和局限,每一点都注明出自哪篇 Paper。不用硬啃,也不靠猜。
脉络
- Hard Prompts Made Easy: Gradient-Based Discrete Optimization for Prompt Tuning and Discovery 引用了 Language Models are Few-Shot Learners
- Large Language Models are Zero-Shot Reasoners 引用了 Language Models are Few-Shot Learners
- Large Language Models are Zero-Shot Reasoners 引用了 Chain-of-Thought Prompting Elicits Reasoning in Large Language Models
- Large Language Models are Human-Level Prompt Engineers 引用了 Large Language Models are Zero-Shot Reasoners
- Large Language Models are Human-Level Prompt Engineers 引用了 Language Models are Few-Shot Learners
- Large Language Models are Human-Level Prompt Engineers 引用了 Chain-of-Thought Prompting Elicits Reasoning in Large Language Models
它引用的 Paper7
- Language Models are Few-Shot LearnersTom B. Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah 等NeurIPS 2020 · 64,255 次引用
- Chain-of-Thought Prompting Elicits Reasoning in Large Language ModelsJason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma 等NeurIPS 2022 · 22,562 次引用
- Large Language Models are Zero-Shot ReasonersTakeshi Kojima, Shixiang Shane Gu, Machel Reid, Yutaka Matsuo 等NeurIPS 2022 · 8,168 次引用
- Hard Prompts Made Easy: Gradient-Based Discrete Optimization for Prompt Tuning and DiscoveryYuxin Wen, Neel Jain, John Kirchenbauer, Micah Goldblum 等NeurIPS 2023 · 454 次引用
- Large Language Models are Human-Level Prompt EngineersYongchao Zhou, Andrei Ioan Muresanu, Ziwen Han, Keiran Paster 等ICLR 2023 · 297 次引用
相关 Paper
- RLPrompt: Optimizing Discrete Text Prompts with Reinforcement LearningMingkai Deng, Jianyu Wang, Cheng-Ping Hsieh, Yihan Wang 等EMNLP 2022 · 141 次引用
- Unleashing the Potential of Large Language Models as Prompt Optimizers: Analogical Analysis with Gradient-based Model OptimizersXinyu Tang, Xiaolei Wang, Wayne Xin Zhao, Siyuan Lu 等AAAI 2025 · 36 次引用
- On Discrete Prompt Optimization for Diffusion ModelsRuochen Wang, Ting Liu, Cho-Jui Hsieh, Boqing GongICML 2024 · 30 次引用
- Two Examples are Better than One: Context Regularization for Gradient-based Prompt TuningHyeonmin Ha, Soyoung Jung, Jinsol Park, Minjoon Seo 等ACL Findings 2023 · 1 次引用
- Automatic Prompt Optimization with "Gradient Descent" and Beam SearchReid Pryzant, Dan Iter, Jerry Li, Yin Tat Lee 等EMNLP 2023 · 137 次引用
