Lune

ACL Findings2024

PACE: Improving Prompt with Actor-Critic Editing for Large Language Model

Yihong Dong, Kangcheng Luo, Xue Jiang, Zhi Jin, Ge Li

Abstract

Large language models (LLMs) have showcased remarkable potential across various tasks by conditioning on prompts. However, the quality of different human-written prompts leads to substantial discrepancies in LLMs' performance, and improving prompts usually necessitates considerable human effort and expertise. To this end, this paper proposes Prompt with Actor-Critic Editing (PACE) for LLMs to enable automatic prompt editing. Drawing inspiration from the actor-critic algorithm in reinforcement learning, PACE leverages LLMs as the dual roles of actors and critics, conceptualizing prompt as a type of policy. PACE refines prompt, taking into account the feedback from both actors performing prompt and critics criticizing response. This process helps LLMs better align prompt to a specific task, thanks to real responses and thinking from LLMs. We conduct extensive experiments on 24 instruction induction tasks and 21 big-bench tasks. Experimental results indicate that PACE elevates the relative performance of medium/low-quality human-written prompts by up to 98%, which has comparable performance to high-quality human-written prompts. Moreover, PACE also exhibits notable efficacy for prompt generation.

Ask Lune

Know this paper in minutes, not an afternoon.

Lune can read every page of it and the top-tier papers that cite it, and gives you its claims, its evidence and its limits, each one cited to the paper it comes from. No skimming, no AI slop.

6 top-tier papers have cited it since. Know what they found before your reviewers do.

Questions to start from

Lune can draw on

the full paper · the top-tier papers that cite it · related work

Get the answer

Free to start. No credit card required.

Lineage

  • A Systematic Survey of Automatic Prompt Optimization Techniques cites Large Language Models are Zero-Shot Reasoners
  • Large Language Models are Zero-Shot Reasoners cites Language Models are Few-Shot Learners
  • Large Language Models are Zero-Shot Reasoners cites Chain-of-Thought Prompting Elicits Reasoning in Large Language Models
  • Large Language Models are Zero-Shot Reasoners cites Training language models to follow instructions with human feedback
  • Preference-Guided Prompt Optimization for Text-to-Image Generation cites Language Models are Few-Shot Learners
  • Preference-Guided Prompt Optimization for Text-to-Image Generation cites Chain-of-Thought Prompting Elicits Reasoning in Large Language Models
  • No Loss, No Gain: Gated Refinement and Adaptive Compression for Prompt Optimization cites A Systematic Survey of Automatic Prompt Optimization Techniques
  • No Loss, No Gain: Gated Refinement and Adaptive Compression for Prompt Optimization cites Large Language Models are Zero-Shot Reasoners
  • No Loss, No Gain: Gated Refinement and Adaptive Compression for Prompt Optimization cites Chain-of-Thought Prompting Elicits Reasoning in Large Language Models
  • Chain-of-Thought Prompting Elicits Reasoning in Large Language Models cites Training language models to follow instructions with human feedback
  • Training language models to follow instructions with human feedback cites Language Models are Few-Shot Learners
  • Textual Equilibrium Propagation for Deep Compound AI Systems cites Chain-of-Thought Prompting Elicits Reasoning in Large Language Models
Builds onCited byLeft to right by year; the more cited, the higher.

Cited by top-tier papers6

  1. StablePrompt : Automatic Prompt Tuning using Reinforcement Learning for Large Language ModelMinchan Kwon, Gaeun Kim, Jongsuk Kim, Haeil Lee et al.EMNLP 2024 · 11 citations
  2. No Loss, No Gain: Gated Refinement and Adaptive Compression for Prompt OptimizationWenhang Shi, Yiren Chen, Shuqing Bian, Xinyi Zhang et al.NeurIPS 2025 · 9 citations
  3. A Systematic Survey of Automatic Prompt Optimization TechniquesKiran Ramnath, Kang Zhou, Sheng Guan, Soumya Smruti Mishra et al.EMNLP 2025 · 5 citations
  4. Preference-Guided Prompt Optimization for Text-to-Image GenerationZhipeng Li, Yi-Chi Liao, Christian HolzCHI 2026 · 1 citation
  5. Textual Equilibrium Propagation for Deep Compound AI SystemsMinghui Chen, Wenlong Deng, James Zou, Han Yu et al.ICLR 2026 · 1 citation

Builds on24

  1. Language Models are Few-Shot LearnersTom B. Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah et al.NeurIPS 2020 · 64,255 citations
  2. Training language models to follow instructions with human feedbackLong Ouyang, Jeffrey Wu, Xu Jiang, Diogo Almeida et al.NeurIPS 2022 · 24,707 citations
  3. Chain-of-Thought Prompting Elicits Reasoning in Large Language ModelsJason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma et al.NeurIPS 2022 · 22,562 citations
  4. BERTScore: Evaluating Text Generation with BERTTianyi Zhang, Varsha Kishore, Felix Wu, Kilian Q. Weinberger et al.ICLR 2020 · 8,443 citations
  5. Large Language Models are Zero-Shot ReasonersTakeshi Kojima, Shixiang Shane Gu, Machel Reid, Yutaka Matsuo et al.NeurIPS 2022 · 8,168 citations

Related papers

  1. Evoke: Evoking Critical Thinking Abilities in LLMs via Reviewer-Author Prompt EditingXinyu Hu, Pengfei Tang, Simiao Zuo, Zihan Wang et al.ICLR 2024 · 14 citations
  2. ACING: Actor-Critic for Instruction Learning in Black-Box LLMsSalma Kharrat, Fares Fourati, Marco CaniniEMNLP 2025 · 1 citation
  3. Large Language Models are Human-Level Prompt EngineersYongchao Zhou, Andrei Ioan Muresanu, Ziwen Han, Keiran Paster et al.ICLR 2023 · 297 citations
  4. Unleashing the Potential of Large Language Models as Prompt Optimizers: Analogical Analysis with Gradient-based Model OptimizersXinyu Tang, Xiaolei Wang, Wayne Xin Zhao, Siyuan Lu et al.AAAI 2025 · 36 citations
  5. PRompt Optimization in Multi-Step Tasks (PROMST): Integrating Human Feedback and Heuristic-based SamplingYongchao Chen, Jacob Arkin, Yilun Hao, Yang Zhang et al.EMNLP 2024 · 6 citations
Dusk over the sea between two cliffs drawn in fine vertical lines