Lune

EMNLP Findings2023

Toward Human Readable Prompt Tuning: Kubrick's The Shining is a good movie, and a good prompt too?

Weijia Shi, Xiaochuang Han, Hila Gonen, Ari Holtzman, Yulia Tsvetkov, Luke Zettlemoyer

摘要

Large language models can perform downstream tasks in a zero-shot fashion, given natural language prompts that specify the desired behavior.Such prompts are typically hand engineered, but can also be learned with gradientbased methods from labeled data.However, it is underexplored what factors make the prompts effective, especially when the prompts are in natural language.In this paper, we investigate common attributes shared by effective prompts in classification problems.We first propose a human readable prompt tuning method (FLUENTPROMPT) based on Langevin dynamics that incorporates a fluency constraint to find a distribution of effective and fluent prompts.Our analysis reveals that effective prompts are topically related to the task domain and calibrate the prior probability of output labels.Based on these findings, we also propose a method for generating prompts using only unlabeled data, outperforming strong baselines by an average of 7.0% accuracy across three tasks.We release our code and data in github.com/swj0419/FluentPrompt. Order randomly determined.

问 Lune

几分钟读透这篇 Paper,不用再花一下午。

Lune 能读完全文,也能读引用它的顶会 Paper,讲清它的结论、证据和局限,每一点都注明出自哪篇 Paper。不用硬啃,也不靠猜。

已有 11 篇顶会 Paper 引用了它。别等审稿人问起,先弄清它们发现了什么。

可以从这些问题问起

Lune 可以查阅

全文 · 引用它的顶会 Paper · 相关的 Paper

获取答案

免费开始,无需绑卡

脉络

  • Use Your INSTINCT: INSTruction optimization for LLMs usIng Neural bandits Coupled with Transformers 引用了 Large Language Models are Zero-Shot Reasoners
  • Use Your INSTINCT: INSTruction optimization for LLMs usIng Neural bandits Coupled with Transformers 引用了 Language Models are Few-Shot Learners
  • Multitask Prompted Training Enables Zero-Shot Task Generalization 引用了 Language Models are Few-Shot Learners
  • Fantastically Ordered Prompts and Where to Find Them: Overcoming Few-Shot Prompt Order Sensitivity 引用了 Language Models are Few-Shot Learners
  • Fantastically Ordered Prompts and Where to Find Them: Overcoming Few-Shot Prompt Order Sensitivity 引用了 Calibrate Before Use: Improving Few-shot Performance of Language Models
  • Large Language Models are Zero-Shot Reasoners 引用了 Multitask Prompted Training Enables Zero-Shot Task Generalization
  • Large Language Models are Zero-Shot Reasoners 引用了 Fantastically Ordered Prompts and Where to Find Them: Overcoming Few-Shot Prompt Order Sensitivity
  • Large Language Models are Zero-Shot Reasoners 引用了 Language Models are Few-Shot Learners
  • Prompting4Debugging: Red-Teaming Text-to-Image Diffusion Models by Finding Problematic Prompts 引用了 Language Models are Few-Shot Learners
  • Putting People in LLMs' Shoes: Generating Better Answers via Question Rewriter 引用了 Large Language Models are Zero-Shot Reasoners
  • SPRIG: Improving Large Language Model Performance by System Prompt Optimization 引用了 Language Models are Few-Shot Learners
  • Calibrate Before Use: Improving Few-shot Performance of Language Models 引用了 Language Models are Few-Shot Learners
它引用的引用它的从左到右按年份排列,引用越多,位置越高。

引用它的顶会 Paper11

  1. Prompting4Debugging: Red-Teaming Text-to-Image Diffusion Models by Finding Problematic PromptsZhi-Yi Chin, Chieh-Ming Jiang, Ching-Chun Huang, Pin-Yu Chen 等ICML 2024 · 155 次引用
  2. SPRIG: Improving Large Language Model Performance by System Prompt OptimizationLechen Zhang, Tolga Ergen, Lajanugen Logeswaran, Moontae Lee 等ICLR 2026 · 35 次引用
  3. Use Your INSTINCT: INSTruction optimization for LLMs usIng Neural bandits Coupled with TransformersXiaoqiang Lin, Zhaoxuan Wu, Zhongxiang Dai, Wenyang Hu 等ICML 2024 · 26 次引用
  4. PR-Attack: Coordinated Prompt-RAG Attacks on Retrieval-Augmented Generation in Large Language Models via Bilevel OptimizationYang Jiao, Xiaodong Wang, Kai YangSIGIR 2025 · 6 次引用
  5. Putting People in LLMs' Shoes: Generating Better Answers via Question RewriterJunhao Chen, Bowen Wang, Zhouqiang Jiang, Yuta NakashimaAAAI 2025 · 5 次引用

它引用的 Paper16

  1. Language Models are Few-Shot LearnersTom B. Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah 等NeurIPS 2020 · 64,255 次引用
  2. Large Language Models are Zero-Shot ReasonersTakeshi Kojima, Shixiang Shane Gu, Machel Reid, Yutaka Matsuo 等NeurIPS 2022 · 8,168 次引用
  3. Multitask Prompted Training Enables Zero-Shot Task GeneralizationVictor Sanh, Albert Webson, Colin Raffel, Stephen H. Bach 等ICLR 2022 · 1,976 次引用
  4. Calibrate Before Use: Improving Few-shot Performance of Language ModelsZihao Zhao, Eric Wallace, Shi Feng, Dan Klein 等ICML 2021 · 1,843 次引用
  5. Fantastically Ordered Prompts and Where to Find Them: Overcoming Few-Shot Prompt Order SensitivityYao Lu, Max Bartolo, Alastair Moore, Sebastian Riedel 等ACL 2022 · 1,494 次引用

相关 Paper

  1. Large Language Models as OptimizersChengrun Yang, Xuezhi Wang, Yifeng Lu, Hanxiao Liu 等ICLR 2024 · 817 次引用
  2. Zero-Shot Continuous Prompt Transfer: Generalizing Task Semantics Across Language ModelsZijun Wu, Yongkang Wu, Lili MouICLR 2024 · 9 次引用
  3. PPT: Pre-trained Prompt Tuning for Few-shot LearningYuxian Gu, Xu Han, Zhiyuan Liu, Minlie HuangACL 2022
  4. Two Examples are Better than One: Context Regularization for Gradient-based Prompt TuningHyeonmin Ha, Soyoung Jung, Jinsol Park, Minjoon Seo 等ACL Findings 2023 · 1 次引用
  5. Large Language Models are Human-Level Prompt EngineersYongchao Zhou, Andrei Ioan Muresanu, Ziwen Han, Keiran Paster 等ICLR 2023 · 297 次引用
黄昏的海面,两侧是细线勾勒的悬崖