Lune

ACL Findings2024

DORY: Deliberative Prompt Recovery for LLM

Lirong Gao, Ru Peng, Yiming Zhang, Junbo Zhao

Abstract

Prompt recovery in large language models (LLMs) is crucial for understanding how LLMs work and addressing concerns regarding privacy, copyright, etc. The trend towards inference-only APIs complicates this task by restricting access to essential outputs for recovery. To tackle this challenge, we extract promptrelated information from limited outputs and identify a strong(negative) correlation between output probability-based uncertainty and the success of prompt recovery. This finding led to the development of Deliberative PrOmpt RecoverY (DORY), our novel approach that leverages uncertainty to recover prompts accurately. DORY involves reconstructing drafts from outputs, refining these with hints, and filtering out noise based on uncertainty. Our evaluation across diverse LLMs and prompt benchmarks shows that DORY outperforms existing baselines, improving performance by approximately 10.82% and establishing a new state-of-the-art record in prompt recovery tasks. Significantly, DORY operates using a single LLM without any external resources or model, offering a cost-effective, user-friendly prompt recovery solution.

Ask Lune

Know this paper in minutes, not an afternoon.

Lune can read every page of it and the top-tier papers that cite it, and gives you its claims, its evidence and its limits, each one cited to the paper it comes from. No skimming, no AI slop.

4 top-tier papers have cited it since. Know what they found before your reviewers do.

Questions to start from

Lune can draw on

the full paper · the top-tier papers that cite it · related work

Get the answer

Free to start. No credit card required.

Lineage

  • PROMPRINT: Prompt Fingerprinting via First-Token Response for LLM App Cloning Detection cites On the Effectiveness of Prompt Stealing Attacks on In-the-Wild Prompts
  • On the Effectiveness of Prompt Stealing Attacks on In-the-Wild Prompts cites Language Models are Few-Shot Learners
  • Jailbroken: How Does LLM Safety Training Fail? cites Language Models are Few-Shot Learners
  • Prompt Inference Attack on Distributed Large Language Model Inference Frameworks cites Language Models are Few-Shot Learners
Builds onCited byLeft to right by year; the more cited, the higher.

Cited by top-tier papers4

  1. Better Language Model Inversion by Compactly Representing Next-Token DistributionsMurtaza Nazir, Matthew Finlayson, John X. Morris, Xiang Ren et al.NeurIPS 2025 · 13 citations
  2. PROMPRINT: Prompt Fingerprinting via First-Token Response for LLM App Cloning DetectionJungmin Lee, Peizhuo Lv, Yeonjoon LeeACL 2026
  3. On the Effectiveness of Prompt Stealing Attacks on In-the-Wild PromptsYicong Tan, Xinyue Shen, Yun Shen, Michael Backes et al.S&P 2025
  4. Prompt Inference Attack on Distributed Large Language Model Inference FrameworksXinjian Luo, Ting Yu, Xiaokui XiaoCCS 2025

Builds on21

  1. Language Models are Few-Shot LearnersTom B. Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah et al.NeurIPS 2020 · 64,255 citations
  2. SimCSE: Simple Contrastive Learning of Sentence EmbeddingsTianyu Gao, Xingcheng Yao, Danqi ChenEMNLP 2021 · 2,496 citations
  3. Jailbroken: How Does LLM Safety Training Fail?Alexander Wei, Nika Haghtalab, Jacob SteinhardtNeurIPS 2023 · 2,230 citations
  4. Stealing Machine Learning Models via Prediction APIsFlorian Tramèr, Fan Zhang, Ari Juels, Michael K. Reiter et al.USENIX Security 2016 · 2,088 citations
  5. Uncertainty Estimation in Autoregressive Structured PredictionAndrey Malinin, Mark J. F. GalesICLR 2021 · 439 citations

Related papers

  1. REALM: Recursive Relevance Modeling for LLM-based Document Re-RankingPinhuan Wang, Zhiqiu Xia, Chunhua Liao, Feiyi Wang et al.EMNLP 2025
  2. Reflect Then Learn: Active Prompting for Information Extraction Guided by Introspective ConfusionDong Zhao, Yadong Wang, Xiang Chen, Chenxi Wang et al.AAAI 2026
  3. Prompting the Unknown: Understanding Response Uncertainty in Large Language ModelsZe Yu Zhang, Arun Verma, Finale Doshi-Velez, Bryan Kian Hsiang LowACL Findings 2026 · 3 citations
  4. Textual Bayes: Quantifying Prompt Uncertainty in LLM-Based SystemsBrendan Leigh Ross, Noël Vouitsis, Atiyeh Ashari Ghomi, Rasa Hosseinzadeh et al.ICLR 2026 · 8 citations
  5. Selecting Better Samples from Pre-trained LLMs: A Case Study on Question GenerationXingdi Yuan, Tong Wang, Yen-Hsiang Wang, Emery Fine et al.ACL Findings 2023 · 21 citations
Dusk over the sea between two cliffs drawn in fine vertical lines