ACL Findings2024
DORY: Deliberative Prompt Recovery for LLM
Abstract
Prompt recovery in large language models (LLMs) is crucial for understanding how LLMs work and addressing concerns regarding privacy, copyright, etc. The trend towards inference-only APIs complicates this task by restricting access to essential outputs for recovery. To tackle this challenge, we extract promptrelated information from limited outputs and identify a strong(negative) correlation between output probability-based uncertainty and the success of prompt recovery. This finding led to the development of Deliberative PrOmpt RecoverY (DORY), our novel approach that leverages uncertainty to recover prompts accurately. DORY involves reconstructing drafts from outputs, refining these with hints, and filtering out noise based on uncertainty. Our evaluation across diverse LLMs and prompt benchmarks shows that DORY outperforms existing baselines, improving performance by approximately 10.82% and establishing a new state-of-the-art record in prompt recovery tasks. Significantly, DORY operates using a single LLM without any external resources or model, offering a cost-effective, user-friendly prompt recovery solution.
Ask Lune
Know this paper in minutes, not an afternoon.
Lune can read every page of it and the top-tier papers that cite it, and gives you its claims, its evidence and its limits, each one cited to the paper it comes from. No skimming, no AI slop.
4 top-tier papers have cited it since. Know what they found before your reviewers do.
Lune can draw on
the full paper · the top-tier papers that cite it · related work
Free to start. No credit card required.
Lineage
- PROMPRINT: Prompt Fingerprinting via First-Token Response for LLM App Cloning Detection cites On the Effectiveness of Prompt Stealing Attacks on In-the-Wild Prompts
- On the Effectiveness of Prompt Stealing Attacks on In-the-Wild Prompts cites Language Models are Few-Shot Learners
- Jailbroken: How Does LLM Safety Training Fail? cites Language Models are Few-Shot Learners
- Prompt Inference Attack on Distributed Large Language Model Inference Frameworks cites Language Models are Few-Shot Learners
Cited by top-tier papers4
- Better Language Model Inversion by Compactly Representing Next-Token DistributionsMurtaza Nazir, Matthew Finlayson, John X. Morris, Xiang Ren et al.NeurIPS 2025 · 13 citations
- PROMPRINT: Prompt Fingerprinting via First-Token Response for LLM App Cloning DetectionJungmin Lee, Peizhuo Lv, Yeonjoon LeeACL 2026
- On the Effectiveness of Prompt Stealing Attacks on In-the-Wild PromptsYicong Tan, Xinyue Shen, Yun Shen, Michael Backes et al.S&P 2025
- Prompt Inference Attack on Distributed Large Language Model Inference FrameworksXinjian Luo, Ting Yu, Xiaokui XiaoCCS 2025
Builds on21
- Language Models are Few-Shot LearnersTom B. Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah et al.NeurIPS 2020 · 64,255 citations
- SimCSE: Simple Contrastive Learning of Sentence EmbeddingsTianyu Gao, Xingcheng Yao, Danqi ChenEMNLP 2021 · 2,496 citations
- Jailbroken: How Does LLM Safety Training Fail?Alexander Wei, Nika Haghtalab, Jacob SteinhardtNeurIPS 2023 · 2,230 citations
- Stealing Machine Learning Models via Prediction APIsFlorian Tramèr, Fan Zhang, Ari Juels, Michael K. Reiter et al.USENIX Security 2016 · 2,088 citations
- Uncertainty Estimation in Autoregressive Structured PredictionAndrey Malinin, Mark J. F. GalesICLR 2021 · 439 citations
Related papers
- REALM: Recursive Relevance Modeling for LLM-based Document Re-RankingPinhuan Wang, Zhiqiu Xia, Chunhua Liao, Feiyi Wang et al.EMNLP 2025
- Reflect Then Learn: Active Prompting for Information Extraction Guided by Introspective ConfusionDong Zhao, Yadong Wang, Xiang Chen, Chenxi Wang et al.AAAI 2026
- Prompting the Unknown: Understanding Response Uncertainty in Large Language ModelsZe Yu Zhang, Arun Verma, Finale Doshi-Velez, Bryan Kian Hsiang LowACL Findings 2026 · 3 citations
- Textual Bayes: Quantifying Prompt Uncertainty in LLM-Based SystemsBrendan Leigh Ross, Noël Vouitsis, Atiyeh Ashari Ghomi, Rasa Hosseinzadeh et al.ICLR 2026 · 8 citations
- Selecting Better Samples from Pre-trained LLMs: A Case Study on Question GenerationXingdi Yuan, Tong Wang, Yen-Hsiang Wang, Emery Fine et al.ACL Findings 2023 · 21 citations
