EMNLP Findings2024
Learning to Paraphrase for Alignment with LLM Preference
Junbo Fu, Guoshuai Zhao, Yimin Deng, Yunqi Mi, Xueming Qian
Abstract
Large Language Models (LLMs) exhibit the issue of paraphrase divergence. This means that when a question is phrased in a slightly different but semantically similar way, LLM may output a wrong response despite being able to answer the original question correctly. Previous research has regarded this issue as a problem of the model's robustness to question paraphrase and proposed a retraining method to address it. However, retraining faces challenges in meeting the computational costs and privacy security demands of LLMs. In this paper, we regard this issue as a problem of alignment with model preferences and propose PEARL (Preference-drivEn pAraphRase Learning). This is a black-box method that enhances model performance by paraphrasing questions in expressions preferred by the model. We validate PEARL across six datasets spanning three tasks: open-domain QA, commonsense reasoning, and math word problem. Extensive experiments demonstrated not only the outstanding performance but also the composability, transferability, and immense potential of PEARL, shedding new light on the blackbox tuning of LLMs 1 .
Ask Lune
Know this paper in minutes, not an afternoon.
Lune can read every page of it and the top-tier papers that cite it, and gives you its claims, its evidence and its limits, each one cited to the paper it comes from. No skimming, no AI slop.
3 top-tier papers have cited it since. Know what they found before your reviewers do.
Lune can draw on
the full paper · the top-tier papers that cite it · related work
Free to start. No credit card required.
Lineage
Cited by top-tier papers3
- Flip-Flop Consistency: Unsupervised Training for Robustness to Prompt Perturbations in LLMsParsa Hejabi, Elnaz Rahmati, Alireza Salkhordeh Ziabari, Morteza DehghaniACL 2026 · 1 citation
- StoryCoder: Narrative Reformulation for Structured Reasoning in LLM Code GenerationGeonhui Jang, Dongyoon Han, Youngjoon YooACL 2026
- Reasoning Compartmentalization: Bridging the Concretization Gap via Abstraction-based RoutingLing-I Wu, Jian Tong, Yu Sun, Xuan Gao et al.ICML 2026
Builds on15
- LoRA: Low-Rank Adaptation of Large Language ModelsEdward J. Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu et al.ICLR 2022 · 18,833 citations
- Finetuned Language Models are Zero-Shot LearnersJason Wei, Maarten Bosma, Vincent Y. Zhao, Kelvin Guu et al.ICLR 2022 · 4,966 citations
- Crosslingual Generalization through Multitask FinetuningNiklas Muennighoff, Thomas Wang, Lintang Sutawika, Adam Roberts et al.ACL 2023 · 319 citations
- Compositional Exemplars for In-context LearningJiacheng Ye, Zhiyong Wu, Jiangtao Feng, Tao Yu et al.ICML 2023 · 188 citations
- BLEURT: Learning Robust Metrics for Text GenerationThibault Sellam, Dipanjan Das, Ankur P. ParikhACL 2020 · 40 citations
Related papers
- Putting People in LLMs' Shoes: Generating Better Answers via Question RewriterJunhao Chen, Bowen Wang, Zhouqiang Jiang, Yuta NakashimaAAAI 2025 · 5 citations
- Latent Paraphrasing: Perturbation on Layers Improves Knowledge Injection in Language ModelsMinki Kang, Sung Ju Hwang, Gibbeum Lee, Jaewoong ChoNeurIPS 2024 · 3 citations
- Mapping from Meaning: Addressing the Miscalibration of Prompt-Sensitive Language ModelsKyle Cox, Jiawei Xu, Yikun Han, Rong Xu et al.AAAI 2025 · 6 citations
- PEARL: Towards Permutation-Resilient LLMsLiang Chen, Li Shen, Yang Deng, Xiaoyan Zhao et al.ICLR 2025
- PURE: Aligning LLM via Pluggable Query Reformulation for Enhanced HelpfulnessWenjin Yao, Yidong Wang, Zhuohao Yu, Rui Xie et al.EMNLP Findings 2024 · 2 citations
