ACL Findings2026
Conjecture and Inquiry: Quantifying Software Performance Requirements via Interactive Retrieval-Augmented Preference Elicitation
Shihai Wang, Tao Chen
Abstract
Since software performance requirements are documented in natural language, quantifying them into mathematical forms is essential for software engineering. Yet, the vagueness in performance requirements and uncertainty of human cognition have caused highly uncertain ambiguity in the interpretations, rendering their automated quantification an unaddressed and challenging problem. In this paper, we formalize the problem and propose IRAP, an approach that quantifies performance requirements into mathematical functions via interactive retrieval-augmented preference elicitation.
IRAP differs from the others in that it explicitly derives from problem-specific knowledge to retrieve and reason the preferences, which also guides the progressive interaction with stakeholders, while reducing the cognitive overhead. Experiment results against 10 state-of-the-art methods on four real-world datasets demonstrate the superiority of IRAP on all cases with up to 40× improvements under as few as five rounds of interactions.
Ask Lune
Know this paper in minutes, not an afternoon.
Lune can read every page of it and the top-tier papers that cite it, and gives you its claims, its evidence and its limits, each one cited to the paper it comes from. No skimming, no AI slop.
Lineage
- Improving Language Models by Retrieving from Trillions of Tokens cites Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks
- Direct Preference Optimization: Your Language Model is Secretly a Reward Model cites Training language models to follow instructions with human feedback
Builds on16
- Training language models to follow instructions with human feedbackLong Ouyang, Jeffrey Wu, Xu Jiang, Diogo Almeida et al.NeurIPS 2022 · 24,707 citations
- Retrieval-Augmented Generation for Knowledge-Intensive NLP TasksPatrick Lewis, Ethan Perez, Aleksandra Piktus, Fabio Petroni et al.NeurIPS 2020 · 19,162 citations
- LoRA: Low-Rank Adaptation of Large Language ModelsEdward J. Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu et al.ICLR 2022 · 18,833 citations
- Direct Preference Optimization: Your Language Model is Secretly a Reward ModelRafael Rafailov, Archit Sharma, Eric Mitchell, Christopher D. Manning et al.NeurIPS 2023 · 10,924 citations
- Improving Language Models by Retrieving from Trillions of TokensSebastian Borgeaud, Arthur Mensch, Jordan Hoffmann, Trevor Cai et al.ICML 2022 · 1,629 citations
Related papers
- Light over Heavy: Automated Performance Requirements Quantification with Linguistic InducementShihai Wang, Tao ChenICSE 2026
- Using Domain-specific Corpora for Improved Handling of Ambiguity in RequirementsSaad Ezzini, Sallam Abualhaija, Chetan Arora, Mehrdad Sabetzadeh et al.ICSE 2021 · 51 citations
- Automatic Extraction of Cause-Effect-Relations from Requirements ArtifactsJulian Frattini, Maximilian Junker, Michael Unterkalmsteiner, Daniel MéndezASE 2020 · 13 citations
- RPO: Retrieval Preference Optimization for Robust Retrieval-Augmented GenerationShi-Qi Yan, Quan Liu, Zhen-Hua LingACL 2025 · 4 citations
- CoQuIR: A Comprehensive Benchmark for Code Quality-Aware Information RetrievalJiahui Geng, Fengyu Cai, Shaobo Cui, Qing Li et al.ACL 2026 · 3 citations
