PGS: Effective LLM Code Refinement via Property-Oriented and Structurally Minimal Feedback
Lehan He, Zeren Chen, Zhe Zhang, Xiang Gao, Lu Sheng
摘要
Large Language Models (LLMs) excel at code generation, yet ensuring the functional correctness of their outputs remains a persistent challenge. Recent studies have applied Test-Driven Development (TDD) to refine code, leveraging execution feedback to guide the model toward correct solutions. However, such feedback is often noisy and uninformative, stemming from the scarcity of high-quality test cases and the abundance of noisy, auto-generated ones. In this work, we shift the focus from test-case generation to feedback quality. We introduce the Property-Generated Solver (PGS), a novel feedback-centric framework designed to generate highly effective feedback via two principles: it must provide semantic guidance beyond simple I/O mismatches through property validation, and be structurally minimal, to reduce cognitive load and isolate root causes. PGS operates by checking high-level program properties (e.g., a sorting function must produce a non-decreasing sequence) then providing the simplest failing counterexample to the LLM. This property-driven, minimal feedback steers LLMs toward correct and generalizable solutions. Across diverse benchmarks, PGS demonstrates superior performance, achieving a bug fix rate 1.4x-1.6x higher than the strongest debugging-based approaches and establishing a new state-of-the-art in automated code refinement.
问问这篇 Paper
智能体会读完全文。
Lune 把这篇 Paper 索引到了每一个公式,引用它的顶会 Paper 也一样。你提问,回答直接引用原文。
它引用的顶会 Paper13
- Chain-of-Thought Prompting Elicits Reasoning in Large Language ModelsJason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma 等NeurIPS 2022 · 被引用 22,562 次
- Reflexion: language agents with verbal reinforcement learningNoah Shinn, Federico Cassano, Ashwin Gopinath, Karthik Narasimhan 等NeurIPS 2023 · 被引用 5,828 次
- Is Your Code Generated by ChatGPT Really Correct? Rigorous Evaluation of Large Language Models for Code GenerationJiawei Liu, Chunqiu Steven Xia, Yuyao Wang, Lingming ZhangNeurIPS 2023 · 被引用 2,317 次
- SWE-bench: Can Language Models Resolve Real-world Github Issues?Carlos E. Jimenez, John Yang, Alexander Wettig, Shunyu Yao 等ICLR 2024 · 被引用 2,082 次
- Teaching Large Language Models to Self-DebugXinyun Chen, Maxwell Lin, Nathanael Schärli, Denny ZhouICLR 2024 · 被引用 1,085 次
相关 Paper
- Revisit Self-Debugging with Self-Generated Tests for Code GenerationXiancai Chen, Zhengwei Tao, Kechi Zhang, Changzhi Zhou 等ACL 2025
- Test-Driven Development and LLM-based Code GenerationNoble Saji Mathews, Meiyappan NagappanASE 2024 · 被引用 15 次
- Learner-Tailored Program Repair: A Solution Generator with Iterative Edit-Driven Retrieval EnhancementZhenlong Dai, Zhuoluo Zhao, Hengning Wang, Xiu Tang 等AAAI 2026
- SeDev: Structured Semantic Exploration for LLM-Driven Code GenerationRonghui Yang, Jie Liu, Jiajie Zeng, Jiexin Wang 等ACL 2026
- Evaluating and Mitigating the Misguidance Effect of Buggy Code in LLM-Generated Unit TestsJunda Zhao, Shurui Zhou, Eldan CohenISSTA 2026 · 被引用 1 次
