EMNLP Findings2023
Intervention-Based Alignment of Code Search with Execution Feedback
Hojae Han, Minsoo Kim, Seung-won Hwang, Nan Duan, Shuai Lu
Abstract
One of the fundamental goals in code search is to retrieve a functionally correct code for a given natural language query.As annotating for correctness requires executing test cases (i.e.obtaining execution feedback), existing code search training datasets approximate text-code co-occurrences as positive execution feedback.However, this approximation may misalign models' retrieval decisions from ground-truth correctness.To address such limitation, we propose Code Intervention-based Reinforcement Learning (CIRL) that perturbs training code to result in misalignment (i.e.code intervention), then tests models' decisions and corrects them with the execution feedback by reinforcement learning.The first technical contribution of CIRL is to induce the execution feedback from perturbation, without actual execution.Secondly, CIRL introduces structural perturbations using abstract syntax trees, going beyond simple lexical changes.Experimental results on various datasets demonstrate the effectiveness of CIRL compared to conventional approaches.Codeforces Codeforces CodeNet CodeNet CodeNet CodeNet CodeNet CodeNet CodeNet CodeNet CodeNet CodeNet
Ask Lune
Know this paper in minutes, not an afternoon.
Lune can read every page of it and the top-tier papers that cite it, and gives you its claims, its evidence and its limits, each one cited to the paper it comes from. No skimming, no AI slop.
Lineage
Builds on21
- CodeBERT: A Pre-Trained Model for Programming and Natural LanguagesZhangyin Feng, Daya Guo, Duyu Tang, Nan Duan et al.EMNLP Findings 2020 · 2,828 citations
- GraphCodeBERT: Pre-training Code Representations with Data FlowDaya Guo, Shuo Ren, Shuai Lu, Zhangyin Feng et al.ICLR 2021 · 1,644 citations
- Learning The Difference That Makes A Difference With Counterfactually-Augmented DataDivyansh Kaushik, Eduard H. Hovy, Zachary Chase LiptonICLR 2020 · 625 citations
- CodeRL: Mastering Code Generation through Pretrained Models and Deep Reinforcement LearningHung Le, Yue Wang, Akhilesh Deepak Gotmare, Silvio Savarese et al.NeurIPS 2022 · 571 citations
- Grounding Large Language Models in Interactive Environments with Online Reinforcement LearningThomas Carta, Clément Romac, Thomas Wolf, Sylvain Lamprier et al.ICML 2023 · 258 citations
Related papers
- CODERL+: Improving Code Generation via Reinforcement with Execution Semantics AlignmentXue Jiang, Yihong Dong, Mengyang Liu, Hongyi Deng et al.ACL 2026 · 18 citations
- RLEF: Grounding Code LLMs in Execution Feedback with Reinforcement LearningJonas Gehring, Kunhao Zheng, Jade Copet, Vegard Mella et al.ICML 2025
- CodeRM-NT: Reward Model for Code RL without Unit TestsXiao Xia, Dan Zhang, Tianrui SunACL Findings 2026
- Automatic Unit Test Data Generation and Actor-Critic Reinforcement Learning for Code SynthesisPhilip John Gorinski, Matthieu Zimmer, Gerasimos Lampouras, Derrick-Goh-Xin Deik et al.EMNLP Findings 2023 · 1 citation
- IRCoCo: Immediate Rewards-Guided Deep Reinforcement Learning for Code CompletionBolun Li, Zhihong Sun, Tao Huang, Hongyu Zhang et al.FSE 2024 · 10 citations
