EMNLP Findings2025
Hierarchical Reward Modeling for Fault Localization in Large Code Repositories
Jiwei Zhang, Jianxun Lian, Haiming Qin, Mingyang Zhou, Kezhong Lu, Rui Mao, Hao Liao
Abstract
Large Language Models (LLMs) exhibit significant potential in complex software engineering tasks, however, their fault localization capabilities within repository are constrained by inherent limitations in max context length. Although Test-Time Scaling (TTS) can generate multiple candidate solutions, traditional selection strategies often fail to identify the optimal one. To solve this problem, we introduces Hierarchical Localization Reward Model (HiLoRM), which specifically designed to evaluate and select the most accurate fault localization candidates (at file, function, and line levels) from the multiple sampled outputs of LLMs, thereby enhancing localization accuracy. Furthermore, we constructed the HiFL-44k dataset, comprising approximately 44,000 fault localization instances, to train HiLoRM. Experimental results demonstrate that on the SWE-Bench-Lite dataset, HiLoRM improves the final line-level localization recall by 12% compared to a baseline model that does not use a reward model. Concurrently, HiLoRM exhibits a strong capability to evaluate predictions from larger LLMs (e.g., 32B parameters) and demonstrates transferability and generalization potential when applied to other fault localization methods. This work provides an effective methodology and an accessible model to significantly improve the accuracy and reliability of LLMs for repository-level fault localization. Our codes and dataset are available at https://github.com/SZU-ZJW/HiFL-Method
Ask Lune
Know this paper in minutes, not an afternoon.
Lune can read every page of it and the top-tier papers that cite it, and gives you its claims, its evidence and its limits, each one cited to the paper it comes from. No skimming, no AI slop.
Lineage
- MAGIS: LLM-Based Multi-Agent Framework for GitHub Issue Resolution cites SWE-bench: Can Language Models Resolve Real-world Github Issues?
- MAGIS: LLM-Based Multi-Agent Framework for GitHub Issue Resolution cites AutoCodeRover: Autonomous Program Improvement
- AutoCodeRover: Autonomous Program Improvement cites SWE-bench: Can Language Models Resolve Real-world Github Issues?
- SoRFT: Issue Resolving with Subtask-oriented Reinforced Fine-Tuning cites SWE-bench: Can Language Models Resolve Real-world Github Issues?
Builds on9
- SWE-bench: Can Language Models Resolve Real-world Github Issues?Carlos E. Jimenez, John Yang, Alexander Wettig, Shunyu Yao et al.ICLR 2024 · 2,082 citations
- MAGIS: LLM-Based Multi-Agent Framework for GitHub Issue ResolutionWei Tao, Yucheng Zhou, Yanlin Wang, Wenqiang Zhang et al.NeurIPS 2024 · 210 citations
- Large Language Models for Test-Free Fault LocalizationAidan Z. H. Yang, Claire Le Goues, Ruben Martins, Vincent J. HellendoornICSE 2024 · 98 citations
- AutoCodeRover: Autonomous Program ImprovementYuntong Zhang, Haifeng Ruan, Zhiyu Fan, Abhik RoychoudhuryISSTA 2024 · 96 citations
- SoRFT: Issue Resolving with Subtask-oriented Reinforced Fine-TuningZexiong Ma, Chao Peng, Pengfei Gao, Xiangxin Meng et al.ACL 2025 · 25 citations
Related papers
- SWE-Swiss: A Multi-Task Fine-Tuning and RL Recipe for High-Performance Issue ResolutionZhenyu He, Qingping Yang, Wei Shen, Xiaojian Zhong et al.ACL Findings 2026
- A Quantitative and Qualitative Evaluation of LLM-Based Explainable Fault LocalizationSungmin Kang, Gabin An, Shin YooFSE 2024 · 69 citations
- SWE-Fixer: Training Open-Source LLMs for Effective and Efficient GitHub Issue ResolutionChengxing Xie, Bowen Li, Chang Gao, He Du et al.ACL Findings 2025 · 10 citations
- FEA-Bench: A Benchmark for Evaluating Repository-Level Code Generation for Feature ImplementationWei Li, Xin Zhang, Zhongxin Guo, Shaoguang Mao et al.ACL 2025 · 40 citations
- Towards Explorative IRBL: Combining Semantic Retrieval with LLM-Driven Iterative Code ExplorationMoumita Asad, Rafed Muhammad Yasir, Sam MalekISSTA 2026
