ACL Findings2026
RepoDistill: Distilling Repository Knowledge through Compression-Aware Budget Allocation and Policy Optimization
Xin Yin, Zixiang Ding, Yiang Zhang, Qiang Wang, Rui Wang, Chao Ni, Zhe Cui
Abstract
Large Language Models (LLMs) have achieved strong performance on many code-related tasks, yet they still struggle with repository-level scenarios where reasoning depends on long, noisy, and structurally complex contexts. While existing retrieval methods, including both similaritybased and graph-based approaches, can identify relevant code snippets, they often retrieve excessive contexts that intensify the "lost-inthe-middle" phenomenon and dilute model attention with redundant contexts. To address this, we present RepoDistill, a novel framework that integrates retrieval with learned budget allocation for fine-grained context compression. RepoDistill first employs a plug-andplay lightweight GraphRAG to retrieve context that follows logical flows. It then applies Compression-Aware Budget Allocation guided by Compression-Aware Policy Optimization, which formulates context management as a multi-step decision problem and learns allocation policies for contexts. Experiments show that RepoDistill outperforms baselines, achieving gains of up to +7.00 on SWE-QA, +24.4% on CoderEval, and +0.25 on Long-CodeU. Furthermore, a compact 4B-parameter model trained with RepoDistill can serve as an effective context compressor for closed-source LLMs, reducing input tokens by up to 66% while maintaining comparable performance. We release our code at https://anonymous. 4open.science/r/RepoDistill-6CE3/.
Ask Lune
Know this paper in minutes, not an afternoon.
Lune can read every page of it and the top-tier papers that cite it, and gives you its claims, its evidence and its limits, each one cited to the paper it comes from. No skimming, no AI slop.
Lineage
- Efficient Streaming Language Models with Attention Sinks cites YaRN: Efficient Context Window Extension of Large Language Models
- DAPO: An Open-Source LLM Reinforcement Learning System at Scale cites Training language models to follow instructions with human feedback
- Group-in-Group Policy Optimization for LLM Agent Training cites DAPO: An Open-Source LLM Reinforcement Learning System at Scale
- Group-in-Group Policy Optimization for LLM Agent Training cites Training language models to follow instructions with human feedback
Builds on23
- Training language models to follow instructions with human feedbackLong Ouyang, Jeffrey Wu, Xu Jiang, Diogo Almeida et al.NeurIPS 2022 · 24,707 citations
- DAPO: An Open-Source LLM Reinforcement Learning System at ScaleQiying Yu, Zheng Zhang, Ruofei Zhu, Yufeng Yuan et al.NeurIPS 2025 · 2,828 citations
- Efficient Streaming Language Models with Attention SinksGuangxuan Xiao, Yuandong Tian, Beidi Chen, Song Han et al.ICLR 2024 · 1,714 citations
- YaRN: Efficient Context Window Extension of Large Language ModelsBowen Peng, Jeffrey Quesnelle, Honglu Fan, Enrico ShippoleICLR 2024 · 508 citations
- Group-in-Group Policy Optimization for LLM Agent TrainingLang Feng, Zhenghai Xue, Tingcong Liu, Bo AnNeurIPS 2025 · 484 citations
Related papers
- LongCodeZip: Compress Long Context for Code Language ModelsYuling Shi, Yichun Qian, Hongyu Zhang, Beijun Shen et al.ASE 2025
- Repoformer: Selective Retrieval for Repository-Level Code CompletionDi Wu, Wasi Uddin Ahmad, Dejiao Zhang, Murali Krishna Ramanathan et al.ICML 2024 · 78 citations
- GraphCoder: Enhancing Repository-Level Code Completion via Coarse-to-fine Retrieval Based on Code Context GraphWei Liu, Ailun Yu, Daoguang Zan, Bo Shen et al.ASE 2024 · 11 citations
- Less Is More: Elevating RAG via Performance-Driven Context CompressionZiqiang Cui, Yunpeng Weng, Xing Tang, Peiyang Liu et al.ICML 2026
- AlignCoder: Aligning Retrieval with Target Intent for Repository-Level Code CompletionTianyue Jiang, Yanlin Wang, Yanli Wang, Daya Guo et al.ASE 2025 · 2 citations
