Lune

EMNLP Findings2024

Rethinking Code Refinement: Learning to Judge Code Efficiency

Minju Seo, Jinheon Baek, Sung Ju Hwang

Abstract

Large Language Models (LLMs) have demonstrated impressive capabilities in understanding and generating codes. Due to these capabilities, many recent methods are proposed to automatically refine the codes with LLMs. However, we should rethink that the refined codes (from LLMs and even humans) are not always more efficient than their original versions. On the other hand, running two different versions of codes and comparing them every time is not ideal and time-consuming. Therefore, in this work, we propose a novel method based on the code language model that is trained to judge the efficiency between two different codes (generated across humans and machines) by either classifying the superior one or predicting the relative improvement. We validate our method on multiple programming languages with multiple refinement steps, demonstrating that the proposed method can effectively distinguish between more and less efficient versions of code.

Ask Lune

Know this paper in minutes, not an afternoon.

Lune can read every page of it and the top-tier papers that cite it, and gives you its claims, its evidence and its limits, each one cited to the paper it comes from. No skimming, no AI slop.

Questions to start from

Lune can draw on

the full paper · related work

Get the answer

Free to start. No credit card required.

Lineage

  • CodeGen: An Open Large Language Model for Code with Multi-Turn Program Synthesis cites Chain-of-Thought Prompting Elicits Reasoning in Large Language Models
  • Learning Performance-Improving Code Edits cites CodeGen: An Open Large Language Model for Code with Multi-Turn Program Synthesis
  • Learning Performance-Improving Code Edits cites Chain-of-Thought Prompting Elicits Reasoning in Large Language Models
Builds onCited byLeft to right by year; the more cited, the higher.

Builds on5

  1. Chain-of-Thought Prompting Elicits Reasoning in Large Language ModelsJason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma et al.NeurIPS 2022 · 22,562 citations
  2. Unsupervised Translation of Programming LanguagesBaptiste Rozière, Marie-Anne Lachaux, Lowik Chanussot, Guillaume LampleNeurIPS 2020 · 606 citations
  3. CodeGen: An Open Large Language Model for Code with Multi-Turn Program SynthesisErik Nijkamp, Bo Pang, Hiroaki Hayashi, Lifu Tu et al.ICLR 2023 · 234 citations
  4. Learning Performance-Improving Code EditsAlexander Shypula, Aman Madaan, Yimeng Zeng, Uri Alon et al.ICLR 2024 · 141 citations
  5. CodeBERTScore: Evaluating Code Generation with Pretrained Models of CodeShuyan Zhou, Uri Alon, Sumit Agarwal, Graham NeubigEMNLP 2023 · 77 citations

Related papers

  1. More Than Just Functional: LLM-as-a-Critique for Efficient Code GenerationDerui Zhu, Dingfan Chen, Jinfu Chen, Jens Grossklags et al.NeurIPS 2025 · 2 citations
  2. Generating Energy-Efficient Code via Large-Language Models - Where are we now?Radu Apsan, Vincenzo Stoico, Michel Albonico, Rudra Dhar et al.ICSE 2026
  3. ECCO: Can We Improve Model-Generated Code Efficiency Without Sacrificing Functional Correctness?Siddhant Waghjale, Vishruth Veerendranath, Zhiruo Wang, Daniel FriedEMNLP 2024 · 3 citations
  4. CYCLE: Learning to Self-Refine the Code GenerationYangruibo Ding, Marcus J. Min, Gail E. Kaiser, Baishakhi RayOOPSLA 2024 · 35 citations
  5. Self-Edit: Fault-Aware Code Editor for Code GenerationKechi Zhang, Zhuo Li, Jia Li, Ge Li et al.ACL 2023 · 42 citations
Dusk over the sea between two cliffs drawn in fine vertical lines