ACL Findings2021
Disentangled Code Representation Learning for Multiple Programming Languages
Jingfeng Zhang, Haiwen Hong, Yin Zhang, Yao Wan, Ye Liu, Yulei Sui
Abstract
Developing effective distributed representations of source code is fundamental yet challenging for many software engineering tasks such as code clone detection, code search, code translation and transformation. However, current code embedding approaches that represent the semantic and syntax of code in a mixed way are less interpretable and the resulting embedding can not be easily generalized across programming languages. In this paper, we propose a disentangled code representation learning approach to separate the semantic from the syntax of source code under a multi-programming-language setting, obtaining better interpretability and generalizability. Specially, we design three losses dedicated to the characteristics of source code to enforce the disentanglement effectively. We conduct comprehensive experiments on a real-world dataset composed of programming exercises implemented by multiple solutions that are semantically identical but grammatically distinguished. The experimental results validate the superiority of our proposed disentangled code representation, compared to several baselines, across three types of downstream tasks, i.e., code clone detection, code translation, and code-to-code search.
Ask Lune
Know this paper in minutes, not an afternoon.
Lune can read every page of it and the top-tier papers that cite it, and gives you its claims, its evidence and its limits, each one cited to the paper it comes from. No skimming, no AI slop.
2 top-tier papers have cited it since. Know what they found before your reviewers do.
Lune can draw on
the full paper · the top-tier papers that cite it · related work
Free to start. No credit card required.
Lineage
- ObscuraCoder: Powering Efficient Code LM Pre-Training Via Obfuscation Grounding cites GraphCodeBERT: Pre-training Code Representations with Data Flow
- CodeBERT: A Pre-Trained Model for Programming and Natural Languages cites Learning and Evaluating Contextual Embedding of Source Code
- Learning and Evaluating Contextual Embedding of Source Code cites CodeBERT: A Pre-Trained Model for Programming and Natural Languages
- An Empirical Comparison of Pre-Trained Models of Source Code cites CodeBERT: A Pre-Trained Model for Programming and Natural Languages
- An Empirical Comparison of Pre-Trained Models of Source Code cites GraphCodeBERT: Pre-training Code Representations with Data Flow
- An Empirical Comparison of Pre-Trained Models of Source Code cites Learning and Evaluating Contextual Embedding of Source Code
Cited by top-tier papers2
Builds on6
- CodeBERT: A Pre-Trained Model for Programming and Natural LanguagesZhangyin Feng, Daya Guo, Duyu Tang, Nan Duan et al.EMNLP Findings 2020 · 2,828 citations
- GraphCodeBERT: Pre-training Code Representations with Data FlowDaya Guo, Shuo Ren, Shuai Lu, Zhangyin Feng et al.ICLR 2021 · 1,644 citations
- Learning and Evaluating Contextual Embedding of Source CodeAditya Kanade, Petros Maniatis, Gogul Balakrishnan, Kensen ShiICML 2020 · 438 citations
- Flow2Vec: value-flow-based precise code embeddingYulei Sui, Xiao Cheng, Guanqin Zhang, Haoyu WangOOPSLA 2020 · 94 citations
- Probing Brain Activation Patterns by Dissociating Semantics and Syntax in SentencesShaonan Wang, Jiajun Zhang, Nan Lin, Chengqing ZongAAAI 2020 · 23 citations
Related papers
- MetaTPTrans: A Meta Learning Approach for Multilingual Code Representation LearningWeiguo Pian, Hanyu Peng, Xunzhu Tang, Tiezhu Sun et al.AAAI 2023 · 18 citations
- Functional code clone detection with syntax and semantics fusion learningChunrong Fang, Zixi Liu, Yangyang Shi, Jeff Huang et al.ISSTA 2020 · 125 citations
- Uncertainty-Aware Contrastive Learning with Hard Negative Sampling for Code Search TasksHan Liu, Jiaqing Zhan, Qin ZhangAAAI 2025 · 1 citation
- CONCORD: Clone-Aware Contrastive Learning for Source CodeYangruibo Ding, Saikat Chakraborty, Luca Buratti, Saurabh Pujar et al.ISSTA 2023 · 6 citations
- The Struggles of LLMs in Cross-Lingual Code Clone DetectionMicheline Bénédicte Moumoula, Abdoul Kader Kaboré, Jacques Klein, Tegawendé F. BissyandéFSE 2025 · 4 citations
