ACL Findings2021
Code Summarization with Structure-induced Transformer
Abstract
Code summarization (CS) is becoming a promising area in recent language understanding, which aims to generate sensible human language automatically for programming language in the format of source code, serving in the most convenience of programmer developing. It is well known that programming languages are highly structured. Thus previous works attempt to apply structurebased traversal (SBT) or non-sequential models like Tree-LSTM and graph neural network (GNN) to learn structural program semantics. However, it is surprising that incorporating SBT into advanced encoder like Transformer instead of LSTM has been shown no performance gain, which lets GNN become the only rest means modeling such necessary structural clue in source code. To release such inconvenience, we propose structureinduced Transformer, which encodes sequential code inputs with multi-view structural clues in terms of a newly-proposed structureinduced self-attention mechanism. Extensive experiments show that our proposed structureinduced Transformer helps achieve new stateof-the-art results on benchmarks.
Ask Lune
Know this paper in minutes, not an afternoon.
Lune can read every page of it and the top-tier papers that cite it, and gives you its claims, its evidence and its limits, each one cited to the paper it comes from. No skimming, no AI slop.
11 top-tier papers have cited it since. Know what they found before your reviewers do.
Lune can draw on
the full paper · the top-tier papers that cite it · related work
Free to start. No credit card required.
Lineage
- Source Code Summarization in the Era of Large Language Models cites On the Evaluation of Neural Code Summarization
- Source Code Summarization in the Era of Large Language Models cites Retrieval-Augmented Generation for Code Summarization via Hybrid GNN
- Are we building on the rock? on the importance of data preprocessing for code summarization cites Leveraging Code Generation to Improve Code Retrieval and Summarization via Dual Learning
- Are we building on the rock? on the importance of data preprocessing for code summarization cites CodeBERT: A Pre-Trained Model for Programming and Natural Languages
- Are we building on the rock? on the importance of data preprocessing for code summarization cites On the Evaluation of Neural Code Summarization
- On the Evaluation of Neural Code Summarization cites Leveraging Code Generation to Improve Code Retrieval and Summarization via Dual Learning
- On the Evaluation of Neural Code Summarization cites CodeBERT: A Pre-Trained Model for Programming and Natural Languages
- SPT-Code: Sequence-to-Sequence Pre-Training for Learning Source Code Representations cites CodeBERT: A Pre-Trained Model for Programming and Natural Languages
Cited by top-tier papers11
- SPT-Code: Sequence-to-Sequence Pre-Training for Learning Source Code RepresentationsChangan Niu, Chuanyi Li, Vincent Ng, Jidong Ge et al.ICSE 2022 · 99 citations
- On the Evaluation of Neural Code SummarizationEnsheng Shi, Yanlin Wang, Lun Du, Junjie Chen et al.ICSE 2022 · 76 citations
- Are we building on the rock? on the importance of data preprocessing for code summarizationLin Shi, Fangwen Mu, Xiao Chen, Song Wang et al.FSE 2022 · 65 citations
- Source Code Summarization in the Era of Large Language ModelsWeisong Sun, Yun Miao, Yuekang Li, Hongyu Zhang et al.ICSE 2025 · 37 citations
- Adversarial Self-Attention for Language UnderstandingHongqiu Wu, Ruixue Ding, Hai Zhao, Pengjun Xie et al.AAAI 2023 · 19 citations
Builds on7
- ALBERT: A Lite BERT for Self-supervised Learning of Language RepresentationsZhenzhong Lan, Mingda Chen, Sebastian Goodman, Kevin Gimpel et al.ICLR 2020 · 7,418 citations
- CodeBERT: A Pre-Trained Model for Programming and Natural LanguagesZhangyin Feng, Daya Guo, Duyu Tang, Nan Duan et al.EMNLP Findings 2020 · 2,828 citations
- Discourse-Aware Neural Extractive Text SummarizationJiacheng Xu, Zhe Gan, Yu Cheng, Jingjing LiuACL 2020 · 264 citations
- Retrieval-Augmented Generation for Code Summarization via Hybrid GNNShangqing Liu, Yu Chen, Xiaofei Xie, Jing Kai Siow et al.ICLR 2021 · 194 citations
- Leveraging Code Generation to Improve Code Retrieval and Summarization via Dual LearningWei Ye, Rui Xie, Jinglei Zhang, Tianxiang Hu et al.WWW 2020 · 83 citations
Related papers
- Learning Sequential and Structural Information for Source Code SummarizationYunSeok Choi, JinYeong Bak, CheolWon Na, Jee-Hyong LeeACL Findings 2021 · 34 citations
- Modeling Hierarchical Syntax Structure with Triplet Position for Source Code SummarizationJuncai Guo, Jin Liu, Yao Wan, Li Li et al.ACL 2022
- AST-Trans: Code Summarization with Efficient Tree-Structured AttentionZe Tang, Xiaoyu Shen, Chuanyi Li, Jidong Ge et al.ICSE 2022 · 57 citations
- What Do They Capture? - A Structural Analysis of Pre-Trained Language Models for Source CodeYao Wan, Wei Zhao, Hongyu Zhang, Yulei Sui et al.ICSE 2022 · 66 citations
- Rethinking Positional Encoding in Tree Transformer for Code RepresentationHan Peng, Ge Li, Yunfei Zhao, Zhi JinEMNLP 2022 · 10 citations
