Lune

ACL Findings2025

CLeVeR: Multi-modal Contrastive Learning for Vulnerability Code Representation

Jiayuan Li, Lei Cui, Sen Zhao, Yun Yang, Lun Li, Hongsong Zhu

Abstract

Automated vulnerability detection has become increasingly important. Many existing methods utilize deep learning models to obtain code representations for vulnerability detection. However, these approaches predominantly capture the overall semantics of the code rather than its intrinsic vulnerability-specific semantics. To address this issue, we propose CLeVeR, the first approach that leverages contrastive learning to generate precise vulnerability code representations under the supervision of vulnerability descriptions. Specifically, we introduce an Adapter, a Representation Refinement module, and a Description Simulator to mitigate the challenges of semantic misalignment and imbalance between code and descriptions, and input data inconsistency between pre-training and fine-tuning stages, respectively. For vulnerability detection and classification tasks, CLeVeR achieves F1 scores of 72.82% (real-world dataset) and 80.34%, outperforming state-of-the-art methods (SOTAs) by 11.85% and 13.61%. Additionally, CLeVeR also outperforms SOTAs in zero-shot inference, demonstrating the transferability of its generated vulnerability code representations.

Ask Lune

Know this paper in minutes, not an afternoon.

Lune can read every page of it and the top-tier papers that cite it, and gives you its claims, its evidence and its limits, each one cited to the paper it comes from. No skimming, no AI slop.

Questions to start from

Lune can draw on

the full paper · related work

Get the answer

Free to start. No credit card required.

Lineage

  • CodeT5: Identifier-aware Unified Pre-trained Encoder-Decoder Models for Code Understanding and Generation cites CodeBERT: A Pre-Trained Model for Programming and Natural Languages
  • Learning Transferable Visual Models From Natural Language Supervision cites A Simple Framework for Contrastive Learning of Visual Representations
Builds onCited byLeft to right by year; the more cited, the higher.

Builds on16

  1. Learning Transferable Visual Models From Natural Language SupervisionAlec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh et al.ICML 2021 · 47,906 citations
  2. A Simple Framework for Contrastive Learning of Visual RepresentationsTing Chen, Simon Kornblith, Mohammad Norouzi, Geoffrey E. HintonICML 2020 · 24,064 citations
  3. CodeBERT: A Pre-Trained Model for Programming and Natural LanguagesZhangyin Feng, Daya Guo, Duyu Tang, Nan Duan et al.EMNLP Findings 2020 · 2,828 citations
  4. GraphCodeBERT: Pre-training Code Representations with Data FlowDaya Guo, Shuo Ren, Shuai Lu, Zhangyin Feng et al.ICLR 2021 · 1,644 citations
  5. CodeT5: Identifier-aware Unified Pre-trained Encoder-Decoder Models for Code Understanding and GenerationYue Wang, Weishi Wang, Shafiq R. Joty, Steven C. H. HoiEMNLP 2021 · 1,224 citations

Related papers

  1. Improving Smart Contract Security with Contrastive Learning-based Vulnerability DetectionYizhou Chen, Zeyu Sun, Zhihao Gong, Dan HaoICSE 2024 · 44 citations
  2. Code Vulnerability Detection via Nearest Neighbor MechanismQianjin Du, Xiaohui Kuang, Gang ZhaoEMNLP Findings 2022 · 6 citations
  3. Applying Contrastive Learning to Code Vulnerability Type ClassificationChen Ji, Su Yang, Hongyu Sun, Yuqing ZhangEMNLP 2024 · 6 citations
  4. Path-sensitive code embedding via contrastive learning for software vulnerability detectionXiao Cheng, Guanqin Zhang, Haoyu Wang, Yulei SuiISSTA 2022 · 98 citations
  5. Joint Geometrical and Statistical Domain Adaptation for Cross-domain Code Vulnerability DetectionQianjin Du, Shiji Zhou, Xiaohui Kuang, Gang Zhao et al.EMNLP 2023 · 4 citations
Dusk over the sea between two cliffs drawn in fine vertical lines