Droid: A Resource Suite for AI-Generated Code Detection

We present DroidCollection 1 2 , the most extensive open data suite for training and evaluating machine-generated code detectors, comprising over a million code samples, seven programming languages, outputs from 43 coding models, and three real-world coding domains. Alongside fully AI-generated examples, our collection includes human-AI co-authored code, as well as adversarial examples explicitly crafted to evade detection. Subsequently, we develop DroidDetect, a suite of encoderonly detectors trained using a multi-task objective over DroidCollection. Our experiments show that existing detectors' performance fails to generalise to diverse coding domains and programming languages outside of their narrow training data. We further demonstrate that while most detectors are easily compromised by humanising the output distributions using superficial prompting and alignment approaches, this problem can be easily amended by training on a small number of adversarial examples. Finally, we demonstrate the effectiveness of metric learning and uncertainty-based resampling as way to enhance detector training on possibly noisy distributions. 1 https://huggingface.co/collections/ project-droid/droid-683360d8b008214a4273099a 2 https://tudatalib.ulb.tu-darmstadt.de/ items/ebc68cfb-186e-4303-bd46-cbd015af2045 Model 2-Class 3-Class