Beyond the Stars: Multimodal Detection of Scams on GitHub

The open-source software development ecosystem has become a prime target for phishing and supply-chain attacks, precisely because its openness encourages trust. Defenders fighting these attacks operate in an environment where adversaries generate malicious artifacts at industrial scale, evade signature-based detection through trivial mutations, and exploit delays inherent in community-driven labeling. This combination produces sparse, biased ground truth and extreme class imbalance—conditions under which conventional evaluation pipelines break down. Unsurprisingly, many detection systems that excel offline degrade sharply in real-world deployments, limiting their impact against sustained campaigns. We present OctoWatch , a multimodal machine learning system for expanding detection of large-scale malicious source code repositories under real-world operational constraints while enabling rapid analyst pivoting through learned representations. The system fuses natural-language representations of repository metadata and file contents with structural embeddings of user–repository interactions and temporal activity statistics. Evaluated on real-world datasets—including large-scale secret-exposure attacks, prior threat-intelligence reporting, and a previously unreported phishing campaign—the approach consistently outperforms prior baselines in both precision and Matthews Correlation Coefficient (MCC). To assess operational viability, we conduct an online deployment that processes over ten million newly active repositories and identifies 14,667 total malicious repositories absent from major OSINT feeds. The system further enables discovery of coordinated campaigns, including one activity cluster associated with $5.9 million USD in cryptocurrency transactions on the Ethereum blockchain. Responsible disclosure of these findings led to the takedown of thousands of malicious repositories, demonstrating that multimodal fusion enables scalable detection of software-ecosystem abuse that systematically evades existing community-driven defenses.