From Large Language Models to Adversarial Malware: How far are we

Large Language Models (LLMs) have achieved notable progress in fields including natural language processing, cyber threat detection, and automated penetration testing, increasingly being applied in practical settings. However, the rapid advancement of these models has also led to their potential misuse, posing new challenges to cyberspace security. Security incidents have already been reported in areas such as phishing attacks and disinformation campaigns. Nevertheless, the progress and potential impact of LLMs in generating adversarial malware remain underexplored. This study systematically investigates the evasion capability of adversarial malware generated by LLMs. By integrating chain of thought into a Markov process and designing prompt based state transition functions and reward mechanisms, this research evaluates the effectiveness and efficiency against mainstream static detection methods on a dataset comprising over 2,000 real-world malware samples. Experimental results demonstrate an average evasion rate of 89.92% across 12 commercial antivirus engines on VirusTotal. The findings reveal that individuals with minimal technical expertise and basic natural language skills can generate malware that evades static detection, which underscores potential vulnerabilities in current cyberspace defense and detection systems regarding adversarial malware countermeasures. CCS Concepts • Security and privacy → Malware and its mitigation.