Your Space is My Zone: Demystifying the Security Risks of AI-Powered Applications on Pre-Trained Model Hubs
Yacong Gu, Lingyun Ying, Zidong Zhang, Yingyuan Pu, Xiaoxue Huang, Jiawei Zhou, Wenjie Zhu, Donghong Sun, Haixin Duan
Abstract
AI-powered Applications (AI-Apps), hosted on platforms such as Hugging Face, are democratizing access to pre-trained models through online inference and fine-tuning services. While lowering AI adoption barriers, these platforms introduce an unexplored attack surface, as AI-Apps are often developed by untrusted parties with weak isolation and misconfigured security settings. In this paper, we present the first systematic security analysis of AI-Apps across three leading platforms. To structure our investigation, we map the AI-App lifecycle to established risk taxonomies (e.g., OWASP), identifying five threat categories and ten attack vectors ranging from generic web flaws to high-impact architectural issues. Our analysis reveals critical failures including broken access control, insecure resource reuse, insufficient input validation, and sensitive data exposure. Notably, we uncover three novel architectural vulnerabilities inherent to platform design and demonstrate how traditional issues (e.g., world-readable logs) are uniquely amplified in this ecosystem. To assess real-world impact, we develop an analysis framework Insightor and apply it to over 970,000 public AI-Apps. Alarmingly, we find thousands of apps leaking credentials, hundreds containing input injection vulnerabilities that allow arbitrary code execution, and tens harboring embedded backdoors -- indicating active exploitation. We have responsibly disclosed all findings to the affected platforms and developers.
Ask about this paper
Your agent reads all of it.
Lune indexed this paper to the last equation, along with the top-tier papers that cite it. Ask a question and the answer quotes them.
Your agent calls
Luneget_paper_fulltext
Free to start. No credit card required.
Terminal
Install the CLIlune papers fulltext 9ceaf76c-f873-4d00-8691-deb3b659e42aBuilds on12
- How Bad Can It Git? Characterizing Secret Leakage in Public GitHub RepositoriesMichael Meli, Matthew R. McNiece, Bradley ReavesNDSS 2019 · 130 citations
- All Your DNS Records Point to Us: Understanding the Security Threats of Dangling DNS RecordsDaiping Liu, Shuai Hao, Haining WangCCS 2016 · 91 citations
- How to End Password Reuse on the WebKe Coby Wang, Michael K. ReiterNDSS 2019 · 49 citations
- Game of Registrars: An Empirical Analysis of Post-Expiration Domain Name TakeoversTobias Lauinger, Abdelberi Chaabane, Ahmet Salih Buyukkayhan, Kaan Onarlioglu et al.USENIX Security 2017 · 39 citations
- Automated Detection of Password Leakage from Public GitHub RepositoriesRunhan Feng, Ziyang Yan, Shiyan Peng, Yuanyuan ZhangICSE 2022 · 36 citations
Related papers
- A Large-Scale Empirical Study of Secret Key Leakage in Hugging Face SpacesShaoxuan Yun, Yuchao Zhang, Zhikun Shi, Liu Wang et al.ICSE 2026
- My Model is Malware to You: Transforming AI Models into Malware by Abusing TensorFlow APIsRuofan Zhu, Ganhao Chen, Wenbo Shen, Xiaofei Xie et al.S&P 2025
- Securing the AI Supply Chain: What Can We Learn From Developer-Reported Security Issues and Solutions of AI Projects?The Anh Nguyen, Triet Huynh Minh Le, M. Ali BabarICSE 2026 · 1 citation
- Hugging Carbon: Quantifying the Training Carbon Emissions of AI Models at ScaleXinlei Wang, Ruibo Ming, Jing Qiu, Junhua Zhao et al.ICML 2026
- Exploring and Exploiting Security Vulnerabilities in Self-Hosted LLM ServicesZhihuang Liu, Ling Hu, Yonghao Tang, Tongqing Zhou et al.WWW 2026
