Understanding and Detecting File Knowledge Leakage in GPT App Ecosystem

OpenAI has enabled third-party developers to build applications around ChatGPT, known as GPTs, to expand its capability to handle complex and specialized tasks. A key feature of GPTs is Retrieval-Augmented Generation (RAG), which allows developers to upload documents containing domain knowledge or application context, referred to as file knowledge. However, these documents often contain sensitive information, and the security mechanisms governing access control in GPTs remains an underexplored area. In this work, we present the first comprehensive study on file knowledge leakage within GPTs. We develop GPTs-Filtor, leveraging the unique characteristics of GPTs deployment, to perform an in-depth analysis and detection of file knowledge leakage at both user interaction (i.e., prompt) and network transmission levels. Applying GPTs-Filtor to 8,000 popular GPTs across eight different categories, we reveal widespread vulnerabilities in the current GPTs development and deployment model. We detect 618 cases of leakage among 1,331 GPTs that involve uploaded file knowledge, leading to the exfiltration of 3,645 file contents that contain highly-sensitive data such as internal bank audit transaction records. Our work underscores the pressing need for improved security practices in GPTs development and deployment, providing crucial insights for the secure development of this young but rapidly evolving ecosystem.