USENIX Security2025Top-tier venue
Exploring User Security and Privacy Attitudes and Concerns Toward the Use of General-Purpose LLM Chatbots for Mental Health
Jabari Kwesi, Jiaxun Cao, Riya Manchanda, Pardis Emami Naeini
Abstract
Individuals are increasingly relying on large language model (LLM)-enabled conversational agents for emotional support. While prior research has examined privacy and security issues in chatbots specifically designed for mental health purposes, these chatbots are overwhelmingly"rule-based"offerings that do not leverage generative AI. Little empirical research currently measures users'privacy and security concerns, attitudes, and expectations when using general-purpose LLM-enabled chatbots to manage and improve mental health. Through 21 semi-structured interviews with U.S. participants, we identified critical misconceptions and a general lack of risk awareness. Participants conflated the human-like empathy exhibited by LLMs with human-like accountability and mistakenly believed that their interactions with these chatbots were safeguarded by the same regulations (e.g., HIPAA) as disclosures with a licensed therapist. We introduce the concept of"intangible vulnerability,"where emotional or psychological disclosures are undervalued compared to more tangible forms of information (e.g., financial or location-based data). To address this, we propose recommendations to safeguard user mental health disclosures with general-purpose LLM-enabled chatbots more effectively.
Ask about this paper
Your agent reads all of it.
Lune indexed this paper to the last equation, along with the top-tier papers that cite it. Ask a question and the answer quotes them.
Your agent calls
Luneget_paper_fulltext
Free to start. No credit card required.
Terminal
Install the CLIlune papers fulltext afc20d89-5e68-48bf-83dd-5077bd770231Cited by top-tier papers5
- The Impact of Security and Privacy Controls on Users' Emotional Engagement with Generative AI ChatbotsJabari Kwesi, Jiaxun Cao, Hailee Cunningham, Pardis Emami-NaeiniUSENIX Security 2026 · 2 citations
- More than Decision Support: Exploring Patients' Longitudinal Usage of Large Language Models in Real-World Healthcare-Seeking JourneysYancheng Cao, Yishu Ji, Chris Yue Fu, Sahiti Dharmavaram et al.CHI 2026 · 2 citations
- Privacy and Safety Experiences and Concerns of US Women Using Generative AI for Seeking Sexual and Reproductive Health InformationIna Kaleva, Xiao Zhan, Ruba Abu-Salma, Jose SuchCHI 2026 · 1 citation
- The Privacy Paradox of LLMs: User Perceptions and the Reality of PII LeakageShuai Cheng, Haitao Xu, Shu Meng, Shuai Hao et al.CHI 2026
- What Users Ask, Policies Miss: Unveiling the Gap Between Community-Expressed Privacy Concerns and LLM Provider PoliciesZhihuang Liu, Zhen Huang, Ling Hu, Yifan Yang et al.USENIX Security 2026
Builds on5
- "I Hear You, I Feel You": Encouraging Deep Self-disclosure through a ChatbotYi-Chieh Lee, Naomi Yamashita, Yun Huang, Wai FuCHI 2020 · 333 citations
- Designing a Chatbot as a Mediator for Promoting Deep Self-Disclosure to a Real Mental Health ProfessionalYi-Chieh Lee, Naomi Yamashita, Yun HuangCSCW 2020 · 190 citations
- "It's the Company, the Government, You and I": User Perceptions of Responsibility for Smart Home Privacy and SecurityJulie M. Haney, Yasemin Acar, Susanne FurmanUSENIX Security 2021 · 49 citations
- The Typing Cure: Experiences with Large Language Model Chatbots for Mental Health SupportInhwa Song, Sachin R. Pendse, Neha Kumar, Munmun De ChoudhuryCSCW 2025 · 41 citations
- Cognitive Reframing of Negative Thoughts through Human-Language Model InteractionAshish Sharma, Kevin Rushton, Inna E. Lin, David Wadden et al.ACL 2023 · 32 citations
Related papers
- Prevalence Overshadows Concerns? Understanding Chinese Users' Privacy Awareness and Expectations Towards LLM-Based Healthcare ConsultationZhihuang Liu, Ling Hu, Tongqing Zhou, Yonghao Tang et al.S&P 2025
- A Conditional Companion: Lived Experiences of People with Mental Health Disorders Using LLMs: Conditional Companion: LLMs & Mental HealthAditya Kumar Purohit, Hendrik HeuerCHI 2026 · 3 citations
- Malicious LLM-Based Conversational AI Makes Users Reveal Personal InformationXiao Zhan, Juan Carlos Carrillo, William Seymour, Jose SuchUSENIX Security 2025
- "It's a Fair Game", or Is It? Examining How Users Navigate Disclosure Risks and Benefits When Using LLM-Based Conversational AgentsZhiping Zhang, Michelle Jia, Hao-Ping (Hank) Lee, Bingsheng Yao et al.CHI 2024 · 92 citations
- 'I've talked to ChatGPT about my issues last night.': Examining Mental Health Conversations with Large Language Models through Reddit AnalysisKyuha Jung, Gyuho Lee, Yuanhui Huang, Yunan ChenCSCW 2025 · 23 citations
