USENIX Security2026Top-tier venue
The Impact of Security and Privacy Controls on Users' Emotional Engagement with Generative AI Chatbots
Jabari Kwesi, Jiaxun Cao, Hailee Cunningham, Pardis Emami-Naeini
Abstract
Chatbots powered by generative AI (e.g., OpenAI's ChatGPT and Google's Gemini) are increasingly being appropriated for emotional support and companionship. These tools offer a suite of security and privacy (S&P) controls, including model training opt-outs and memory toggles, yet how the presence of these controls influences users' attitudes toward emotionally sensitive disclosure remains understudied. We conducted a mixed-methods vignette study with 354 U.S. participants to examine how S&P controls influence users' willingness to engage with generative AI chatbots for emotional support, their perceptions of how protected they are when using these systems, and their perceptions of how effective the chatbots are for providing support. Controls enabling deletion of disclosures had the largest positive impact: these offerings outperformed technically sophisticated controls such as local-only processing and model training opt-outs, where participants expressed difficulty understanding the underlying mechanisms. Yet trust remains fragile, and participants often doubted S&P controls would function as promised. We conclude with actionable recommendations informed by our results to bridge users' comprehension gaps, build credible assurances, and properly calibrate barriers for users in distress.
Ask about this paper
Your agent reads all of it.
Lune indexed this paper to the last equation, along with the top-tier papers that cite it. Ask a question and the answer quotes them.
Your agent calls
Luneget_paper_fulltext
Free to start. No credit card required.
Terminal
Install the CLIlune papers fulltext 55c81d5a-8d46-48ff-b72a-a7958492917aBuilds on11
- Extracting Training Data from Large Language ModelsNicholas Carlini, Florian Tramèr, Eric Wallace, Matthew Jagielski et al.USENIX Security 2021 · 2,866 citations
- "I Hear You, I Feel You": Encouraging Deep Self-disclosure through a ChatbotYi-Chieh Lee, Naomi Yamashita, Yun Huang, Wai FuCHI 2020 · 333 citations
- Beyond Memorization: Violating Privacy via Inference with Large Language ModelsRobin Staab, Mark Vero, Mislav Balunovic, Martin T. VechevICLR 2024 · 211 citations
- Can LLMs Keep a Secret? Testing Privacy Implications of Language Models via Contextual Integrity TheoryNiloofar Mireshghallah, Hyunwoo Kim, Xuhui Zhou, Yulia Tsvetkov et al.ICLR 2024 · 198 citations
- UX Research on Conversational Human-AI Interaction: A Literature Review of the ACM Digital LibraryQingxiao Zheng, Yiliu Tang, Yiren Liu, Weizi Liu et al.CHI 2022 · 99 citations
Related papers
- Privacy and Trust vs. Utility: Adoption of Commercial vs. Institutional AI assistants Among University UsersYuting Yang, Zixin Wang, Rongjun Ma, Florian SchaubCHI 2026 · 1 citation
- Exploring User Security and Privacy Attitudes and Concerns Toward the Use of General-Purpose LLM Chatbots for Mental HealthJabari Kwesi, Jiaxun Cao, Riya Manchanda, Pardis Emami NaeiniUSENIX Security 2025
- Understanding Parents' Desires in Moderating Children's Interactions with GenAI Chatbots through LLM-Generated ProbesJohn Driscoll, Yulin Chen, Viki Shi, Izak Vucharatavintara et al.CHI 2026 · 2 citations
- Relational Gains, Privacy Strains: Exploring Users' Perceptions and Experiences with ChatGPT's Memory FeatureCheng Chen, Maria D. Molina, Mengqi Liao, Eugene Cho SnyderCHI 2026 · 1 citation
- Understanding Attitudes and Trust of Generative AI Chatbots for Social Anxiety SupportYimeng Wang, Yinzhou Wang, Kelly Crace, Yixuan ZhangCHI 2025 · 25 citations
