USENIX Security2026Top-tier venue
What Users Ask, Policies Miss: Unveiling the Gap Between Community-Expressed Privacy Concerns and LLM Provider Policies
Zhihuang Liu, Zhen Huang, Ling Hu, Yifan Yang, Zhiping Cai
Abstract
Large Language Models(LLMs) process millions of conversations containing sensitive information daily, yet whether their privacy policies adequately address users' publicly expressed concerns remains unexplored. This paper presents the first large-scale, user-centered audit of privacy policy adequacy in LLM services. We systematically extract privacy concerns from Reddit communities of five major LLM providers and assess whether their latest privacy policies address these concerns. Our semi-automated pipeline analyzes 1,531 threads to extract 4,994 authentic privacy concerns, which we organize into a 20-topic taxonomy spanning four thematic groups. We then identify 3,137 policy gap instances and classify them into six categories: four policy coverage gaps (detail vague, AI feature unaddressed, vulnerable group neglected, and jurisdiction unclear) and two user perception gaps (explicit distrust and awareness deficit). Our analysis reveals that coverage gaps and perception gaps contribute nearly equally (50.2% vs. 49.8%), with AI-specific feature gaps (36.6%) and user awareness deficits (40.7%) together comprising the vast majority (77.3%) of all identified gaps. Notably, we also surface user-reported evidence suggesting potential discrepancies between stated policies and observed system behavior, highlighting the need for verifiable privacy guarantees. These findings demonstrate that improving LLM privacy requires dual-pronged interventions addressing both inadequate policy disclosures and user comprehension barriers, offering actionable insights for relevant stakeholders.
Ask about this paper
Your agent reads all of it.
Lune indexed this paper to the last equation, along with the top-tier papers that cite it. Ask a question and the answer quotes them.
Your agent calls
Luneget_paper_fulltext
Free to start. No credit card required.
Terminal
Install the CLIlune papers fulltext 9f719c3b-fd23-4a9f-a9c8-a529b8be5bf6Builds on38
- Tree of Thoughts: Deliberate Problem Solving with Large Language ModelsShunyu Yao, Dian Yu, Jeffrey Zhao, Izhak Shafran et al.NeurIPS 2023 · 5,068 citations
- NExT-GPT: Any-to-Any Multimodal LLMShengqiong Wu, Hao Fei, Leigang Qu, Wei Ji et al.ICML 2024 · 786 citations
- Polisis: Automated Analysis and Presentation of Privacy Policies Using Deep LearningHamza Harkous, Kassem Fawaz, Rémi Lebret, Florian Schaub et al.USENIX Security 2018 · 400 citations
- How Well Do My Results Generalize? Comparing Security and Privacy Survey Results from MTurk, Web, and Telephone SamplesElissa M. Redmiles, Sean Kross, Michelle L. MazurekS&P 2019 · 222 citations
- PolicyLint: Investigating Internal Privacy Policy Contradictions on Google PlayBenjamin Andow, Samin Yaseer Mahmud, Wenyu Wang, Justin Whitaker et al.USENIX Security 2019 · 185 citations
Related papers
- Privacy Control in Conversational LLM Platforms: A Walkthrough StudyZhuoyang Li, Yanlai Wu, Yao Li, Xinning Gui et al.CHI 2026 · 1 citation
- Beyond Memorization: Violating Privacy via Inference with Large Language ModelsRobin Staab, Mark Vero, Mislav Balunovic, Martin T. VechevICLR 2024 · 211 citations
- Prevalence Overshadows Concerns? Understanding Chinese Users' Privacy Awareness and Expectations Towards LLM-Based Healthcare ConsultationZhihuang Liu, Ling Hu, Tongqing Zhou, Yonghao Tang et al.S&P 2025
- The Privacy Paradox of LLMs: User Perceptions and the Reality of PII LeakageShuai Cheng, Haitao Xu, Shu Meng, Shuai Hao et al.CHI 2026
- Exploring User Security and Privacy Attitudes and Concerns Toward the Use of General-Purpose LLM Chatbots for Mental HealthJabari Kwesi, Jiaxun Cao, Riya Manchanda, Pardis Emami NaeiniUSENIX Security 2025
