Who Controls the Conversation? User Perspectives on Generative AI (LLM) System Prompts
Anna Neumann, Yulu Pi, Jatinder Singh
Abstract
System prompts-instructions that shape the behaviour of generative AI systems-strongly influence system outputs and users' experiences. They define the model's guidelines, 'personality', and guardrails, taking precedence over user inputs. Despite their influence, transparency is limited: system prompts are generally not made public and most platforms instruct models to conceal them, leaving users disconnected from and unaware of a key mechanism guiding and governing their AI interactions.
This paper argues that system prompts warrant explicit, usercentred design attention and, focusing on large language models (LLMs), asks: what do system prompts contain, how do end-users perceive them, and what do these perceptions offer for design and governance practice?
Our results reveal user perspectives on: the benefits and risks of system prompts; the values they prefer to be associated with prompt-design; their levels of comfort with different types of prompts; and degrees of transparency and user control regarding prompt content. From these findings emerge considerations for how designers can better align system prompt mechanisms with user expectations and preferences over these mechanisms that directly shape how generative AI systems behave.
Ask about this paper
Your agent reads all of it.
Lune indexed this paper to the last equation, along with the top-tier papers that cite it. Ask a question and the answer quotes them.
Your agent calls
Luneget_paper_fulltext
Free to start. No credit card required.
Terminal
Install the CLIlune papers fulltext 77f09417-c071-45e1-b41f-82cfdd9a499dCited by top-tier papers1
Ask how each one uses itBuilds on37
- Fine-tuning Aligned Language Models Compromises Safety, Even When Users Do Not Intend To!Xiangyu Qi, Yi Zeng, Tinghao Xie, Pin-Yu Chen et al.ICLR 2024 · 1,104 citations
- Why Johnny Can't Prompt: How Non-AI Experts Try (and Fail) to Design LLM PromptsJ. D. Zamfirescu-Pereira, Richmond Y. Wong, Bjoern Hartmann, Qian YangCHI 2023 · 892 citations
- Quantifying Language Models' Sensitivity to Spurious Features in Prompt Design or: How I learned to start worrying about prompt formattingMelanie Sclar, Yejin Choi, Yulia Tsvetkov, Alane SuhrICLR 2024 · 682 citations
- Expanding Explainability: Towards Social Transparency in AI systemsUpol Ehsan, Q. Vera Liao, Michael J. Muller, Mark O. Riedl et al.CHI 2021 · 505 citations
- LLMs Get Lost In Multi-Turn ConversationPhilippe Laban, Hiroaki Hayashi, Yingbo Zhou, Jennifer NevilleICLR 2026 · 491 citations
Related papers
- Bridging Gulfs in UI Generation through Semantic GuidanceSeokhyeon Park, Soohyun Lee, Eugene Choi, Hyunwoo Kim et al.CHI 2026 · 3 citations
- Data-Prompt Co-Evolution: Growing Test Sets to Refine LLM BehaviorMinjae Lee, Minsuk KahngCHI 2026 · 1 citation
- LLM Whisperer: An Inconspicuous Attack to Bias LLM ResponsesWeiran Lin, Anna Gerchanovsky, Omer Akgul, Lujo Bauer et al.CHI 2025 · 23 citations
- Prompt Coaching for Inclusiveness: A Media Literacy Approach to Increase Users' Awareness of Algorithmic Bias and Prompting EfficacyCheng Chen, Mengqi Liao, Aditya Anand Phadnis, Yao Li et al.CHI 2026 · 3 citations
- Leveraging Large Language Models to Power Chatbots for Collecting User Self-Reported DataJing Wei, Sungdong Kim, Hyunhoon Jung, Young-Ho KimCSCW 2024 · 82 citations
