Using AI Assistants in Software Development: A Qualitative Study on Security Practices and Concerns
Jan H. Klemmer, Stefan Albert Horstmann, Nikhil Patnaik, Cordelia Ludden, Cordell Burton Jr., Carson Powers, Fabio Massacci, Akond Rahman, Daniel Votipka, Heather Richter Lipford, Awais Rashid, Alena Naiakshina, Sascha Fahl
Abstract
Following the recent release of AI assistants, such as OpenAI's ChatGPT and GitHub Copilot, the software industry quickly utilized these tools for software development tasks, e.g., generating code or consulting AI for advice. While recent research has demonstrated that AI-generated code can contain security issues, how software professionals balance AI assistant usage and security remains unclear. This paper investigates how software professionals use AI assistants in secure software development, what security implications and considerations arise, and what impact they foresee on secure software development. We conducted 27 semi-structured interviews with software professionals, including software engineers, team leads, and security testers. We also reviewed 190 relevant Reddit posts and comments to gain insights into the current discourse surrounding AI assistants for software development. Our analysis of the interviews and Reddit posts finds that despite many security and quality concerns, participants widely use AI assistants for security-critical tasks, e.g., code generation, threat modeling, and vulnerability detection. Their overall mistrust leads to checking AI suggestions in similar ways to human code, although they expect improvements and, therefore, a heavier use for security tasks in the future. We conclude with recommendations for software professionals to critically check AI suggestions, AI creators to improve suggestion security and capabilities for ethical security tasks, and academic researchers to consider general-purpose AI in software development.
Ask about this paper
Your agent reads all of it.
Lune indexed this paper to the last equation, along with the top-tier papers that cite it. Ask a question and the answer quotes them.
Your agent calls
Luneget_paper_fulltext
Free to start. No credit card required.
Terminal
Install the CLIlune papers fulltext 04f5d088-b477-466d-b446-5fee28915a7dCited by top-tier papers13
- LLM Hallucinations in Practical Code Generation: Phenomena, Mechanism, and MitigationZiyao Zhang, Chong Wang, Yanlin Wang, Ensheng Shi et al.ISSTA 2025 · 53 citations
- An Investigation of Interaction and Information Needs for Protocol Reverse Engineering AutomationSamantha Katcher, James Mattei, Jared Chandler, Daniel VotipkaCHI 2025 · 7 citations
- "That's another doom I haven't thought about": A User Study on AI Labels as a Safeguard Against Image-Based MisinformationSandra Höltervennhoff, Jonas Ricker, Maike M. Raphael, Charlotte Schwedes et al.CHI 2026 · 2 citations
- "Impressively Scary: ' Exploring User Perceptions and Reactions to Unraveling Machine Learning Models in Social Media ApplicationsJack West, Bengisu Cagiltay, Shirley Zhang, Jingjie Li et al.CHI 2025 · 2 citations
- 'Tab, Tab, Bug': Security Pitfalls of Next Edit Suggestions in AI-Integrated IDEsYunlong Lyu, Yixuan Tang, Peng Chen, Tian Dong et al.CCS 2026 · 1 citation
Builds on22
- Asleep at the Keyboard? Assessing the Security of GitHub Copilot's Code ContributionsHammond Pearce, Baleegh Ahmad, Benjamin Tan, Brendan Dolan-Gavitt et al.S&P 2022 · 725 citations
- You Get Where You're Looking for: The Impact of Information Sources on Code SecurityYasemin Acar, Michael Backes, Sascha Fahl, Doowon Kim et al.S&P 2016 · 325 citations
- Stack Overflow Considered Harmful? The Impact of Copy&Paste on Android Application SecurityFelix Fischer, Konstantin Böttinger, Huang Xiao, Christian Stransky et al.S&P 2017 · 293 citations
- Do Users Write More Insecure Code with AI Assistants?Neil Perry, Megha Srivastava, Deepak Kumar, Dan BonehCCS 2023 · 150 citations
- Is Stack Overflow Obsolete? An Empirical Study of the Characteristics of ChatGPT Answers to Stack Overflow QuestionsSamia Kabir, David N. Udo-Imeh, Bonan Kou, Tianyi ZhangCHI 2024 · 149 citations
Related papers
- Poisoned ChatGPT Finds Work for Idle Hands: Exploring Developers' Coding Practices with Insecure Suggestions from Poisoned AI ModelsSanghak Oh, Kiho Lee, Seonhye Park, Doowon Kim et al.S&P 2024 · 42 citations
- Exploring the Impact of Intervention Methods on Developers' Security Behavior in a Manipulated ChatGPT StudyRaphael Serafini, Asli Yardim, Alena NaiakshinaCHI 2025 · 5 citations
- A User-centered Security Evaluation of CopilotOwura Asare, Meiyappan Nagappan, N. AsokanICSE 2024 · 11 citations
- "The AI tool can't make it any worse." Investigating Developers' Security Behavior with AI Assistants in a Password Storage StudyAsli Yardim, Raphael Serafini, Nadine Jost, Anna-Marie Ortloff et al.CHI 2026 · 1 citation
- 'Always Nice and Confident, Sometimes Wrong': Developer's Experiences Engaging Generative AI Chatbots Versus Human-Powered Q&A PlatformsJiachen Li, Elizabeth D. Mynatt, Varun Mishra, Jonathan BellCSCW 2025 · 8 citations
