"I'm regretting that I hit run": In-situ Assessment of Potential Malware

We conduct the first ever two-session controlled lab study (n = 36) where end-users are prompted to install real benign and malicious software on a standard Windows laptop. The first session observes typical decision making strategies when participants assess software for potential threats without any instructions. The second session repeats the experiment after introducing an "enhanced task manager" application with information like CPU usage, files accessed, and network destination country to examine if decision making strategies change with more system-level information. The time, confidence, and accuracy to classify software as benign or malicious is recorded, along with participant comments using a "thinkaloud" protocol. These comments form a dataset of 2,651 excerpts that are coded into four top-level categories of "indicators" with 25 sub-categories. These indicators provide a perspective into how end-users examine and analyze software in-situ. Overall, end-users are surprisingly accurate at classifying malware and become even better when provided with more process-level statistics. Our analysis uncovers common misconceptions, shows reliance on indicators that bad actors could circumvent, and provides actionable insights for software and operating system providers to improve user interfaces and notifications.