Lending a Hand: The Effectiveness of Support Systems in Assisting Users to Detect Phishing Attacks

We investigate the effectiveness of anti-phishing support systems through a quantitative study involving 453 participants. To this end, we developed a tool that allows participants to immerse themselves in a realistic setting, tasked with classifying emails as either phishing or legitimate, while being assisted by support systems. Despite the prevalence of support systems in webmailers and email clients, our results indicate no significant difference in correctly assessing emails of varying difficulty between these systems and the control group. We found a minor negative effect of the support system that uses tooltips compared to other support systems. In the subsequent survey, we found that the support systems are appreciated and considered helpful by users, as supported by the results of the UEQ-S, even if they have no observable effect. Email context, such as the contact list, as well as hovering over the links, had stronger effects on the classification than the tested support systems.