SoK: Hate, Harassment, and the Changing Landscape of Online Abuse
Kurt Thomas, Devdatta Akhawe, Michael D. Bailey, Dan Boneh, Elie Bursztein, Sunny Consolvo, Nicola Dell, Zakir Durumeric, Patrick Gage Kelley, Deepak Kumar, Damon McCoy, Sarah Meiklejohn
Abstract
We argue that existing security, privacy, and antiabuse protections fail to address the growing threat of online hate and harassment. In order for our community to understand and address this gap, we propose a taxonomy for reasoning about online hate and harassment. Our taxonomy draws on over 150 interdisciplinary research papers that cover disparate threats ranging from intimate partner violence to coordinated mobs. In the process, we identify seven classes of attacks-such as toxic content and surveillance-that each stem from different attacker capabilities and intents. We also provide longitudinal evidence from a three-year survey that hate and harassment is a pervasive, growing experience for online users, particularly for at-risk communities like young adults and people who identify as LGBTQ+. Responding to each class of hate and harassment requires a unique strategy and we highlight five such potential research directions that ultimately empower individuals, communities, and platforms to do so.
Ask about this paper
Your agent reads all of it.
Lune indexed this paper to the last equation, along with the top-tier papers that cite it. Ask a question and the answer quotes them.
Your agent calls
Luneget_paper_fulltext
Free to start. No credit card required.
Terminal
Install the CLIlune papers fulltext 598983e5-7740-4d4e-a17f-1016e559200cCited by top-tier papers67
- Trauma-Informed Computing: Towards Safer Technology Experiences for AllJanet X. Chen, Allison McDonald, Yixin Zou, Emily Tseng et al.CHI 2022 · 163 citations
- SoK: A Framework for Unifying At-Risk User ResearchNoel Warford, Tara Matthews, Kaitlyn Yang, Omer Akgul et al.S&P 2022 · 101 citations
- "It's stressful having all these phones": Investigating Sex Workers' Safety Goals, Risks, and Practices OnlineAllison McDonald, Catherine Barwulor, Michelle L. Mazurek, Florian Schaub et al.USENIX Security 2021 · 75 citations
- Automated Content Moderation Increases Adherence to Community GuidelinesManoel Horta Ribeiro, Justin Cheng, Robert WestWWW 2023 · 53 citations
- SoK: Safer Digital-Safety Research Involving At-Risk UsersRosanna Bellini, Emily Tseng, Noel Warford, Alaa Daffalla et al.S&P 2024 · 48 citations
Builds on7
- The Spyware Used in Intimate Partner ViolenceRahul Chatterjee, Periwinkle Doerfler, Hadas Orgad, Sam Havron et al.S&P 2018 · 167 citations
- See No Evil, Hear No Evil, Speak No Evil: How Collegiate Players Define, Experience and Cope with ToxicitySelen Türkay, Jessica Formosa, Sonam Adinolf, Robert Cuthbert et al.CHI 2020 · 126 citations
- Clinical Computer Security for Victims of Intimate Partner ViolenceSam Havron, Diana Freed, Rahul Chatterjee, Damon McCoy et al.USENIX Security 2019 · 118 citations
- Many Faced Hate: A Cross Platform Study of Content Framing and Information Sharing by Online Hate GroupsShruti Phadke, Tanushree MitraCHI 2020 · 64 citations
- To Catch a Ratter: Monitoring the Behavior of Amateur DarkComet RAT Operators in the WildBrown Farinholt, Mohammad Rezaeirad, Paul Pearce, Hitesh Dharmdasani et al.S&P 2017 · 48 citations
Related papers
- Understanding the Behaviors of Toxic Accounts on RedditDeepak Kumar, Jeff T. Hancock, Kurt Thomas, Zakir DurumericWWW 2023 · 31 citations
- The Tools and Tactics Used in Intimate Partner Surveillance: An Analysis of Online Infidelity ForumsEmily Tseng, Rosanna Bellini, Nora McDonald, Matan Danos et al.USENIX Security 2020
- "It's common and a part of being a content creator": Understanding How Creators Experience and Cope with Hate and Harassment OnlineKurt Thomas, Patrick Gage Kelley, Sunny Consolvo, Patrawat Samermit et al.CHI 2022 · 63 citations
- Harassment in Virtual Reality: A Systematic ReviewJiong Dong, Yijun Lu, Shuai Wu, Yuyin Ma et al.CHI 2026 · 1 citation
- Investigating Moderation Challenges to Combating Hate and Harassment: The Case of Mod-Admin Power Dynamics and Feature Misuse on RedditMadiha Tabassum, Alana Mackey, Ashley Schuett, Ada LernerUSENIX Security 2024 · 10 citations
