Taking the control back - An adventure in developing personalized content moderation

Online platforms are riddled with harassment, which significantly impacts the well-being of users. Unfortunately, the content moderation solutions provided by platforms often disappoint end-users as they fail to equip individuals with sufficient controls for their personal situations. In this work, the author, who personally experienced a sustained harassment campaign on Twitter, decided to regain control by constructing an automated, personalized, and collaborative anti-harassment system to protect herself, which has proven itself to be effective. The experience of developing—and re-developing in the face of repeated platform API changes and restrictions—this personalized content moderation system highlights many design issues that make managing severe online harassment such a challenge and invites critical study of the power dynamics between large online platforms and individual users. Through this analysis, this report aims to inform better designs to help platforms more effectively protect victims.