Lune

EMNLP Findings2023

Towards Detecting Contextual Real-Time Toxicity for In-Game Chat

Zachary Yang, Nicolas Grenon-Godbout, Reihaneh Rabbany

Abstract

Real-time toxicity detection in online environments poses a significant challenge, due to the increasing prevalence of social media and gaming platforms. We introduce ToxBuster, a simple and scalable model that reliably detects toxic content in real-time for a line of chat by including chat history and metadata. ToxBuster consistently outperforms conventional toxicity models across popular multiplayer games, including Rainbow Six Siege, For Honor, and DOTA 2. We conduct an ablation study to assess the importance of each model component and explore ToxBuster's transferability across the datasets. Furthermore, we showcase Tox-Buster's efficacy in post-game moderation, successfully flagging 82.1% of chat-reported players at a precision level of 90.0%. Additionally, we show how an additional 6% of unreported toxic players can be proactively moderated.

Ask Lune

Know this paper in minutes, not an afternoon.

Lune can read every page of it and the top-tier papers that cite it, and gives you its claims, its evidence and its limits, each one cited to the paper it comes from. No skimming, no AI slop.

1 top-tier paper has cited it since. Know what they found before your reviewers do.

Questions to start from

Lune can draw on

the full paper · the top-tier papers that cite it · related work

Get the answer

Free to start. No credit card required.

Lineage

  • CONDA: a CONtextual Dual-Annotated dataset for in-game toxicity understanding and detection cites Toxicity Detection: Does Context Really Matter?
  • An Empirical Survey of the Effectiveness of Debiasing Techniques for Pre-trained Language Models cites Language (Technology) is Power: A Critical Survey of "Bias" in NLP
Builds onCited byLeft to right by year; the more cited, the higher.

Cited by top-tier papers1

  1. Needling Through the Threads: A Visualization Tool for Navigating Threaded Online DiscussionsYijun Liu, Frederick Choi, Eshwar ChandrasekharanCHI 2026 · 1 citation

Builds on4

  1. An Empirical Survey of the Effectiveness of Debiasing Techniques for Pre-trained Language ModelsNicholas Meade, Elinor Poole-Dayan, Siva ReddyACL 2022 · 160 citations
  2. Language (Technology) is Power: A Critical Survey of "Bias" in NLPSu Lin Blodgett, Solon Barocas, Hal Daumé III, Hanna M. WallachACL 2020 · 68 citations
  3. CONDA: a CONtextual Dual-Annotated dataset for in-game toxicity understanding and detectionHenry Weld, Guanghao Huang, Jean Lee, Tongshu Zhang et al.ACL Findings 2021 · 21 citations
  4. Toxicity Detection: Does Context Really Matter?John Pavlopoulos, Jeffrey Sorensen, Lucas Dixon, Nithum Thain et al.ACL 2020 · 11 citations

Related papers

  1. Traumatizing or Just Annoying? Unveiling the Spectrum of Gamer Toxicity in the StarCraft II CommunitySamuli Laato, Bastian Kordyaka, Juho HamariCHI 2024 · 22 citations
  2. ToxicChat: Unveiling Hidden Challenges of Toxicity Detection in Real-World User-AI ConversationZi Lin, Zihan Wang, Yongqi Tong, Yangkun Wang et al.EMNLP Findings 2023 · 44 citations
  3. ''Shut Up and Let Me Enjoy My Otome': Understanding and Measuring the Toxicity in Otome Game CommunitiesYage Zhang, Xinyue Shen, Yukun Jiang, Michael Backes et al.CCS 2026
  4. Flag and Flaggability in Automated Moderation: The Case of Reporting Toxic Behavior in an Online Game CommunityYubo Kou, Xinning GuiCHI 2021 · 85 citations
  5. Transparency, Fairness, and Coping: How Players Experience Moderation in Multiplayer Online GamesRenkai Ma, Yao Li, Yubo KouCHI 2023 · 23 citations
Dusk over the sea between two cliffs drawn in fine vertical lines