Lune

ACL Findings2024

Implanting LLM's Knowledge via Reading Comprehension Tree for Toxicity Detection

Hankun Kang, Tieyun Qian

Abstract

Warning: This paper contains several toxic and offensive statements.

Toxicity detection plays a crucial role in maintaining the peace of the society. Existing methods can be roughly categorized as small language model (SLM) based and large language model (LLM) based. However, due to the limitation of SLMs on general knowledge and the potential embedded bias in LLMs despite their large amount of knowledge, it is not a good idea to detect toxicity only with either SLM or LLM based method.

In this work, we propose to implant LLM's knowledge into SLM based methods such that we can stick to both types of models' strengths. To this end, we develop a reading comprehension (RC) tree to transfer knowledge between two models. Specifically, we first construct the RC tree, from an extensive to intensive reading perspective, to capture the local and global information in the text. We then model samples encoded by SLM and knowledge extracted from LLM as two distributions using the constructed RT tree. We finally transfer knowledge via optimal transportation between two distributions. Extensive experiments prove the effectiveness of our method on real-world and machine-generated datasets. Our code and data are available at https://github.com/ khk-abc/toxic-detection.

Ask Lune

Know this paper in minutes, not an afternoon.

Lune can read every page of it and the top-tier papers that cite it, and gives you its claims, its evidence and its limits, each one cited to the paper it comes from. No skimming, no AI slop.

1 top-tier paper has cited it since. Know what they found before your reviewers do.

Questions to start from

Lune can draw on

the full paper · the top-tier papers that cite it · related work

Get the answer

Free to start. No credit card required.

Lineage

  • Topic-Aware Multi-turn Dialogue Modeling cites Retrospective Reader for Machine Reading Comprehension
  • Retrospective Reader for Machine Reading Comprehension cites Topic-Aware Multi-turn Dialogue Modeling
Builds onCited byLeft to right by year; the more cited, the higher.

Cited by top-tier papers1

  1. Evaluation and Facilitation of Online Discussions in the LLM Era: A SurveyKaterina Korre, Dimitris Tsirmpas, Nikos Gkoumas, Emma Cabalé et al.EMNLP 2025

Builds on17

  1. Towards Understanding and Mitigating Social Biases in Language ModelsPaul Pu Liang, Chiyu Wu, Louis-Philippe Morency, Ruslan SalakhutdinovICML 2021 · 495 citations
  2. Retrospective Reader for Machine Reading ComprehensionZhuosheng Zhang, Junjie Yang, Hai ZhaoAAAI 2021 · 237 citations
  3. Topic-Aware Multi-turn Dialogue ModelingYi Xu, Hai Zhao, Zhuosheng ZhangAAAI 2021 · 93 citations
  4. You Only Prompt Once: On the Capabilities of Prompt Learning on Large Language Models to Tackle Toxic ContentXinlei He, Savvas Zannettou, Yun Shen, Yang ZhangS&P 2024 · 74 citations
  5. WinoQueer: A Community-in-the-Loop Benchmark for Anti-LGBTQ+ Bias in Large Language ModelsVirginia K. Felkner, Ho-Chun Herbert Chang, Eugene Jang, Jonathan MayACL 2023 · 46 citations

Related papers

  1. Enhancing LLM-based Hatred and Toxicity Detection with Meta-Toxic Knowledge GraphYibo Zhao, Jiapeng Zhu, Can Xu, Yao Liu et al.ACL Findings 2025 · 8 citations
  2. Just KIDDIN' : Knowledge Infusion and Distillation for Detection of INdecent MemesRahul Garg, Trilok Padhi, Hemang Jain, Ugur Kursuncu et al.ACL Findings 2025 · 2 citations
  3. Seeking Neural Nuggets: Knowledge Transfer in Large Language Models from a Parametric PerspectiveMing Zhong, Chenxin An, Weizhu Chen, Jiawei Han et al.ICLR 2024 · 16 citations
  4. From the Detection of Toxic Spans in Online Discussions to the Analysis of Toxic-to-Civil TransferJohn Pavlopoulos, Léo Laugier, Alexandros Xenos, Jeffrey Sorensen et al.ACL 2022 · 35 citations
  5. Unveiling the Implicit Toxicity in Large Language ModelsJiaxin Wen, Pei Ke, Hao Sun, Zhexin Zhang et al.EMNLP 2023 · 21 citations
Dusk over the sea between two cliffs drawn in fine vertical lines