ViHateT5: Enhancing Hate Speech Detection in Vietnamese With a Unified Text-to-Text Transformer Model

Recent advancements in hate speech detection (HSD) in Vietnamese have made significant progress, primarily attributed to the emergence of transformer-based pre-trained language models, particularly those built on the BERT architecture. However, the necessity for specialized fine-tuned models has resulted in the complexity and fragmentation of developing a multitasking HSD system. Moreover, most current methodologies focus on fine-tuning general pre-trained models, primarily trained on formal textual datasets like Wikipedia, which may not accurately capture human behavior on online platforms. In this research, we introduce VIHATET5, a T5-based model pre-trained on our proposed large-scale domain-specific dataset named VOZ-HSD. By harnessing the power of a text-to-text architecture, VIHATET5 can tackle multiple tasks using a unified model and obtain state-of-the-art performance on all benchmark HSD datasets in Vietnamese. The experiments also underscore the significance of label distribution in pretraining data on model efficacy. We provide our experimental materials for research purposes, including the VOZ-HSD dataset 1 , pretrained checkpoint 2 , the unified HSD-multitask VIHATET5 model 3 , and related source code on GitHub 4 publicly. Warning: This paper contains examples from actual content on social media platforms that could be considered toxic and offensive.