Task-Aware Preference Calibration for Direct Preference Optimization

Food safety is a critical global concern, and automating the detection of food hazards from recall reports can improve public health monitoring and regulatory compliance. This paper presents our submission for SemEval-2025 Task 9: The Food Hazard Detection Challenge. We tackle the inherent class imbalance in this task by leveraging advanced data augmentation techniques, including LLM-based synthetic data generation, synonym replacement, and back-translation. We employ transformer-based models such as DistilBERT, fine-tuned with these augmented datasets, to enhance performance. Our system achieves significant improvements, obtaining a Macro-F1 score of 0.7882 in ST1 and 0.5099 in ST2. 1 Additionally, we analyze the impact of augmentation strategies and compare multiple architectures, highlighting challenges in handling implicit hazards. Our findings underscore the effectiveness of LLM-based augmentation in addressing extreme class imbalance while demonstrating the strengths and limitations of transformer models in food safety applications. 1 Our Code: https://github.com/msaadg/hu_ semeval_task9 2025). This task is crucial for enhancing food security and public health interventions. It consists of two subtasks: • ST1: Classifying the hazard category and product category. • ST2: Identifying the exact hazard and exact product mentioned in the report.