Contextual Interaction for Argument Post Quality Assessment

Recently, there has been an increased emphasis on assessing the quality of natural language arguments. Existing approaches primarily focus on evaluating the quality of individual argument posts. However, they often fall short when it comes to effectively distinguishing arguments that possess a narrow quality margin. To address this limitation, this paper delves into two alternative methods for modeling the relative quality of different arguments: 1) Supervised contrastive learning that captures the intricate interactions between arguments. By incorporating this approach, we aim to enhance the assessment of argument quality by effectively distinguishing between arguments with subtle differences. 2) Large language models (LLMs) with in-context examples that harness the power of LLMs and enrich them with incontext demonstration. Through extensive evaluation and analysis on the publicly available IBM-Rank-30k dataset, we demonstrate the superiority of our contrastive interaction approach over state-of-the-art baselines. On the other hand, while LLMs with in-context examples demonstrate a commendable ability to identify high-quality argument posts, they exhibit relatively limited effectiveness in quantifying argument quality and distinguishing between arguments with a narrow quality gap. Code is available at https://github.com/ucasYW/Contextual-Interaction-for-AQA .