Context-based Collective Preference Aggregation for Prioritizing Crowd Opinions in Social Decision-making

Given a social issue that needs to be solved, decision-makers need to listen to the crowd opinions and preferences. However, existing online voting systems with limited capabilities cannot conduct such investigations. Our idea is that decision-makers can collect many human opinions from crowds on the web and then prioritize them for social decision-making. A solution of the prioritization entails collecting a large amount of pairwise preference comparisons from crowds and utilizing the aggregated preference labels as the collective preferences on the opinions. In practice, because there is a large number of combinations of all candidate opinion pairs, we can only collect a small number of labels for a small subset of pairs. How to utilize only a small number of pairwise crowd preferences on the opinions to estimate collective preferences is the problem. Existing works on preference aggregation methods for general scenarios utilize only the pairwise preference labels. In our scenario, additional contextual information, such as the text contents of the opinions, can potentially promote the aggregation performance. Therefore, we propose preference aggregation approaches that can effectively incorporate contextual information by externally or internally building the relations between the opinion contexts and preference scores. We propose approaches for both the homogeneous and heterogeneous settings of modeling the evaluators. The experiments conducted on real datasets collected from real-world crowdsourcing platform show that our approaches can generate better aggregation results than the baselines for estimating collective preferences, especially when there are only a small number of preference labels available.