Selective Preference Aggregation

Many applications in machine learning and decision making rely on procedures to aggregate human preferences. In such tasks, individuals express ordinal preferences over a set of items by voting, ratings, or comparing them. We then aggregate these data into a ranking that reveals their collective preferences. Standard methods for preference aggregation are designed to return rankings that arbitrate conflicting preferences between individuals. In this work, we introduce a paradigm for selective aggregation where we abstain from comparison rather than arbitrate dissent. We summarize collective preferences as a selective ranking -i.e., a partial order that reflects all collective preferences where at least 100 • (1 -τ )% of individuals agree. We develop algorithms to build selective rankings that achieve all possible tradeoffs between comparability and disagreement, and derive formal guarantees on their recovery and robustness. We conduct an extensive set of experiments on real-world datasets to benchmark our approach and demonstrate its functionality. Our results show how selective rankings can promote transparency and robustness by revealing disagreement and abstaining from arbitration.