R-Fairness: Assessing Fairness of Ranking in Subjective Data

Subjective data, reflecting individual opinions, permeate collaborative rating platforms like Yelp and Amazon, influencing everyday decisions. Despite the prevalence of such platforms, little attention has been given to fairness in their context, where groups of reviewers writing best-ranked reviews for best-ranked items have more influence on users' behavior. In this paper, we design and evaluate a new framework for the assessment of fairness of rankings for different reviewer groups in collaborative rating platforms. The key contributions are evaluating group exposure for different queries and platforms and comparing how various fairness definitions behave in different settings. Experiments on real datasets reveal insights into the impact of item ranking on fairness computation and the varying robustness of these measures.