Towards Fair, Equitable, and Efficient Peer Review

Peer review is the backbone of academia. The rapid growth of the number of submissions to leading publication venues has identified a need for automation of some parts of the peerreview pipeline and nowadays human referees are required to interact with various interfaces and technologies in this process. However, there exists evidence that if such interactions are not carefully designed, they can exacerbate various problems related to fairness and efficiency of the process. In my research, I aim to design a Human-AI collaboration pipeline in peer review to mitigate these issues and ensure that science progresses in a fair, equitable, and efficient manner. Despite peer review being the primary mechanism of science dissemination for decades, the rapid growth of the number of submissions to leading AI and ML conferences has challenged its sustainability in two ways: • It has brought up a call for automated tools to assist human decision-makers. • It has amplified the shortcomings of the peer-review procedure, making them more visible to the community and stressing the importance of research on peer review. These issues motivate my thesis research and I am passionate about working at the intersection of machine learning, operations research, social choice theory, and humancomputer interaction, to understand and develop a principled approach towards scientific peer review. Specifically, I believe that a carefully designed Human-AI collaboration is crucial for sustainability of peer review and in my work I aim at designing tools to support this collaboration. My research touches both algorithmic and human sides of the Human-AI collaboration and in the sequel I first describe my projects on supporting each of these sides. I then outline a direction for future work on bringing these sides to a closer interaction with a goal of improving the peer-review process. On a higher level, my work comprises novel theoretical and empirical contributions: I aim to design practical algorithms that are supported by strong theoretical guarantees and are evaluated in a carefully designed real-world experiments. The preliminary results I discuss below have already had a considerable impact in practice with some tools deployed in ICML 2020, and this inspires me to continue my work towards fair, equitable and efficient peer review.