A One-Size-Fits-All Approach to Improving Randomness in Paper Assignment
Yixuan Even Xu, Steven Jecmen, Zimeng Song, Fei Fang
Abstract
The assignment of papers to reviewers is a crucial part of the peer review processes of large publication venues, where organizers (e.g., conference program chairs) rely on algorithms to perform automated paper assignment. As such, a major challenge for the organizers of these processes is to specify paper assignment algorithms that find appropriate assignments with respect to various desiderata. Although the main objective when choosing a good paper assignment is to maximize the expertise of each reviewer for their assigned papers, several other considerations make introducing randomization into the paper assignment desirable: robustness to malicious behavior, the ability to evaluate alternative paper assignments, reviewer diversity, and reviewer anonymity. However, it is unclear in what way one should randomize the paper assignment in order to best satisfy all of these considerations simultaneously. In this work, we present a practical, one-size-fits-all method for randomized paper assignment intended to perform well across different motivations for randomness. We show theoretically and experimentally that our method outperforms currently-deployed methods for randomized paper assignment on several intuitive randomness metrics, demonstrating that the randomized assignments produced by our method are general-purpose.
Ask about this paper
Your agent reads all of it.
Lune indexed this paper to the last equation, along with the top-tier papers that cite it. Ask a question and the answer quotes them.
Your agent calls
Luneget_paper_fulltext
Free to start. No credit card required.
Terminal
Install the CLIlune papers fulltext 1dae1de7-e611-4a56-b35b-9fc90cd81d4bCited by top-tier papers2
- AgentReview: Exploring Peer Review Dynamics with LLM AgentsYiqiao Jin, Qinlin Zhao, Yiyang Wang, Hao Chen et al.EMNLP 2024 · 24 citations
- Vulnerability of Text-Matching in ML/AI Conference Reviewer Assignments to CollusionsJhih-Yi Hsieh, Aditi Raghunathan, Nihar B. ShahUSENIX Security 2025
Builds on4
- Mitigating Manipulation in Peer Review via Randomized Reviewer AssignmentsSteven Jecmen, Hanrui Zhang, Ryan Liu, Nihar B. Shah et al.NeurIPS 2020 · 90 citations
- Making Paper Reviewing Robust to Bid Manipulation AttacksRuihan Wu, Chuan Guo, Felix Wu, Rahul Kidambi et al.ICML 2021 · 27 citations
- A Dataset on Malicious Paper Bidding in Peer ReviewSteven Jecmen, Minji Yoon, Vincent Conitzer, Nihar B. Shah et al.WWW 2023 · 22 citations
- Combating Collusion Rings Is Hard but PossibleNiclas Boehmer, Robert Bredereck, André NichterleinAAAI 2022 · 14 citations
Related papers
- Counterfactual Evaluation of Peer-Review Assignment PoliciesMartin Saveski, Steven Jecmen, Nihar B. Shah, Johan UganderNeurIPS 2023 · 19 citations
- No more Reviewer #2: Subverting Automatic Paper-Reviewer Assignment using Adversarial LearningThorsten Eisenhofer, Erwin Quiring, Jonas Möller, Doreen Riepel et al.USENIX Security 2023
- "Flawed, but like democracy we don't have a better system": The Experts' Insights on the Peer Review Process of Evaluating Security PapersAnanta Soneji, Faris Bugra Kokulu, Carlos E. Rubio-Medrano, Tiffany Bao et al.S&P 2022 · 17 citations
- A Market-Inspired Bidding Scheme for Peer Review Paper AssignmentReshef Meir, Jérôme Lang, Julien Lesca, Nicholas Mattei et al.AAAI 2021 · 33 citations
- A Principled Approach to Randomized Selection under Uncertainty: Applications to Peer Review and Grant FundingAlexander Goldberg, Giulia Fanti, Nihar B. ShahNeurIPS 2025 · 3 citations
