The Cost vs the Benefit of Adding an Extra Code Reviewer to Mitigate Developer Turnover through Reviewer Recommenders

Developer turnover is inevitable in software projects, leading to knowledge loss, reduced productivity, and increased defects. Prior works have quantified the knowledge distribution ability of reviewer recommenders at the replacement level. However, they did not consider adding reviewers. In this work, we target pull requests containing abandoned or hoarded files, which are more prone to significant knowledge loss from developer turnover, i.e. Pull Requests with Knowledge at Risk to Turnover (PRKRs). On PRKRs, we know that the knowledge is not distributed across the team. In this study, we aim to suggest an extra reviewer for PRKRs to spread their knowledge more broadly among the developers while limiting the additional work for the development team. Our most aggressive recommender is TurnoverRec++, which adds an extra reviewer to all PRKRs. This recommender achieves an average reduction of -82% in turnover risk, however, there is a significant cost, with extra reviewers being added to an average 16.6% of all pull requests. Our simulations show a tight trade-off between reducing the number of files at risk to turnover and the extra reviews. We also show that this trade-off is non-linear, and it is possible to significantly decrease the turnover risk with relatively few extra reviews. For example, the novel recommender, FarAwareRec, reduces files at risk to turnover by -42% but only adds an extra reviewer to 1.9%. Ultimately, our results show that managers should determine which strategy best suits their situation by considering factors such as turnover rates and available budgets. Our scripts and data are available in our replication package [17].