Opportunities and Challenges in Repeated Revisions to Pull-Requests: An Empirical Study

Background: The Pull-Request (PR) model is a widespread approach adopted by open source software (OSS) projects to support collaborative software development. However, it is often challenging to continuously evaluate and revise PRs in several iterations of code reviews involving technical and social aspects. Aim: Our objective is twofold: identifying best practices for effective collaboration in continuous PR improvement and uncovering problems that deserve special attention to improve collaboration efficiency and productivity. Method: We conducted a mixed-methods empirical study of repeatedly revised PRs (i.e., those that have undergone a high number of revisions). Historical trace data of five long-lived popular GitHub projects were used for manual investigation of practices for requesting changes to PRs and reasons for nonacceptance of repeatedly revised PRs. Surveys of OSS practitioners were conducted to evaluate the results of manual analysis and to provide additional insights into developers' willingness regarding PR revisions and factors causing avoidable revisions in practice. Results: The main results of our research were as follows: (1) We identified 15 code review practices for requesting changes to PRs, among which practices with respect to explaining the reasoning behind requested changes and tracking the progress of PR review and revision were undervalued by reviewers; (2) While submitters can in general undergo 1-5 rounds of revisions, they are willing to offer more revisions when they are in a friendly community and receive helpful feedback; (3) We revealed 11 factors causing avoidable revisions regarding to reviewers' feedback, code review policy, pre-submission issues, and implementation of new revisions; and (4) Nonacceptance of repeatedly revised PRs was due mainly to inactivity of submitters or reviewers and being superseded for better maintenance. Finally, based on these findings, we proposed recommendations and implications for OSS practitioners and tool designers to facilitate efficient collaboration in PR revisions.

CCS Concepts: • Software and its engineering → Collaboration in software development.