PRIORITIZING PULL REQUESTS THROUGH DUAL PREDICTION OF ACCEPTANCE AND INTEGRATOR RESPONSE

Abstract

Modern open-source software development involves lots of incoming pull requests that jam the code review and integration process. Maintainers rarely have any actionable visibility around which PRs are likely to be accepted otherwise or need more input - time that can mean the difference between a contributor sticking around and the project growing stale. To tackle this problem, we propose AR-Prioritizer which is a data-driven method that models pull request acceptance and integrator response likelihood simultaneously via machine learning. Trained on a gold enriched set of 41 trending GitHub projects (consisting of more than 278K pull requests), our approach utilizes two XGBoost classifiers, whose results are combined through an exponential value function to result in an integrated prioritization score. We achieve high predictive performance (AUC of 0.919 for both tasks) that is better than previous methods. AR-Prioritizer, which is available as an opensource command line tool, is evaluated via live deployment on popular repositories such as tensorflow, joomla-cms and PyGithub and a developer survey (n = 10). Of those surveyed, 66% said the tool helped them find high-impact pull requests, while 48% said they were able to quantify time savings in their review process. We show that jointly modeling acceptance and responsiveness leads to more actionable, human-oriented prioritization of work in collaborative software development. All data and models, together with source code are provided for reproducibilityand future studies.