Exploring the Impact of GitHub Actions on Pull Request Reviews in Machine Learning Projects

Continuous Integration (CI) tools like GitHub Actions were originally designed to streamline development workflows in traditional software systems by automating tasks such as building and testing, which has proven beneficial in improving review efficiency. However, ML projects present additional complexities—such as non-determinism, challenging testing processes, and longer build durations—that may limit the effectiveness of CI in supporting efficient reviews. Given these unique challenges, it is essential to reassess how CI tools impact the review process within ML contexts. This study empirically investigates the impact of GitHub Actions on PR review dynamics across 55 GitHub-based ML projects, focusing on metrics such as time to close a PR (i.e., PR latency), PR churn, comments, and PR submission frequency. Using a Regression Discontinuity Design (RDD), we analyze PR data from 12 months before and after the adoption of GitHub Actions. Our results show that GitHub Actions does not significantly reduce PR review times in ML projects, with factors such as PR churn and backlog size playing a larger role in influencing review efficiency. Additionally, rejected PRs were characterized by higher churn and more extensive discussions. These findings suggest that, while CI tools automate repetitive tasks and reduce manual workload, they may not fully address the unique demands of ML project reviews. We provide practical recommendations to enhance review efficiency in ML workflows, including strategies for incremental PR submissions and optimized backlog management.