Learning to characterize performance regression introducing code changes

For many software projects, performance is a critical requirement. Such projects often create performance tests alongside their other tests which specifically check for performance issues. Detecting code changes responsible for performance regression is challenging due to the increasing number of daily committed changes. And so, it is hard to individually monitor these changes while being committed to the main branch. Existing studies try to solve this issue by predicting whether or not a commit will cause a performance regression to occur. The aim is that by creating a system that can successfully evaluate the need to run the performance tests on a change, we can reduce the number of times we run the tests, and thus save cost. In this paper, we propose an approach that directs testing to commits causing performance regression by building classifiers. The approach is addressing the severity of class imbalance any project could have due to the fact that code changes introducing regression are rare but costly. We found that Decision Forest with Random Over Sampling provides better detection for code changes introducing regression without hindering testing team by conducting unnecessary tests. With the aid of resampling techniques, the detection rate of code changes introducing regression improved by about 50% from traditional classifiers.