Exploring the industry's challenges in software testing: An empirical study

Abstract Context Software testing is an important and costly software engineering activity in the industry. Despite the efforts of the software testing research community in the last several decades, various studies show that still many practitioners in the industry report challenges in their software testing tasks. Objective To shed light on industry's challenges in software testing, we characterize and synthesize the challenges reported by practitioners. Such concrete challenges can then be used for a variety of purposes, eg, research collaborations between industry and academia. Method Our empirical research method is opinion survey. By designing an online survey, we solicited practitioners' opinions about their challenges in different testing activities. Our dataset includes data from 72 practitioners from eight different countries. Results Our results show that test management and test automation are considered the most challenging among all testing activities by practitioners. Our results also include a set of 104 concrete challenges in software testing that may need further investigations by the research community. Conclusion We conclude that the focal points of industrial work and academic research in software testing differ. Furthermore, the paper at hand provides valuable insights concerning practitioners' “pain” points and, thus, provides researchers with a source of important research topics of high practical relevance.