Mining Bug Repositories for Multi-Fault Programs
Datasets such as Defects4J and BugsInPy that contain bugs from real-world software projects are necessary for a realistic evaluation of automated debugging tools. However, these datasets largely identify only a single bug in each entry, while real-world software projects (including those used in Defects4J and BugsInPy) typically contain multiple bugs at the same time. We lift this limitation and describe an automated approach to identify multiple existing bugs in the individual dataset entries. We use test case transplantation and fault location translation to expose and locate the bugs, respectively. We identified 9.2 faults on average in each of the 311 versions of the 5 projects in Defects4J, and 18.6 faults in 501 versions of the 17 projects in BugsInPy. We thus provide datasets of true multi-fault versions within real-world software projects, which maintain the properties, format and usability of the original datasets.