MSR 2025
Mon 28 - Tue 29 April 2025 Ottawa, Ontario, Canada
co-located with ICSE 2025
Mon 28 Apr 2025 11:50 - 12:00 at 214 - Defects, bugs, and issues Chair(s): Minhaz Zibran

For software testing research, Defects4J stands out as the primary benchmark dataset, offering a controlled environment to study real bugs from prominent open-source systems. While Defects4J provides a clean and valuable dataset, our goal is to explore how fault localization techniques perform under less-controlled development scenarios. In this paper, we revisited Defects4J to study the changes that developers made to fault-triggering tests after the bugs were reported/fixed. We aim to introduce a new evaluation scenario within Defects4J, focusing on the implications of regression tests and test changes added after the bug was fixed. We analyze when these tests were modified relative to bug report creation and examine SBFL performance in less-controlled settings. Our findings show that 1) 55% of the fault-triggering tests were added to replicate the bug or test for regression; 2) 22% of the tests were changed after the bug reports, incorporating information related to the bug; 3) developers often update tests with new assertions or changes to match source code updates; and 4) SBFL performance differs significantly in less-controlled settings (down by at most 415% for Mean First Rank). Our study points out the diverse development scenarios in the studied bugs, highlighting new settings for future SBFL evaluations and bug benchmarks.

Mon 28 Apr

Displayed time zone: Eastern Time (US & Canada) change

11:00 - 12:30
Defects, bugs, and issuesData and Tool Showcase Track / Technical Papers / Registered Reports at 214
Chair(s): Minhaz Zibran Idaho State University
11:00
10m
Talk
Learning from Mistakes: Understanding Ad-hoc Logs through Analyzing Accidental Commits
Technical Papers
Yi-Hung Chou University of California, Irvine, Yiyang Min Amazon, April Wang ETH Zürich, James Jones University of California at Irvine
Pre-print
11:10
10m
Talk
On the calibration of Just-in-time Defect Prediction
Technical Papers
Xhulja Shahini paluno - University of Duisburg-Essen, Jone Bartel University of Duisburg-Essen, paluno, Klaus Pohl University of Duisburg-Essen, paluno
11:20
10m
Talk
An Empirical Study on Leveraging Images in Automated Bug Report Reproduction
Technical Papers
Dingbang Wang University of Connecticut, Zhaoxu Zhang University of Southern California, Sidong Feng Monash University, William G.J. Halfond University of Southern California, Tingting Yu University of Connecticut
11:30
10m
Talk
It’s About Time: An Empirical Study of Date and Time Bugs in Open-Source Python Software
Technical Papers
Shrey Tiwari Carnegie Mellon University, Serena Chen University of California, San Diego, Alexander Joukov Stony Brook University, Peter Vandervelde University of California, Santa Barbara, Ao Li Carnegie Mellon University, Rohan Padhye Carnegie Mellon University
11:40
10m
Talk
Enhancing Just-In-Time Defect Prediction Models with Developer-Centric Features
Technical Papers
Emanuela Guglielmi University of Molise, Andrea D'Aguanno University of Molise, Rocco Oliveto University of Molise, Simone Scalabrino University of Molise
11:50
10m
Talk
Revisiting Defects4J for Fault Localization in Diverse Development Scenarios
Technical Papers
Md Nakhla Rafi Concordia University, An Ran Chen University of Alberta, Tse-Hsun (Peter) Chen Concordia University, Shaohua Wang Central University of Finance and Economics
12:00
5m
Talk
Mining Bug Repositories for Multi-Fault Programs
Data and Tool Showcase Track
Dylan Callaghan Stellenbosch University, Bernd Fischer Stellenbosch University
12:05
5m
Talk
HaPy-Bug - Human Annotated Python Bug Resolution Dataset
Data and Tool Showcase Track
Piotr Przymus Nicolaus Copernicus University in Toruń, Poland, Mikołaj Fejzer Nicolaus Copernicus University in Toruń, Jakub Narębski Nicolaus Copernicus University in Toruń, Radosław Woźniak Nicolaus Copernicus University in Toruń, Łukasz Halada University of Wrocław, Poland, Aleksander Kazecki Nicolaus Copernicus University in Toruń, Mykhailo Molchanov Igor Sikorsky Kyiv Polytechnic Institute, Ukraine, Krzysztof Stencel University of Warsaw
Pre-print
12:10
5m
Talk
SPRINT: An Assistant for Issue Report Management
Data and Tool Showcase Track
Ahmed Adnan , Antu Saha William & Mary, Oscar Chaparro William & Mary
Pre-print
12:15
5m
Talk
Identifying and Replicating Code Patterns Driving Performance Regressions in Software Systems
Registered Reports
Denivan Campos University of Molise, Luana Martins University of Salerno, Emanuela Guglielmi University of Molise, Michele Tucci University of L'Aquila, Daniele Di Pompeo University of L'Aquila, Simone Scalabrino University of Molise, Vittorio Cortellessa University of L'Aquila, Dario Di Nucci University of Salerno, Rocco Oliveto University of Molise