MSR 2025
Mon 28 - Tue 29 April 2025 Ottawa, Ontario, Canada
co-located with ICSE 2025
Mon 28 Apr 2025 16:30 - 16:40 at 215 - Software evolution and analysis Chair(s): Mauricio Verano Merino

Obsolete and redundant tests increase regression testing costs. Therefore, developers should remove them from test suites; however, identifying these tests is non-trivial. Automated techniques for identifying obsolete and redundant tests could help developers reduce regression testing costs. Nonetheless, we have limited empirical evidence of how and why developers delete tests. Therefore, in this work, we first create DelTest, a dataset of 24,431 manually confirmed deleted tests, by analyzing 449,592 commits from seven open-source Java projects. We then perform an empirical study on DelTest to understand test deletion. Our findings show that test deletion frequency and the number of deleted tests vary significantly across projects, suggesting that test deletion is more likely driven by the project-specific needs than the broader development cycle. Developers delete only one or two tests in most commits, suggesting test deletion is mostly small and incremental. In DelTest, 83.2% of tests are deleted along with the corresponding test classes, while 16.8% are deleted individually. We find that 91.4% of deleted tests in six projects are obsolete tests (i.e., production code is deleted), 7% are redundant tests (i.e., passing tests), and 1.6% are failing tests. The deletion of 20% of redundant tests reduces code coverage or mutation scores. We also evaluate test suite reduction (TSR) approaches on DelTest and find that a TSR approach identifies up to 54% of the redundant tests. Our findings can help improve automated techniques for identifying obsolete and redundant tests.

Mon 28 Apr

Displayed time zone: Eastern Time (US & Canada) change

16:00 - 17:30
Software evolution and analysisData and Tool Showcase Track / Technical Papers / Industry Track at 215
Chair(s): Mauricio Verano Merino Vrije Universiteit Amsterdam
16:00
10m
Talk
50 Years of Programming Language Evolution through the Software Heritage looking glass
Technical Papers
Adèle Desmazières Sorbonne Unversité, Roberto Di Cosmo Inria, France / University of Paris Diderot, France, Valentin Lorentz Inria Foundation
Pre-print
16:10
10m
Talk
It Works (only) on My Machine: A Study on Reproducibility Smells in Ansible Scripts
Technical Papers
Ghazal Sobhani Dalhousie University, Israat Haque Dalhousie University, Tushar Sharma Dalhousie University
Pre-print
16:20
10m
Talk
Are the Majority of Public Computational Notebooks Pathologically Non-Executable?
Technical Papers
Waris Gill Virginia Tech, Muhammad Ali Gulzar Virginia Tech, Tien Nguyen Virginia Tech
Pre-print
16:30
10m
Talk
Understanding Test Deletion in Java Applications
Technical Papers
Suraj Bhatta North Dakota State University, Frank Kendemah North Dakota State University, Ajay Jha North Dakota State University
Pre-print
16:40
10m
Talk
A Public Benchmark of REST APIs
Technical Papers
Alix Decrop University of Namur, Sara Eraso University of Valle, Xavier Devroey University of Namur, Gilles Perrouin Fonds de la Recherche Scientifique - FNRS & University of Namur
Pre-print
16:50
5m
Talk
What Do Contribution Guidelines Say About Software Testing?
Technical Papers
Pre-print
16:55
5m
Talk
Measuring InnerSource Value
Industry Track
17:00
5m
Talk
CoUpJava: A Dataset of Code Upgrade Histories in Open-Source Java Repositories
Data and Tool Showcase Track
Kaihang Jiang University of Waterloo, Bihui Jin University of Waterloo, Pengyu Nie University of Waterloo
17:05
5m
Talk
EvoChain: A Framework for Tracking and Visualizing Smart Contract Evolution
Data and Tool Showcase Track
Ilham Qasse Reykjavik University, Mohammad Hamdaqa Polytechnique Montréal, Björn Þór Jónsson Reykjavik University
17:10
5m
Talk
CoDocBench: A Dataset for Code-Documentation Alignment in Software Maintenance
Data and Tool Showcase Track
Kunal Suresh Pai UC Davis, Prem Devanbu University of California at Davis, Toufique Ahmed IBM Research
Pre-print
17:15
5m
Talk
RefExpo: Unveiling Software Project Structures through Advanced Dependency Graph Extraction
Data and Tool Showcase Track
Vahid Haratian Bilkent Univeristy, Pouria Derakhshanfar JetBrains Research, Vladimir Kovalenko JetBrains Research, Eray Tüzün Bilkent University
17:20
5m
Talk
HyperAST: Incrementally Mining Large Source Code Repositories
Data and Tool Showcase Track
Quentin Le Dilavrec TU Delft, Netherlands, Andy Zaidman Delft University of Technology
Pre-print
Hide past events