Since its inception in 2004, the MSR landscape profoundly changed. The availability of large, diverse, and heterogenous forges and repositories, as well as the progress of technology (including, above all, the vast adoption of generative Artificial Intelligence in several pieces of research) have big implications on how we conduct empirical studies and how we evaluate our novel methods and tools. One of the consequences we observe is that publication standards are “apparently” increasing, requiring larger and larger evaluations like never before. Would this be necessary (and sufficient) to increase the significance, generalizability and impactfulness of research in the future? In this talk, I provide my view on how empirical evaluation is changing with the field evolution, the open challenges, and how ongoing research is trying to tackle such challenges.

Massimiliano Di Penta is a full professor at the University of Sannio, Italy. His research interests include software evolution, software analytics, recommender systems for software engineering, DevOps, and software engineering with/for AI. He is author of over 260 papers that appeared in international journals, conferences, and workshops. He has received several awards for research and service, including four ACM SIGSOFT Distinguished Paper awards. Most importantly, he has received several distinguished reviewer awards. He serves and has served in the organizing and program committees of more than 100 conferences, including ICSE, FSE, ASE, and ICSME. He has been program co-chair of ICSE 2023, ESEC/FSE 2021, ASE 2017, and several other conferences. He is associate editor-in-chief of IEEE Transactions on Software Engineering, and co-editor-in-chief of the Journal of Software: Evolution and Processes edited by Wiley. He is an editorial board member of the Empirical Software Engineering Journal edited by Springer. He has served on the editorial board of the ACM Transactions on Software Engineering and Methodology and IEEE Transactions on Software Engineering.