Agents for Software Development
Graham Neubig (Carnegie Mellon University)
Software is one of the most powerful tools that we humans have at our disposal; it allows a skilled programmer to interact with the world in complex and profound ways. However, at the same time software systems are complex, fragile, and even dangerous. Can we develop AI agents that help us develop real-world software, particularly in the context of real-world software development tasks in large software repositories, in all their complexity? In this tutorial I will discuss the state-of-the-art in software development agents, including challenges with respect to identifying which files to edit, how to edit them, how to test edits and recover, and how to train and evaluate models. In addition, I will address some challenges beyond simple writing code, such as how to process multimodal data, how to combine web browsing with coding, and how to perform data science tasks with software development models. I will provide examples from OpenHands, an open-source toolkit that implements many of the methods that I discuss: https://github.com/All-Hands-AI/OpenHands
Harmonized Coding with AI: LLMs for Qualitative Analysis in Software Engineering Research
Christoph Treude (Singapore Management University), Youmei Fan (Nara Institute of Science and Technology), Tao Xiao (Nara Institute of Science and Technology), and Hideaki Hata (Shinshu University)
Qualitative bottom-up coding is essential for identifying themes and patterns in complex data. This tutorial demonstrates how LLMs such as ChatGPT can support the qualitative coding process for software engineering research. Participants will walk through an entire coding exercise, learning to identify themes through open coding, consolidate themes by refining and merging codes, and conduct inter-rater agreement by standardizing codebooks and testing agreement between human and AI coders. Using hands-on exercises and real-world examples, this session highlights effective human-AI collaboration and strategies to ensure transparency, trustworthiness, and methodological rigor. Designed for researchers at all levels of experience, this tutorial equips participants with practical techniques to analyze software engineering data.
Dates
Tracks
This program is tentative and subject to change.
Mon 28 AprDisplayed time zone: Eastern Time (US & Canada) change
Mon 28 Apr
Displayed time zone: Eastern Time (US & Canada) change
11:00 - 12:30 | |||
11:00 10mTalk | Learning from Mistakes: Understanding Ad-hoc Logs through Analyzing Accidental Commits Technical Papers Yi-Hung Chou University of California, Irvine, Yiyang Min Amazon, April Wang ETH Zürich, James Jones University of California at Irvine Pre-print | ||
11:10 10mTalk | On the calibration of Just-in-time Defect Prediction Technical Papers Xhulja Shahini paluno - University of Duisburg-Essen, Jone Bartel University of Duisburg-Essen, paluno, Klaus Pohl University of Duisburg-Essen, paluno | ||
11:20 10mTalk | An Empirical Study on Leveraging Images in Automated Bug Report Reproduction Technical Papers Dingbang Wang University of Connecticut, Zhaoxu Zhang University of Southern California, Sidong Feng Monash University, William G.J. Halfond University of Southern California, Tingting Yu University of Connecticut | ||
11:30 10mTalk | It’s About Time: An Empirical Study of Date and Time Bugs in Open-Source Python Software Technical Papers Shrey Tiwari Carnegie Mellon University, Serena Chen University of California, San Diego, Alexander Joukov Stony Brook University, Peter Vandervelde University of California, Santa Barbara, Ao Li Carnegie Mellon University, Rohan Padhye Carnegie Mellon University | ||
11:40 10mTalk | Enhancing Just-In-Time Defect Prediction Models with Developer-Centric Features Technical Papers Emanuela Guglielmi University of Molise, Andrea D'Aguanno University of Molise, Rocco Oliveto University of Molise, Simone Scalabrino University of Molise | ||
11:50 10mTalk | Revisiting Defects4J for Fault Localization in Diverse Development Scenarios Technical Papers Md Nakhla Rafi Concordia University, An Ran Chen University of Alberta, Tse-Hsun (Peter) Chen Concordia University, Shaohua Wang Central University of Finance and Economics | ||
12:00 5mTalk | Mining Bug Repositories for Multi-Fault Programs Data and Tool Showcase Track | ||
12:05 5mTalk | HaPy-Bug - Human Annotated Python Bug Resolution Dataset Data and Tool Showcase Track Piotr Przymus Nicolaus Copernicus University in Toruń, Poland, Mikołaj Fejzer Nicolaus Copernicus University in Toruń, Jakub Narębski Nicolaus Copernicus University in Toruń, Radosław Woźniak Nicolaus Copernicus University in Toruń, Łukasz Halada University of Wrocław, Poland, Aleksander Kazecki Nicolaus Copernicus University in Toruń, Mykhailo Molchanov Igor Sikorsky Kyiv Polytechnic Institute, Ukraine, Krzysztof Stencel University of Warsaw | ||
12:10 5mTalk | SPRINT: An Assistant for Issue Report Management Data and Tool Showcase Track | ||
11:00 - 12:30 | |||
11:00 10mTalk | Wolves in the Repository: A Software Engineering Analysis of the XZ Utils Supply Chain Attack Technical Papers | ||
11:10 10mTalk | Software Composition Analysis and Supply Chain Security in Apache Projects: an Empirical Study Technical Papers Sabato Nocera University of Salerno, Sira Vegas Universidad Politecnica de Madrid, Giuseppe Scanniello University of Salerno, Natalia Juristo Universidad Politecnica de Madrid Pre-print | ||
11:20 10mTalk | Good practice versus reality: a landscape analysis of Research Software metadata adoption in European Open Science Clusters Technical Papers | ||
11:30 10mTalk | Towards Security Commit Message Standardization Technical Papers Sofia Reis Instituto Superior Técnico, U. Lisboa & INESC-ID, Rui Abreu INESC-ID; University of Porto, Corina Pasareanu CMU, NASA, KBR | ||
11:40 10mTalk | From Industrial Practices to Academia: Uncovering the Gap in Vulnerability Research and Practice Technical Papers | ||
11:50 5mTalk | Patch Me If You Can—Securing the Linux Kernel Industry Track Gunnar Kudrjavets Amazon Web Services, USA Pre-print | ||
11:55 5mTalk | OSS License Identification at Scale: A Comprehensive Dataset Using World of Code Data and Tool Showcase Track Mahmoud Jahanshahi Research Assistant, University of Tennessee Knoxville, David Reid University of Tennessee, Adam McDaniel University of Tennessee Knoxville, Audris Mockus The University of Tennessee | ||
12:00 5mTalk | SCRUBD: Smart Contracts Reentrancy and Unhandled Exceptions Vulnerability Dataset Data and Tool Showcase Track Chavhan Sujeet Yashavant Indian Institute of Technology, Kanpur, Mitrajsinh Chavda Indian Institute of Technology Kanpur, India, Saurabh Kumar Indian Institute of Technology Hyderabad, India, Amey Karkare IIT Kanpur, Angshuman Karmakar Indian Institute of Technology Kanpur, India | ||
12:05 5mTalk | ICVul: A Well-labeled C/C++ Vulnerability Dataset with Comprehensive Metadata and VCCs Data and Tool Showcase Track Chaomeng Lu DistriNet Group-T, KU Leuven, Tianyu Li DistriNet Group-T, KU Leuven, Toon Dehaene KU Leuven, Bert Lagaisse DistriNet Group-T, KU Leuven | ||
12:10 5mTalk | A Dataset of Software Bill of Materials for Evaluating SBOM Consumption Tools Data and Tool Showcase Track Rio Kishimoto Osaka University, Tetsuya Kanda Notre Dame Seishin University, Yuki Manabe The University of Fukuchiyama, Katsuro Inoue Nanzan University, Shi Qiu Toshiba, Yoshiki Higo Osaka University | ||
12:15 5mTalk | Wild SBOMs: a Large-scale Dataset of Software Bills of Materials from Public Code Data and Tool Showcase Track Luis Soeiro LTCI, Télécom Paris, Institut Polytechnique de Paris, Thomas Robert LTCI, Télécom Paris, Institut Polytechnique de Paris, Stefano Zacchiroli Télécom Paris, Polytechnic Institute of Paris | ||
12:20 5mTalk | MaLAware: Automating the Comprehension of Malicious Software Behaviours using Large Language Models (LLMs) Data and Tool Showcase Track BIKASH SAHA Indian Institute of Technology Kanpur, Nanda Rani Indian Institute of Technology Kanpur, Sandeep K. Shukla Indian Institute of Technology Kanpur | ||
14:00 - 15:30 | MSR 2025 Mining ChallengeMining Challenge at 215 Chair(s): Joyce El Haddad Université Paris Dauphine - PSL , Damien Jaime Université Paris Nanterre & LIP6, Pascal Poizat Université Paris Nanterre & LIP6 | ||
14:00 4mTalk | Analyzing Dependency Clusters and Security Risks in the Maven Central Repository Mining Challenge | ||
14:04 4mTalk | Chasing the Clock: How Fast Are Vulnerabilities Fixed in the Maven Ecosystem? Mining Challenge Md Fazle Rabbi Idaho State University, Arifa Islam Champa Idaho State University, Rajshakhar Paul Wayne State University, Minhaz F. Zibran Idaho State University | ||
14:08 4mTalk | Decoding Dependency Risks: A Quantitative Study of Vulnerabilities in the Maven Ecosystem Mining Challenge Costain Nachuma Idaho State University, Md Mosharaf Hossan Idaho State University, Asif Kamal Turzo Wayne State University, Minhaz F. Zibran Idaho State University | ||
14:12 4mTalk | Faster Releases, Fewer Risks: A Study on Maven Artifact Vulnerabilities and Lifecycle Management Mining Challenge Md Shafiullah Shafin Rajshahi University of Engineering & Technology (RUET), Md Fazle Rabbi Idaho State University, S. M. Mahedy Hasan Rajshahi University of Engineering & Technology, Minhaz F. Zibran Idaho State University | ||
14:16 4mTalk | Insights into Dependency Maintenance Trends in the Maven Ecosystem Mining Challenge Barisha Chowdhury Rajshahi University of Engineering & Technology, Md Fazle Rabbi Idaho State University, S. M. Mahedy Hasan Rajshahi University of Engineering & Technology, Minhaz F. Zibran Idaho State University | ||
14:20 4mTalk | Insights into Vulnerability Trends in Maven Artifacts: Recurrence, Popularity, and User Behavior Mining Challenge Courtney Bodily Idaho State University, Eric Hill Idaho State University, Andreas Kramer Idaho State University, Leslie Kerby Idaho State University, Minhaz F. Zibran Idaho State University | ||
14:24 4mTalk | Understanding Software Vulnerabilities in the Maven Ecosystem: Patterns, Timelines, and Risks Mining Challenge Md Fazle Rabbi Idaho State University, Rajshakhar Paul Wayne State University, Arifa Islam Champa Idaho State University, Minhaz F. Zibran Idaho State University | ||
14:28 4mTalk | Dependency Update Adoption Patterns in the Maven Software Ecosystem Mining Challenge Baltasar Berretta College of Wooster, Augustus Thomas College of Wooster, Heather Guarnera The College of Wooster | ||
14:32 4mTalk | Analyzing Vulnerability Overestimation in Software Projects Mining Challenge Taha Draoui University of Michigan-Flint, Faten Jebari University of Michigan-Flint, Chawki Ben Slimen University of Michigan-Flint, Munjaap Uppal University of Michigan-Flint, Mohamed Wiem Mkaouer University of Michigan - Flint | ||
14:36 4mTalk | Dependency Dilemmas: A Comparative Study of Independent and Dependent Artifacts in Maven Ecosystem Mining Challenge Mehedi Hasan Shanto Khulna University, Muhammad Asaduzzman University of Windsor, Manishankar Mondal Khulna University, Shaiful Chowdhury University of Manitoba | ||
14:40 4mTalk | Cascading Effects: Analyzing Project Failure Impact in the Maven Central Ecosystem Mining Challenge Mina Shehata Belmont University, Saidmakhmud Makhkamjonoov Belmont University, Mahad Syed Belmont University, Esteban Parra Belmont University | ||
14:45 4mTalk | Do Developers Depend on Deprecated Library Versions? A Mining Study of Log4j Mining Challenge Haruhiko Yoshioka Nara Institute of Science and Technology, Sila Lertbanjongngam Nara Institute of Science and Technology, Masayuki Inaba Nara Institute of Science and Technology, Youmei Fan Nara Institute of Science and Technology, Takashi Nakano Nara Institute of Science and Technology, Kazumasa Shimari Nara Institute of Science and Technology, Raula Gaikovina Kula Osaka University, Kenichi Matsumoto Nara Institute of Science and Technology | ||
14:49 4mTalk | Mining for Lags in Updating Critical Security Threats: A Case Study of Log4j Library Mining Challenge Hidetake Tanaka Nara Institute of Science and Technology, Kazuma Yamasaki Nara Institute of Science and Technology, Momoka Hirose Nara Institute of Science and Technology, Takashi Nakano Nara Institute of Science and Technology, Youmei Fan Nara Institute of Science and Technology, Kazumasa Shimari Nara Institute of Science and Technology, Raula Gaikovina Kula Osaka University, Kenichi Matsumoto Nara Institute of Science and Technology | ||
14:53 4mTalk | On the Evolution of Unused Dependencies in Java Project Releases: An Empirical Study Mining Challenge Nabhan Suwanachote Nara Institute of Science and Technology, Yagut Shakizada Nara Institute of Science and Technology, Yutaro Kashiwa Nara Institute of Science and Technology, Bin Lin Radboud University, Hajimu Iida Nara Institute of Science and Technology | ||
14:57 4mTalk | Out of Sight, Still at Risk: The Lifecycle of Transitive Vulnerabilities in Maven Mining Challenge Piotr Przymus Nicolaus Copernicus University in Toruń, Poland, Mikołaj Fejzer Nicolaus Copernicus University in Toruń, Jakub Narębski Nicolaus Copernicus University in Toruń, Krzysztof Rykaczewski Nicolaus Copernicus University in Toruń, Poland, Krzysztof Stencel University of Warsaw | ||
15:01 4mTalk | Popularity and Innovation in Maven Central Mining Challenge Nkiru Ede Victoria University of Wellington, Jens Dietrich Victoria University of Wellington, Ulrich Zülicke Victoria University of Wellington | ||
15:05 4mTalk | Software Bills of Materials in Maven Central Mining Challenge Yogya Gamage Universtité de Montréal, Nadia Gonzalez Fernandez Université de Montréal, Martin Monperrus KTH Royal Institute of Technology, Benoit Baudry | ||
15:09 4mTalk | The Ripple Effect of Vulnerabilities in Maven Central: Prevalence, Propagation, and Mitigation Challenges Mining Challenge | ||
15:13 4mTalk | Tracing Vulnerabilities in Maven: A Study of CVE lifecycles and Dependency Networks Mining Challenge Pre-print | ||
15:17 4mTalk | Understanding Abandonment and Slowdown Dynamics in the Maven Ecosystem Mining Challenge Kazi Amit Hasan Queen's University, Canada, Jerin Yasmin Queen's University, Canada, Huizi Hao Queen's University, Canada, Yuan Tian Queen's University, Kingston, Ontario, Safwat Hassan University of Toronto, Steven Ding Pre-print | ||
15:21 4mTalk | Characterizing Packages for Vulnerability Prediction Mining Challenge Saviour Owolabi University of Calgary, Francesco Rosati University of Calgary, Ahmad Abdellatif University of Calgary, Lorenzo De Carli University of Calgary, Canada | ||
15:25 4mTalk | Understanding the Popularity of Packages in Maven Ecosystem Mining Challenge Sadman Jashim Sakib University of Windsor, Muhammad Asaduzzaman University of Windsor, Curtis Bright University of Windsor, Cole Morgan University of Windsor | ||
Tue 29 AprDisplayed time zone: Eastern Time (US & Canada) change
Tue 29 Apr
Displayed time zone: Eastern Time (US & Canada) change
11:00 - 12:30 | |||
11:00 10mTalk | Build Scripts Need Maintenance Too: A Study on Refactoring and Technical Debt in Build Systems Technical Papers Anwar Ghammam Oakland University, Dhia Elhaq Rzig University of Michigan - Dearborn, Mohamed Almukhtar Oakland University, Rania Khalsi University of Michigan - Flint, Foyzul Hassan University of Michigan at Dearborn, Marouane Kessentini Grand Valley State University | ||
11:10 10mTalk | LLMSecConfig: An LLM-Based Approach for Fixing Software Container Misconfigurations Technical Papers Ziyang Ye The University of Adelaide, Triet Le The University of Adelaide, Muhammad Ali Babar School of Computer Science, The University of Adelaide | ||
11:20 10mTalk | How Do Infrastructure-as-Code Practitioners Update Their Dependencies? An Empirical Study on Terraform Module Updates Technical Papers Mahi Begoug , Ali Ouni ETS Montreal, University of Quebec, Moataz Chouchen Department of Electrical and Computer Engineering, Concordia University, Montreal, Canada | ||
11:30 5mTalk | TerraDS: A Dataset for Terraform HCL Programs Data and Tool Showcase Track Christoph Buehler University of St. Gallen, David Spielmann University of St. Gallen, Roland Meier armasuisse, Guido Salvaneschi University of St. Gallen | ||
11:35 5mTalk | OSPtrack: A Labeled Dataset Targeting Simulated Execution of Open-Source Software Data and Tool Showcase Track Zhuoran Tan University of Glasgow, Christos Anagnostopoulos University of Glasgow, Jeremy Singer University of Glasgow | ||
11:40 5mTalk | CARDS: A collection of package, revision, and miscelleneous dependency graphs Data and Tool Showcase Track Euxane TRAN-GIRARD LIGM, CNRS, Université Gustave Eiffel, Laurent BULTEAU LIGM, CNRS, Université Gustave Eiffel, Pierre-Yves DAVID Octobus S.c.o.p. Pre-print | ||
11:45 5mTalk | GHALogs: Large-scale dataset of GitHub Actions runs Data and Tool Showcase Track Florent Moriconi EURECOM, AMADEUS, Thomas Durieux TU Delft, Jean-Rémy Falleri Bordeaux INP, Raphaël Troncy EURECOM, Aurélien Francillon EURECOM | ||
14:00 - 15:30 | |||
14:00 10mTalk | Automatic High-Level Test Case Generation using Large Language Models Technical Papers Navid Bin Hasan Bangladesh University of Engineering and Technology, Md. Ashraful Islam Bangladesh University of Engineering and Technology, Junaed Younus Khan Bangladesh University of Engineering and Technology, Sanjida Senjik Bangladesh University of Engineering and Technology, Anindya Iqbal Bangladesh University of Engineering and Technology Dhaka, Bangladesh | ||
14:10 10mTalk | Prompting in the Wild: An Empirical Study of Prompt Evolution in Software Repositories Technical Papers Mahan Tafreshipour University of California at Irvine, Aaron Imani University of California, Irvine, Eric Huang University of California, Irvine, Eduardo Santana de Almeida Federal University of Bahia, Thomas Zimmermann University of California, Irvine, Iftekhar Ahmed University of California at Irvine Pre-print | ||
14:20 10mTalk | Towards Detecting Prompt Knowledge Gaps for Improved LLM-guided Issue Resolution Technical Papers Ramtin Ehsani Drexel University, Sakshi Pathak Drexel University, Preetha Chatterjee Drexel University, USA Pre-print | ||
14:30 10mTalk | Intelligent Semantic Matching (ISM) for Video Tutorial Search using Transformer Models Technical Papers | ||
14:40 10mTalk | Language Models in Software Development Tasks: An Experimental Analysis of Energy and Accuracy Technical Papers Negar Alizadeh Universiteit Utrecht, Boris Belchev University of Twente, Nishant Saurabh Utrecht University, Patricia Kelbert Fraunhofer IESE, Fernando Castor University of Twente | ||
14:50 10mTalk | TriGraph: A Probabilistic Subgraph-Based Model for Visual Code Completion in Pure Data Technical Papers Anisha Islam Department of Computing Science, University of Alberta, Abram Hindle University of Alberta | ||
15:00 5mTalk | Inferring Questions from Programming Screenshots Technical Papers Faiz Ahmed York University, Xuchen Tan York University, Folajinmi Adewole York University, Suprakash Datta York University, Maleknaz Nayebi York University | ||
15:05 5mTalk | Human-In-The-Loop Software Development Agents: Challenges and Future Directions Industry Track Jirat Pasuksmit Atlassian, Wannita Takerngsaksiri Monash University, Patanamon Thongtanunam University of Melbourne, Kla Tantithamthavorn Monash University, Ruixiong Zhang Atlassian, Shiyan Wang Atlassian, Fan Jiang Atlassian, Jing Li Atlassian, Evan Cook Atlassian, Kun Chen Atlassian, Ming Wu Atlassian | ||
15:10 5mTalk | FormalSpecCpp: A Dataset of C++ Formal Specifications Created Using LLMs Data and Tool Showcase Track Madhurima Chakraborty University of California, Riverside, Peter Pirkelbauer Lawrence Livermore National Laboratory, Qing Yi Lawrence Livermore National Laboratory | ||
Unscheduled Events
| Not scheduled Talk | Agents for Software Development Tutorials Graham Neubig Carnegie Mellon University | ||
| Not scheduled Talk | Harmonized Coding with AI: LLMs for Qualitative Analysis in Software Engineering Research Tutorials Christoph Treude Singapore Management University, Youmei Fan Nara Institute of Science and Technology, Tao Xiao Kyushu University, Hideaki Hata Shinshu University |
Accepted Papers
| Title | |
|---|---|
| Agents for Software Development Tutorials | |
| Harmonized Coding with AI: LLMs for Qualitative Analysis in Software Engineering Research Tutorials |
Call for Tutorials
The MSR Tutorials track aims to invite experienced software repository miners to give informative tutorials to our broad community, whether newcomers or mining experts. These tutorials will cover various topics related to mining software repositories.
We encourage researchers to submit an abstract outlining a talk, with a maximum length of one page (plus up to one additional page of references). We are soliciting abstracts in two categories:
- Research Problem: the talk outlines a single problem in an academic/industrial context that could be addressed using data from software repositories.
- Lessons Learnt & Opportunities: the talk outlines a series of actionable implications and provides promising directions for future researchers.
If you have any questions, please contact the co-chairs: Dong Wang (d.wang@ait.kyushu-u.ac.jp) and Ayushi Rastogi (a.rastogi@rug.nl)
Submission
A one-page PDF file is required for submission, including the title, speaker names, affiliations, and the outline of the tutorial talk. Additionally, up to one page of references can be included. We do not require any specific template for the submission stage.
If your submission is accepted, you can have your abstract published in the conference proceedings. In this case, your abstract must conform to the official “ACM Primary Article Template”, which can be obtained from the ACM Proceedings Template page. If you are using LaTeX, please use the “sigconf” option, as well as the “review” option, to produce line numbers for easy reference by the reviewers. To do so, please include the following LaTeX code at the start of your document:
\documentclass\[sigconf,review,anonymous]{acmart}
\acmConference\[ICSE 2024]{46th International Conference on Software Engineering}{April 2024}{Lisbon, Portugal}
If you would like to submit your tutorial for consideration for the 2024 MSR Tutorials, please submit it at https://msr24tutorial.hotcrp.com/
Evaluation Criteria
The submissions will be evaluated by the two tutorial track co-chairs. We will assess the relevance and clarity of the problem description, the lessons learned, and the research opportunities, along with the context, in submissions from researchers.
Accepted Papers
- The official publication date is when the proceedings become available in the ACM or IEEE Digital Libraries. Note that this date may be up to two weeks before the first day of ICSE 2024. It’s important to remember that the official publication date may affect the deadline for any patent filings related to published work.
- Purchases of additional pages in the proceedings are not allowed.
Important Dates (all dates are in AoE)
- Tue 19th of December 2023 (Proposal submission deadline),
- Thu 21st of December 2023 (Author Notification),
- Sun 28th of January 2024 (Camera-ready version)