Seminar: Mutating Matters: Analyzing the Influence and the Hidden Cost of Mutation Testing in Large Programming Courses
Rifat Sabbir Mansur
PhD candidate in the Digital Education Research Lab
Virginia Tech
Friday, October 18, 2024
2:30 - 3:45PM
3100 Torgersen Hall
Abstract
Mutation testing (MT) is a powerful technique for evaluating the quality of software test suites. MT introduces faults or “mutations” into the code and checks whether the tests then fail as appropriate. Although it has been applied extensively in the software industry, mutation testing's use in programming courses faces both computational and pedagogical barriers. In this talk, I will present the impact of mutation testing on student performance and its computational demand on an auto-grading system, Web-CAT, in a post-CS2 Data Structures and Algorithms course with 4-week life-cycle programming projects. We will investigate how students using mutation testing, as a group, demonstrate higher quality test suites and write better solution code compared to students using traditional code coverage methods. Students using mutation testing are also more likely to exhibit incremental testing practices. On the computational side, we will observe how the time required to run mutation testing on student submissions is manageable, even for large classes. I will talk on the effect of introducing mutation testing on other courses running on the same auto-grading system. Furthermore, I will discuss a potential mitigating strategy: the use of local mutation testing plug-in which helps reduce the total number of submissions to the auto-grader.
Biography
Rifat Sabbir Mansur is a PhD candidate in the Digital Education Research lab at Virginia Tech, advised by Dr. Cliff Shaffer and Dr. Steve Edwards. He earned his Bachelor’s degree from Bangladesh University of Engineering and Technology (BUET), and his Master’s degree from Virginia Tech. His research interests lie in the areas of software engineering (SE), human-computer interaction (HCI), and computer education research (CER). He has published in top conferences such as SIGCSE, CHI, CSCW, and IEEE. Regarding extracurricular activities, he has served in several leadership roles, including graduate representative for the Diversity Committee and CS Grad Council in the Computer Science department at Virginia Tech. Additionally, he held the position of president for ABS@VT and contributed as a hackathon judge for both VTHacks and HackViolet events.