Regression Testing: Theory and Practice

Seminar
Monday, March 02, 2015
11:00 AM to 12:00 PM
UTA 7.532
Free and open to the public

Developers often build regression test suites that are automatically run for each code revision to check that code changes did not break any functionality. While regression testing is important, it is also expensive due to both the number of revisions and the number of tests. For example, Google recently reported that they observed a quadratic increase in daily test-suite run time (a linear increase in the number of revisions per day and a linear increase in the number of tests per revision).

In this talk, I present a technique, called Ekstazi, to substantially reduce test-suite run time. Ekstazi introduces a novel approach to regression test selection, which runs only a subset of tests whose dependencies may be affected by the latest changes; Ekstazi keeps file dependencies for each test. Ekstazi also speeds up test-suite runs for software that uses modern distributed version-control systems; by modeling different branch and merge commands directly, Ekstazi computes test sets that can be significantly smaller than the entire test suite. I developed Ekstazi for JVM languages and evaluated it on several hundred revisions of 32 open-source projects (totaling 5M lines of code).  Ekstazi can reduce test-suite run time an order of magnitude, including runs for merge revisions. Finally, only a few months after the initial release, Ekstazi was adopted and used daily by many developers from several open-source projects, including Apache Camel, Commons Math, and CXF.

x x

Speaker

Milos Gligoric

Milos Gligoric

University of Illinois at Urbana-Champaign

Milos Gligoric is a PhD candidate in Computer Science at the University of Illinois at Urbana-Champaign (UIUC). His research interests are in software engineering and formal methods, especially in designing techniques and tools that improve software quality and developers’ productivity.  His PhD work has explored test input generation, test quality assessment, testing concurrent code, and regression testing.  He won an ACM SIGSOFT Distinguished Paper Award (ICSE 2010), and three of his papers were invited for a journal submission. He was awarded the Saburo Muroga Fellowship (2009), the C.L. and Jane W-S. Liu Award (2012), and the C. W. Gear Outstanding Graduate Award (2014) from the UIUC Department of Computer Science and the Mavis Future Faculty Fellowship (2014) from the UIUC College of Engineering. He did internships at NASA Ames, Intel, Max Planck Institute for Software Systems, and Microsoft Research. Milos holds a BS (2007) and MS (2009) from the University of Belgrade, Serbia.