Time-aware test suite prioritization

Walcott, Kristen R. and Soffa, Mary Lou and Kapfhammer, Gregory M. and Roos, Robert S.
Proceedings of the International Symposium on Software Testing and Analysis, 2006


Regression testing is frequently performed in a time constrained environment. This paper explains how 0/1 knapsack solvers (e.g., greedy, dynamic programming, and the core algorithm) can identify a test suite reordering that rapidly covers the test requirements and always terminates within a specified testing time limit. We conducted experiments that reveal fundamental trade-offs in the (i) time and space costs that are associated with creating a reordered test suite and (ii) quality of the resulting prioritization. We find knapsack-based prioritizers that ignore the overlap in test case coverage incur a low time overhead and a moderate to high space overhead while creating prioritizations exhibiting a minor to modest decrease in effectiveness. We also find that the most sophisticated 0/1 knapsack solvers do not always identify the most effective prioritization, suggesting that overlap-aware prioritizers with a higher time overhead are useful in certain testing contexts.



Walcott, K. R., Soffa, M. L., Kapfhammer, G. M., & Roos, R. S. (2006). Time-aware test suite prioritization. Proceedings of the International Symposium on Software Testing and Analysis.


Want to cite this paper? Look in the BiBTeX file of gkapfham/research-bibliography for the key "Walcott2006".

Return to the List of Papers

Please support my work!

View the source.