Author
Dirim, S., Sözer, Hasan
Publication Date
2020-10
Publication Place
-
IEEE
Subject
Certification testing, Test case prioritization, Fault severity, Test cost, Industrial case study
Type
Document
Language
English
Digital
Yes
Manuscript
No
Library
Özyeğin University
Library Asset ID
978-1-7281-1075-2
Record ID
54d2f712-35b0-4e1a-80a8-d444c88829c9
Library Location
Computer Science
Date
2020-10
Sample Text
We present an industrial case study on the application of test case prioritization techniques in the context of certification testing in consumer electronics domain. Test execution times and fault severities are subject to high variations in this domain. As a result, most of the existing techniques and metrics turn out to be inappropriate for this application context. We discuss such deficiencies and the room for improvement based on our case study with the certification test suites of 3 Smart TV applications as real experimental objects. We also propose a new metric, LAPFD, which is based on the calculation of the average of the percentage of faults detected. This calculation is weighted according to the cost of test cases and calculated separately per severity class. Then, a lexicographic ordering is performed based on these classes. We compared the baseline (random) ordering of test cases with respect to an alternative ordering based on cost, measured as the test execution time. These alternative orderings are evaluated by using the LAPFD metric. We observed that cost-based ordering of test cases consistently outperformed random ordering. Another observation is that there is a large room for improvement regarding the effectiveness of test case prioritization in this application domain.
DOI
10.1109/ICSTW50294.2020.00069
