Prioritization of test cases with varying test costs and fault severities for certification testing

Name: Prioritization of test cases with varying test costs and fault severities for certification testing
Author: Dirim, S., Sözer, Hasan

İsim	Prioritization of test cases with varying test costs and fault severities for certification testing
Yazar	Dirim, S., Sözer, Hasan
Basım Tarihi:	2020-10
Basım Yeri	- IEEE
Konu	Certification testing, Test case prioritization, Fault severity, Test cost, Industrial case study
Tür	Belge
Dil	İngilizce
Dijital	Evet
Yazma	Hayır
Kütüphane:	Özyeğin Üniversitesi
Demirbaş Numarası	978-1-7281-1075-2
Kayıt Numarası	54d2f712-35b0-4e1a-80a8-d444c88829c9
Lokasyon	Computer Science
Tarih	2020-10
Örnek Metin	We present an industrial case study on the application of test case prioritization techniques in the context of certification testing in consumer electronics domain. Test execution times and fault severities are subject to high variations in this domain. As a result, most of the existing techniques and metrics turn out to be inappropriate for this application context. We discuss such deficiencies and the room for improvement based on our case study with the certification test suites of 3 Smart TV applications as real experimental objects. We also propose a new metric, LAPFD, which is based on the calculation of the average of the percentage of faults detected. This calculation is weighted according to the cost of test cases and calculated separately per severity class. Then, a lexicographic ordering is performed based on these classes. We compared the baseline (random) ordering of test cases with respect to an alternative ordering based on cost, measured as the test execution time. These alternative orderings are evaluated by using the LAPFD metric. We observed that cost-based ordering of test cases consistently outperformed random ordering. Another observation is that there is a large room for improvement regarding the effectiveness of test case prioritization in this application domain.
DOI	10.1109/ICSTW50294.2020.00069

Kaynağa git Özyeğin Üniversitesi

Aramaya Dön

Özyeğin Üniversitesi

Kaynağa git

Prioritization of test cases with varying test costs and fault severities for certification testing

Yazar Dirim, S., Sözer, Hasan

Basım Tarihi 2020-10

Basım Yeri - IEEE

Konu Certification testing, Test case prioritization, Fault severity, Test cost, Industrial case study

Tür Belge

Dil İngilizce

Dijital Evet

Yazma Hayır

Kütüphane Özyeğin Üniversitesi

Demirbaş Numarası 978-1-7281-1075-2

Kayıt Numarası 54d2f712-35b0-4e1a-80a8-d444c88829c9

Lokasyon Computer Science

Tarih 2020-10

Örnek Metin We present an industrial case study on the application of test case prioritization techniques in the context of certification testing in consumer electronics domain. Test execution times and fault severities are subject to high variations in this domain. As a result, most of the existing techniques and metrics turn out to be inappropriate for this application context. We discuss such deficiencies and the room for improvement based on our case study with the certification test suites of 3 Smart TV applications as real experimental objects. We also propose a new metric, LAPFD, which is based on the calculation of the average of the percentage of faults detected. This calculation is weighted according to the cost of test cases and calculated separately per severity class. Then, a lexicographic ordering is performed based on these classes. We compared the baseline (random) ordering of test cases with respect to an alternative ordering based on cost, measured as the test execution time. These alternative orderings are evaluated by using the LAPFD metric. We observed that cost-based ordering of test cases consistently outperformed random ordering. Another observation is that there is a large room for improvement regarding the effectiveness of test case prioritization in this application domain.

DOI 10.1109/ICSTW50294.2020.00069