Author
Kıraç, Mustafa Furkan, Aktemur, Tankut Barış, Sözer, Hasan
Publication Date
2018-02
Publication Place
-
The ACM Digital Library
Subject
Black-box testing, Test oracle, Computer vision, Image processing, Test automation
Type
Periodical
Language
English
Digital
Yes
Manuscript
No
Library
Özyeğin University
Library Asset ID
0164-1212
Record ID
08925302-f268-4e4d-883d-935bf84d823f
Library Location
Computer Science
Date
2018-02
Sample Text
A test oracle automation approach proposed for systems that produce visual output.Root causes of accuracy issues analyzed for test oracles based on image comparison.Image processing techniques employed to improve the accuracy of test oracles.A fast image processing pipeline developed as an automated test oracle.An industrial case study performed for automated regression testing of Digital TVs. Test oracles differentiate between the correct and incorrect system behavior. Hence, test oracle automation is essential to achieve overall test automation. Otherwise, testers have to manually check the system behavior for all test cases. A common test oracle automation approach for testing systems with visual output is based on exact matching between a snapshot of the observed output and a previously taken reference image. However, images can be subject to scaling and translation variations. These variations lead to a high number of false positives, where an error is reported due to a mismatch between the compared images although an error does not exist. To address this problem, we introduce an automated test oracle, named VISOR, that employs a fast image processing pipeline. This pipeline includes a series of image filters that align the compared images and remove noise to eliminate differences caused by scaling and translation. We evaluated our approach in the context of an industrial case study for regression testing of Digital TVs. Results show that VISOR can avoid 90% of false positive cases after training the system for 4h. Following this one-time training, VISOR can compare thousands of image pairs within seconds on a laptop computer.
DOI
10.1016/j.jss.2017.06.023
Cilt
136
