Abstract test case prioritization using repeated small-strength level-combination coverage
Abstract—Abstract Test Cases (ATCs) have been widely used in practice, including in combinatorial testing and in software product line testing. When constructing a set of ATCs, due to limited testing resources in practice (for example in regression testing), Test Case Prioritization (TCP) has been p...
| Main Authors: | , , , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
2020
|
| Subjects: | |
| Online Access: | https://eprints.nottingham.ac.uk/60295/ |
| _version_ | 1848799746039218176 |
|---|---|
| author | Huang, Rubing Sun, Weifeng Chen, Tsong Yueh Towey, Dave Chen, Jinfu Zong, Weiwen Zhou, Yunan |
| author_facet | Huang, Rubing Sun, Weifeng Chen, Tsong Yueh Towey, Dave Chen, Jinfu Zong, Weiwen Zhou, Yunan |
| author_sort | Huang, Rubing |
| building | Nottingham Research Data Repository |
| collection | Online Access |
| description | Abstract—Abstract Test Cases (ATCs) have been widely used in practice, including in combinatorial testing and in software product line testing. When constructing a set of ATCs, due to limited testing resources in practice (for example in regression testing), Test Case Prioritization (TCP) has been proposed to improve the testing quality, aiming at ordering test cases to increase the speed with which faults are detected. One intuitive and extensively studied TCP technique for ATCs is λ-wise Level-combination Coverage based Prioritization (λLCP), a static, black-box prioritization technique that only uses the ATC information to guide the prioritization process. A challenge facing λLCP, however, is the necessity for the selection of the fixed prioritization strength λ before testing — testers need to choose an appropriate λ value before testing begins. Choosing higher λ values may improve the testing effectiveness of λLCP (for example, by finding faults faster), but may reduce the testing efficiency (by incurring additional prioritization costs). Conversely, choosing lower λ values may improve the efficiency, but may also reduce the effectiveness. In this paper, we propose a new family of λLCP techniques, Repeated Small-strength Level-combination Coverage-based Prioritization (RSLCP), that repeatedly achieves the full combination coverage at lower strengths. RSLCP maintains λLCP’s advantages of being static and black box, but avoids the challenge of prioritization strength selection. We performed an empirical study involving five different versions of each of five C programs. Compared with λLCP, and Incremental strength LCP (ILCP), our results show that RSLCP could provide a good trade-off between testing effectiveness and efficiency. Our results also show that RSLCP is more effective and efficient than two popular techniques of Similarity-based Prioritization (SP). In addition, the results of empirical studies also show that RSLCP can remain robust over multiple system releases. |
| first_indexed | 2025-11-14T20:40:33Z |
| format | Article |
| id | nottingham-60295 |
| institution | University of Nottingham Malaysia Campus |
| institution_category | Local University |
| language | English |
| last_indexed | 2025-11-14T20:40:33Z |
| publishDate | 2020 |
| recordtype | eprints |
| repository_type | Digital Repository |
| spelling | nottingham-602952020-04-13T01:38:39Z https://eprints.nottingham.ac.uk/60295/ Abstract test case prioritization using repeated small-strength level-combination coverage Huang, Rubing Sun, Weifeng Chen, Tsong Yueh Towey, Dave Chen, Jinfu Zong, Weiwen Zhou, Yunan Abstract—Abstract Test Cases (ATCs) have been widely used in practice, including in combinatorial testing and in software product line testing. When constructing a set of ATCs, due to limited testing resources in practice (for example in regression testing), Test Case Prioritization (TCP) has been proposed to improve the testing quality, aiming at ordering test cases to increase the speed with which faults are detected. One intuitive and extensively studied TCP technique for ATCs is λ-wise Level-combination Coverage based Prioritization (λLCP), a static, black-box prioritization technique that only uses the ATC information to guide the prioritization process. A challenge facing λLCP, however, is the necessity for the selection of the fixed prioritization strength λ before testing — testers need to choose an appropriate λ value before testing begins. Choosing higher λ values may improve the testing effectiveness of λLCP (for example, by finding faults faster), but may reduce the testing efficiency (by incurring additional prioritization costs). Conversely, choosing lower λ values may improve the efficiency, but may also reduce the effectiveness. In this paper, we propose a new family of λLCP techniques, Repeated Small-strength Level-combination Coverage-based Prioritization (RSLCP), that repeatedly achieves the full combination coverage at lower strengths. RSLCP maintains λLCP’s advantages of being static and black box, but avoids the challenge of prioritization strength selection. We performed an empirical study involving five different versions of each of five C programs. Compared with λLCP, and Incremental strength LCP (ILCP), our results show that RSLCP could provide a good trade-off between testing effectiveness and efficiency. Our results also show that RSLCP is more effective and efficient than two popular techniques of Similarity-based Prioritization (SP). In addition, the results of empirical studies also show that RSLCP can remain robust over multiple system releases. 2020-03-01 Article PeerReviewed application/pdf en cc_by https://eprints.nottingham.ac.uk/60295/1/Dave%20merged.pdf Huang, Rubing, Sun, Weifeng, Chen, Tsong Yueh, Towey, Dave, Chen, Jinfu, Zong, Weiwen and Zhou, Yunan (2020) Abstract test case prioritization using repeated small-strength level-combination coverage. IEEE Transactions on Reliability, 69 (1). pp. 349-372. ISSN 0018-9529 Software testing regression testing abstract test case test case prioritization level-combination coverage http://dx.doi.org/10.1109/TR.2019.2908068 doi:10.1109/TR.2019.2908068 doi:10.1109/TR.2019.2908068 |
| spellingShingle | Software testing regression testing abstract test case test case prioritization level-combination coverage Huang, Rubing Sun, Weifeng Chen, Tsong Yueh Towey, Dave Chen, Jinfu Zong, Weiwen Zhou, Yunan Abstract test case prioritization using repeated small-strength level-combination coverage |
| title | Abstract test case prioritization using repeated small-strength level-combination coverage |
| title_full | Abstract test case prioritization using repeated small-strength level-combination coverage |
| title_fullStr | Abstract test case prioritization using repeated small-strength level-combination coverage |
| title_full_unstemmed | Abstract test case prioritization using repeated small-strength level-combination coverage |
| title_short | Abstract test case prioritization using repeated small-strength level-combination coverage |
| title_sort | abstract test case prioritization using repeated small-strength level-combination coverage |
| topic | Software testing regression testing abstract test case test case prioritization level-combination coverage |
| url | https://eprints.nottingham.ac.uk/60295/ https://eprints.nottingham.ac.uk/60295/ https://eprints.nottingham.ac.uk/60295/ |