Metamorphic testing: testing the untestable
What if we could know that a program is buggy, even if we could not tell whether or not its observed output is correct? This is one of the key strengths of metamorphic testing, a technique where failures are not revealed by checking an individual concrete output, but by checking the relations among...
| Main Authors: | , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
2018
|
| Subjects: | |
| Online Access: | https://eprints.nottingham.ac.uk/59999/ |
| _version_ | 1848799709117808640 |
|---|---|
| author | Segura, Sergio Towey, Dave Zhou, Zhi Quan Chen, Tsong Yueh |
| author_facet | Segura, Sergio Towey, Dave Zhou, Zhi Quan Chen, Tsong Yueh |
| author_sort | Segura, Sergio |
| building | Nottingham Research Data Repository |
| collection | Online Access |
| description | What if we could know that a program is buggy, even if we could not tell whether or not its observed output is correct? This is one of the key strengths of metamorphic testing, a technique where failures are not revealed by checking an individual concrete output, but by checking the relations among the inputs and outputs of multiple executions of the program under test. Two decades after its introduction, metamorphic testing has become a fully-fledged testing technique with successful applications in multiple domains, including online search engines, autonomous machinery, compilers, Web APIs, and deep learning programs, among others. This article serves as a hands-on entry point for newcomers to metamorphic testing, describing examples, possible applications, and current limitations, providing readers with the basics for the application of the technique in their own projects. IEEE |
| first_indexed | 2025-11-14T20:39:58Z |
| format | Article |
| id | nottingham-59999 |
| institution | University of Nottingham Malaysia Campus |
| institution_category | Local University |
| language | English |
| last_indexed | 2025-11-14T20:39:58Z |
| publishDate | 2018 |
| recordtype | eprints |
| repository_type | Digital Repository |
| spelling | nottingham-599992020-02-28T03:55:31Z https://eprints.nottingham.ac.uk/59999/ Metamorphic testing: testing the untestable Segura, Sergio Towey, Dave Zhou, Zhi Quan Chen, Tsong Yueh What if we could know that a program is buggy, even if we could not tell whether or not its observed output is correct? This is one of the key strengths of metamorphic testing, a technique where failures are not revealed by checking an individual concrete output, but by checking the relations among the inputs and outputs of multiple executions of the program under test. Two decades after its introduction, metamorphic testing has become a fully-fledged testing technique with successful applications in multiple domains, including online search engines, autonomous machinery, compilers, Web APIs, and deep learning programs, among others. This article serves as a hands-on entry point for newcomers to metamorphic testing, describing examples, possible applications, and current limitations, providing readers with the basics for the application of the technique in their own projects. IEEE 2018-12-08 Article PeerReviewed application/pdf en cc_by https://eprints.nottingham.ac.uk/59999/1/combinepdf%20%282%29.pdf Segura, Sergio, Towey, Dave, Zhou, Zhi Quan and Chen, Tsong Yueh (2018) Metamorphic testing: testing the untestable. IEEE Software . p. 1. ISSN 0740-7459 Software testing; metamorphic testing; oracle problem; test case generation http://dx.doi.org/10.1109/MS.2018.2875968 doi:10.1109/MS.2018.2875968 doi:10.1109/MS.2018.2875968 |
| spellingShingle | Software testing; metamorphic testing; oracle problem; test case generation Segura, Sergio Towey, Dave Zhou, Zhi Quan Chen, Tsong Yueh Metamorphic testing: testing the untestable |
| title | Metamorphic testing: testing the untestable |
| title_full | Metamorphic testing: testing the untestable |
| title_fullStr | Metamorphic testing: testing the untestable |
| title_full_unstemmed | Metamorphic testing: testing the untestable |
| title_short | Metamorphic testing: testing the untestable |
| title_sort | metamorphic testing: testing the untestable |
| topic | Software testing; metamorphic testing; oracle problem; test case generation |
| url | https://eprints.nottingham.ac.uk/59999/ https://eprints.nottingham.ac.uk/59999/ https://eprints.nottingham.ac.uk/59999/ |