Comparing experts and novices in Martian surface feature change detection and identification
Change detection in satellite images is a key concern of the Earth Observation field for environmental and climate change monitoring. Satellite images also provide important clues to both the past and present surface conditions of other planets, which cannot be validated on the ground. With the volu...
| Main Authors: | , , , , , , |
|---|---|
| Format: | Article |
| Published: |
Elsevier
2017
|
| Subjects: | |
| Online Access: | https://eprints.nottingham.ac.uk/43671/ |
| _version_ | 1848796740615929856 |
|---|---|
| author | Wardlaw, Jessica Sprinks, James Houghton, Robert Muller, Jan-Peter Sidiropoulos, Panagiotis Bamford, Steven Marsh, Stuart |
| author_facet | Wardlaw, Jessica Sprinks, James Houghton, Robert Muller, Jan-Peter Sidiropoulos, Panagiotis Bamford, Steven Marsh, Stuart |
| author_sort | Wardlaw, Jessica |
| building | Nottingham Research Data Repository |
| collection | Online Access |
| description | Change detection in satellite images is a key concern of the Earth Observation field for environmental and climate change monitoring. Satellite images also provide important clues to both the past and present surface conditions of other planets, which cannot be validated on the ground. With the volume of satellite imagery continuing to grow, the inadequacy of computerised solutions to manage and process imagery to the required professional standard is of critical concern. Whilst studies find the crowd sourcing approach suitable for the counting of impact craters in single images, images of higher resolution contain a much wider range of features, and the performance of novices in identifying more complex features and detecting change, remains unknown.
This paper presents a first step towards understanding whether novices can identify and annotate changes in different geomorphological features. A website was developed to enable visitors to flick between two images of the same location on Mars taken at different times and classify 1) if a surface feature changed and if so, 2) what feature had changed from a pre-defined list of six. Planetary scientists provided “expert” data against which classifications made by novices could be compared when the project subsequently went public.
Whilst no significant difference was found in images identified with surface changes by expert and novices, results exhibited differences in consensus within and between experts and novices when asked to classify the type of change. Experts demonstrated higher levels of agreement in classification of changes as dust devil tracks, slope streaks and impact craters than other features, whilst the consensus of novices was consistent across feature types; furthermore, the level of consensus amongst regardless of feature type. These trends are secondary to the low levels of consensus found, regardless of feature type or classifier expertise. These findings demand the attention of researchers who want to use crowd-sourcing for similar scientific purposes, particularly for the supervised training of computer algorithms, and inform the scope and design of future projects. |
| first_indexed | 2025-11-14T19:52:47Z |
| format | Article |
| id | nottingham-43671 |
| institution | University of Nottingham Malaysia Campus |
| institution_category | Local University |
| last_indexed | 2025-11-14T19:52:47Z |
| publishDate | 2017 |
| publisher | Elsevier |
| recordtype | eprints |
| repository_type | Digital Repository |
| spelling | nottingham-436712020-05-04T18:49:48Z https://eprints.nottingham.ac.uk/43671/ Comparing experts and novices in Martian surface feature change detection and identification Wardlaw, Jessica Sprinks, James Houghton, Robert Muller, Jan-Peter Sidiropoulos, Panagiotis Bamford, Steven Marsh, Stuart Change detection in satellite images is a key concern of the Earth Observation field for environmental and climate change monitoring. Satellite images also provide important clues to both the past and present surface conditions of other planets, which cannot be validated on the ground. With the volume of satellite imagery continuing to grow, the inadequacy of computerised solutions to manage and process imagery to the required professional standard is of critical concern. Whilst studies find the crowd sourcing approach suitable for the counting of impact craters in single images, images of higher resolution contain a much wider range of features, and the performance of novices in identifying more complex features and detecting change, remains unknown. This paper presents a first step towards understanding whether novices can identify and annotate changes in different geomorphological features. A website was developed to enable visitors to flick between two images of the same location on Mars taken at different times and classify 1) if a surface feature changed and if so, 2) what feature had changed from a pre-defined list of six. Planetary scientists provided “expert” data against which classifications made by novices could be compared when the project subsequently went public. Whilst no significant difference was found in images identified with surface changes by expert and novices, results exhibited differences in consensus within and between experts and novices when asked to classify the type of change. Experts demonstrated higher levels of agreement in classification of changes as dust devil tracks, slope streaks and impact craters than other features, whilst the consensus of novices was consistent across feature types; furthermore, the level of consensus amongst regardless of feature type. These trends are secondary to the low levels of consensus found, regardless of feature type or classifier expertise. These findings demand the attention of researchers who want to use crowd-sourcing for similar scientific purposes, particularly for the supervised training of computer algorithms, and inform the scope and design of future projects. Elsevier 2017-06-12 Article PeerReviewed Wardlaw, Jessica, Sprinks, James, Houghton, Robert, Muller, Jan-Peter, Sidiropoulos, Panagiotis, Bamford, Steven and Marsh, Stuart (2017) Comparing experts and novices in Martian surface feature change detection and identification. International Journal of Applied Earth Observation and Geoinformation . ISSN 0303-2434 Crowd sourcing; Citizen science; Volunteered geographic information; Planetary science; Change detection; Image analysis http://www.sciencedirect.com/science/article/pii/S030324341730123X doi:10.1016/j.jag.2017.05.014 doi:10.1016/j.jag.2017.05.014 |
| spellingShingle | Crowd sourcing; Citizen science; Volunteered geographic information; Planetary science; Change detection; Image analysis Wardlaw, Jessica Sprinks, James Houghton, Robert Muller, Jan-Peter Sidiropoulos, Panagiotis Bamford, Steven Marsh, Stuart Comparing experts and novices in Martian surface feature change detection and identification |
| title | Comparing experts and novices in Martian surface feature change detection and identification |
| title_full | Comparing experts and novices in Martian surface feature change detection and identification |
| title_fullStr | Comparing experts and novices in Martian surface feature change detection and identification |
| title_full_unstemmed | Comparing experts and novices in Martian surface feature change detection and identification |
| title_short | Comparing experts and novices in Martian surface feature change detection and identification |
| title_sort | comparing experts and novices in martian surface feature change detection and identification |
| topic | Crowd sourcing; Citizen science; Volunteered geographic information; Planetary science; Change detection; Image analysis |
| url | https://eprints.nottingham.ac.uk/43671/ https://eprints.nottingham.ac.uk/43671/ https://eprints.nottingham.ac.uk/43671/ |