Increasing the accuracy of crowdsourced information on land cover via a voting procedure weighted by information inferred from the contributed data
Simple consensus methods are often used in crowdsourcing studies to label cases when data are provided by multiple contributors. A basic majority vote rule is often used. This approach weights the contributions from each contributor equally but the contributors may vary in the accuracy with which th...
| Main Authors: | , , , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
MDPI
2018
|
| Subjects: | |
| Online Access: | https://eprints.nottingham.ac.uk/50075/ |
| _version_ | 1848798146387247104 |
|---|---|
| author | Foody, Giles See, Linda Fritz, Steffen Moorthy, Inian Perger, Christoph Schill, Christian Boyd, Doreen |
| author_facet | Foody, Giles See, Linda Fritz, Steffen Moorthy, Inian Perger, Christoph Schill, Christian Boyd, Doreen |
| author_sort | Foody, Giles |
| building | Nottingham Research Data Repository |
| collection | Online Access |
| description | Simple consensus methods are often used in crowdsourcing studies to label cases when data are provided by multiple contributors. A basic majority vote rule is often used. This approach weights the contributions from each contributor equally but the contributors may vary in the accuracy with which they can label cases. Here, the potential to increase the accuracy of crowdsourced data on land cover identified from satellite remote sensor images through the use of weighted voting strategies is explored. Critically, the information used to weight contributions based on the accuracy with which a contributor labels cases of a class and the relative abundance of class are inferred entirely from the contributed data only via a latent class analysis. The results show that consensus approaches do yield a classification that is more accurate than that achieved by any individual contributor. Here, the most accurate individual could classify the data with an accuracy of 73.91% while a basic consensus label derived from the data provided by all seven volunteers contributing data was 76.58%. More importantly, the results show that weighting contributions can lead to a statistically significant increase in the overall accuracy to 80.60% by ignoring the contributions from the volunteer adjudged to be the least accurate in labelling. |
| first_indexed | 2025-11-14T20:15:08Z |
| format | Article |
| id | nottingham-50075 |
| institution | University of Nottingham Malaysia Campus |
| institution_category | Local University |
| language | English |
| last_indexed | 2025-11-14T20:15:08Z |
| publishDate | 2018 |
| publisher | MDPI |
| recordtype | eprints |
| repository_type | Digital Repository |
| spelling | nottingham-500752018-03-22T07:40:59Z https://eprints.nottingham.ac.uk/50075/ Increasing the accuracy of crowdsourced information on land cover via a voting procedure weighted by information inferred from the contributed data Foody, Giles See, Linda Fritz, Steffen Moorthy, Inian Perger, Christoph Schill, Christian Boyd, Doreen Simple consensus methods are often used in crowdsourcing studies to label cases when data are provided by multiple contributors. A basic majority vote rule is often used. This approach weights the contributions from each contributor equally but the contributors may vary in the accuracy with which they can label cases. Here, the potential to increase the accuracy of crowdsourced data on land cover identified from satellite remote sensor images through the use of weighted voting strategies is explored. Critically, the information used to weight contributions based on the accuracy with which a contributor labels cases of a class and the relative abundance of class are inferred entirely from the contributed data only via a latent class analysis. The results show that consensus approaches do yield a classification that is more accurate than that achieved by any individual contributor. Here, the most accurate individual could classify the data with an accuracy of 73.91% while a basic consensus label derived from the data provided by all seven volunteers contributing data was 76.58%. More importantly, the results show that weighting contributions can lead to a statistically significant increase in the overall accuracy to 80.60% by ignoring the contributions from the volunteer adjudged to be the least accurate in labelling. MDPI 2018-02-25 Article PeerReviewed application/pdf en cc_by https://eprints.nottingham.ac.uk/50075/1/ijgi-07-00080.pdf Foody, Giles, See, Linda, Fritz, Steffen, Moorthy, Inian, Perger, Christoph, Schill, Christian and Boyd, Doreen (2018) Increasing the accuracy of crowdsourced information on land cover via a voting procedure weighted by information inferred from the contributed data. ISPRS International Journal of Geo-Information, 7 (3). p. 80. ISSN 2220-9964 crowdsourcing; volunteered geographic information (VGI); ensemble; classification accuracy; latent class analysis https://doi.org/10.3390/ijgi7030080 doi:10.3390/ijgi7030080 doi:10.3390/ijgi7030080 |
| spellingShingle | crowdsourcing; volunteered geographic information (VGI); ensemble; classification accuracy; latent class analysis Foody, Giles See, Linda Fritz, Steffen Moorthy, Inian Perger, Christoph Schill, Christian Boyd, Doreen Increasing the accuracy of crowdsourced information on land cover via a voting procedure weighted by information inferred from the contributed data |
| title | Increasing the accuracy of crowdsourced information on land cover via a voting procedure weighted by information inferred from the contributed data |
| title_full | Increasing the accuracy of crowdsourced information on land cover via a voting procedure weighted by information inferred from the contributed data |
| title_fullStr | Increasing the accuracy of crowdsourced information on land cover via a voting procedure weighted by information inferred from the contributed data |
| title_full_unstemmed | Increasing the accuracy of crowdsourced information on land cover via a voting procedure weighted by information inferred from the contributed data |
| title_short | Increasing the accuracy of crowdsourced information on land cover via a voting procedure weighted by information inferred from the contributed data |
| title_sort | increasing the accuracy of crowdsourced information on land cover via a voting procedure weighted by information inferred from the contributed data |
| topic | crowdsourcing; volunteered geographic information (VGI); ensemble; classification accuracy; latent class analysis |
| url | https://eprints.nottingham.ac.uk/50075/ https://eprints.nottingham.ac.uk/50075/ https://eprints.nottingham.ac.uk/50075/ |