Paper Abstract
"The growing use of data to derive insights and information presents many challenges and opportunities. Further, the increased awareness of the potential of crowdworking and machine learning technologies has created a need to understand the benefits and caveats of these approaches. By reviewing current research and then comparing a novice based crowdworking approach against experts and machine learning benchmarks we seek to assess the trade-offs. The task specifically requires users to interpret satellite imagery and determine the location of residences or businesses. We are able to demonstrate that a novice approach can provide value where the data collected meets an accuracy tolerance that closely matches the expert users. Further, the potential for equivalent results is shown to be possible based on potential improvements to the system and user familiarity with the task."

The full paper is available through SpringerLink, provided that you have proper access rights.