Affordable Access

deepdyve-link
Publisher Website

Mechanical Turk data collection in addiction research: utility, concerns and best practices.

Authors
  • Mellis, Alexandra M1
  • Bickel, Warren K1
  • 1 Addiction Recovery Research Center, Fralin Biomedical Research Institute at VTC, VA, USA.
Type
Published Article
Journal
Addiction (Abingdon, England)
Publication Date
Oct 01, 2020
Volume
115
Issue
10
Pages
1960–1968
Identifiers
DOI: 10.1111/add.15032
PMID: 32135574
Source
Medline
Keywords
Language
English
License
Unknown

Abstract

Amazon Mechanical Turk (MTurk) provides a crowdsourcing platform for the engagement of potential research participants with data collection instruments. This review (1) provides an introduction to the mechanics and validity of MTurk research; (2) gives examples of MTurk research; and (3) discusses current limitations and best practices in MTurk research. We review four use cases of MTurk for research relevant to addictions: (1) the development of novel measures, (2) testing interventions, (3) the collection of longitudinal use data to determine the feasibility of longer-term studies of substance use and (4) the completion of large batteries of assessments to characterize the relationships between measured constructs. We review concerns with the platform, ways of mitigating these and important information to include when presenting findings. MTurk has proved to be a useful source of data for behavioral science more broadly, with specific applications to addiction science. However, it is still not appropriate for all use cases, such as population-level inference. To live up to the potential of highly transparent, reproducible science from MTurk, researchers should clearly report inclusion/exclusion criteria, data quality checks and reasons for excluding collected data, how and when data were collected and both targeted and actual participant compensation. Although on-line survey research is not a substitute for random sampling or clinical recruitment, the Mechanical Turk community of both participants and researchers has developed multiple tools to promote data quality, fairness and rigor. Overall, Mechanical Turk has provided a useful source of convenience samples despite its limitations and has demonstrated utility in the engagement of relevant groups for addiction science. © 2020 Society for the Study of Addiction.

Report this publication

Statistics

Seen <100 times