Crowdsourcing hypothesis tests: Making transparent how design choices shape research results.

Justin F. Landy; ML Jia; Isabel L. Ding; Domenico Viganola; Warren Tierney; Anna Dreber; Magnus Johannesson; Thomas Pfeiffer; Charles R. Ebersole; Quentin F. Gronau; A Ly; den Bergh D van; Maarten Marsman; Koen Derks; E-J Wagenmakers; Andrew Proctor; Daniel M. Bartels; Christopher W. Bauman; William J. Brady; F Cheung; Andrei Cimpian; Simone Dohle; M. Brent Donnellan; Adam Hahn; Michael P. Hall; William Jiménez-Leal; DJ Johnson; Richard E. Lucas; BenoÎt Monin; Andres Montealegre; Elizabeth Mullen; Jun Pang; Jennifer Ray; Diego A. Reinero; J Reynolds; Walter Sowden; Daniel Storage; Runkun Su; Christina M. Tworek; Bavel JJ Van; Daniel Walco; Julian Wills; Xiaobing Xu; Kai Chi Yam; Xiaoyu Yang; William A. Cunningham; Martin Schweinsberg; Molly Urwitz; The Crowdsourcing Hypothesis Tests Collaboration; Eric L. Uhlmann; Jan K. Woike

doi:10.1037/bul0000220

Crowdsourcing hypothesis tests: Making transparent how design choices shape research results.

Justin F. Landy^*, ML Jia, Isabel L. Ding, Domenico Viganola, Warren Tierney, Anna Dreber, Magnus Johannesson, Thomas Pfeiffer, Charles R. Ebersole, Quentin F. Gronau, A Ly, den Bergh D van, Maarten Marsman, Koen Derks, E-J Wagenmakers, Andrew Proctor, Daniel M. Bartels, Christopher W. Bauman, William J. Brady, F CheungAndrei Cimpian, Simone Dohle, M. Brent Donnellan, Adam Hahn, Michael P. Hall, William Jiménez-Leal, DJ Johnson, Richard E. Lucas, BenoÎt Monin, Andres Montealegre, Elizabeth Mullen, Jun Pang, Jennifer Ray, Diego A. Reinero, J Reynolds, Walter Sowden, Daniel Storage, Runkun Su, Christina M. Tworek, Bavel JJ Van, Daniel Walco, Julian Wills, Xiaobing Xu, Kai Chi Yam, Xiaoyu Yang, William A. Cunningham, Martin Schweinsberg, Molly Urwitz, The Crowdsourcing Hypothesis Tests Collaboration, Eric L. Uhlmann, Jan K. Woike

^*Corresponding author for this work

School of Psychology

Research output: Contribution to journal › Article › peer-review

27 Downloads (Pure)

Original language	English
Pages (from-to)	451-479
Number of pages	0
Journal	Psychological Bulletin
Volume	146
Issue number	5
Early online date	May 2020
DOIs	https://doi.org/10.1037/bul0000220
Publication status	Published - May 2020

Access to Document

10.1037/bul0000220

BUL-2018-1302 R3

Cite this

Landy, J. F., Jia, ML., Ding, I. L., Viganola, D., Tierney, W., Dreber, A., Johannesson, M., Pfeiffer, T., Ebersole, C. R., Gronau, Q. F., Ly, A., van, D. B. D., Marsman, M., Derks, K., Wagenmakers, E.-J., Proctor, A., Bartels, D. M., Bauman, C. W., Brady, W. J., ... Woike, J. K. (2020). Crowdsourcing hypothesis tests: Making transparent how design choices shape research results. Psychological Bulletin, 146(5), 451-479. https://doi.org/10.1037/bul0000220

@article{e3c0676e6372414b96554f4a62804cce,

title = "Crowdsourcing hypothesis tests: Making transparent how design choices shape research results.",

author = "Landy, {Justin F.} and ML Jia and Ding, {Isabel L.} and Domenico Viganola and Warren Tierney and Anna Dreber and Magnus Johannesson and Thomas Pfeiffer and Ebersole, {Charles R.} and Gronau, {Quentin F.} and A Ly and van, {den Bergh D} and Maarten Marsman and Koen Derks and E-J Wagenmakers and Andrew Proctor and Bartels, {Daniel M.} and Bauman, {Christopher W.} and Brady, {William J.} and F Cheung and Andrei Cimpian and Simone Dohle and Donnellan, {M. Brent} and Adam Hahn and Hall, {Michael P.} and William Jim{\'e}nez-Leal and DJ Johnson and Lucas, {Richard E.} and Beno{\^I}t Monin and Andres Montealegre and Elizabeth Mullen and Jun Pang and Jennifer Ray and Reinero, {Diego A.} and J Reynolds and Walter Sowden and Daniel Storage and Runkun Su and Tworek, {Christina M.} and Van, {Bavel JJ} and Daniel Walco and Julian Wills and Xiaobing Xu and Yam, {Kai Chi} and Xiaoyu Yang and Cunningham, {William A.} and Martin Schweinsberg and Molly Urwitz and Collaboration, {The Crowdsourcing Hypothesis Tests} and Uhlmann, {Eric L.} and Woike, {Jan K.}",

year = "2020",

month = may,

doi = "10.1037/bul0000220",

language = "English",

volume = "146",

pages = "451--479",

journal = "Psychological Bulletin",

issn = "0033-2909",

publisher = "American Psychological Association",

number = "5",

}

Landy, JF, Jia, ML, Ding, IL, Viganola, D, Tierney, W, Dreber, A, Johannesson, M, Pfeiffer, T, Ebersole, CR, Gronau, QF, Ly, A, van, DBD, Marsman, M, Derks, K, Wagenmakers, E-J, Proctor, A, Bartels, DM, Bauman, CW, Brady, WJ, Cheung, F, Cimpian, A, Dohle, S, Donnellan, MB, Hahn, A, Hall, MP, Jiménez-Leal, W, Johnson, DJ, Lucas, RE, Monin, B, Montealegre, A, Mullen, E, Pang, J, Ray, J, Reinero, DA, Reynolds, J, Sowden, W, Storage, D, Su, R, Tworek, CM, Van, BJJ, Walco, D, Wills, J, Xu, X, Yam, KC, Yang, X, Cunningham, WA, Schweinsberg, M, Urwitz, M, Collaboration, TCHT, Uhlmann, EL & Woike, JK 2020, 'Crowdsourcing hypothesis tests: Making transparent how design choices shape research results.', Psychological Bulletin, vol. 146, no. 5, pp. 451-479. https://doi.org/10.1037/bul0000220

TY - JOUR

T1 - Crowdsourcing hypothesis tests: Making transparent how design choices shape research results.

AU - Landy, Justin F.

AU - Jia, ML

AU - Ding, Isabel L.

AU - Viganola, Domenico

AU - Tierney, Warren

AU - Dreber, Anna

AU - Johannesson, Magnus

AU - Pfeiffer, Thomas

AU - Ebersole, Charles R.

AU - Gronau, Quentin F.

AU - Ly, A

AU - van, den Bergh D

AU - Marsman, Maarten

AU - Derks, Koen

AU - Wagenmakers, E-J

AU - Proctor, Andrew

AU - Bartels, Daniel M.

AU - Bauman, Christopher W.

AU - Brady, William J.

AU - Cheung, F

AU - Cimpian, Andrei

AU - Dohle, Simone

AU - Donnellan, M. Brent

AU - Hahn, Adam

AU - Hall, Michael P.

AU - Jiménez-Leal, William

AU - Johnson, DJ

AU - Lucas, Richard E.

AU - Monin, BenoÎt

AU - Montealegre, Andres

AU - Mullen, Elizabeth

AU - Pang, Jun

AU - Ray, Jennifer

AU - Reinero, Diego A.

AU - Reynolds, J

AU - Sowden, Walter

AU - Storage, Daniel

AU - Su, Runkun

AU - Tworek, Christina M.

AU - Van, Bavel JJ

AU - Walco, Daniel

AU - Wills, Julian

AU - Xu, Xiaobing

AU - Yam, Kai Chi

AU - Yang, Xiaoyu

AU - Cunningham, William A.

AU - Schweinsberg, Martin

AU - Urwitz, Molly

AU - Collaboration, The Crowdsourcing Hypothesis Tests

AU - Uhlmann, Eric L.

AU - Woike, Jan K.

PY - 2020/5

Y1 - 2020/5

U2 - 10.1037/bul0000220

DO - 10.1037/bul0000220

M3 - Article

SN - 0033-2909

VL - 146

SP - 451

EP - 479

JO - Psychological Bulletin

JF - Psychological Bulletin

IS - 5

ER -