Examining the generalizability of research findings from archival data

Andrew Delios*, Elena Giulia Clemente, Tao Wu, Hongbin Tan, Yong Wang, Michael Gordon, Domenico Viganola, Z Chen, Anna Dreber, Magnus Johannesson, Thomas Pfeiffer, Eric Luis Uhlmann*, Al-Aziz AM Abd, Ajay T. Abraham, Jais Trojan, Matus Adamkovic, Elena Agadullina, Jungsoo Ahn, Cinla Akinci, Handan AkkasDavid Albrecht, Shilaan Alzahawi, Marcio Amaral-Baptista, Rahul Anand, Kevin Francis U. Ang, Frederik Anseel, John Jamir Benzon R. Aruta, Mujeeba Ashraf, Bradley J. Baker, Xueqi Bao, Ernest Baskin, Hanoku Bathula, Christopher W. Bauman, Jozef Bavolar, Secil Bayraktar, Stephanie E. Beckman, Aaron S. Benjamin, Stephanie E.V. Brown, Jeffrey Buckley, Ricardo E. Buitrago, Jefferson L. Bution, Nick Byrd, Clara Carrera, Eugene M. Caruso, M Chen, L Chen, Eyyub Ensari Cicerali, Eric D. Cohen, Marcus Crede, J Cummins, L Dahlander, DP Daniels, LL Daskalo, IGJ Dawson, MV Day, E Dietl, A Domurat, J Dsilva, CD Plessis, DI Dubrov, S Edris, CT Elbaek, MM Elsherif, TR Evans, MR Fellenz, S Fiedler, M Firat, R Freitag, RA Furrer, R Gautam, DK Gautam, B Gearin, S Gerschewski, O Ghasemi, Z Ghasemi, A Ghosh, C Giani, MH Goldberg, M Goswami, L Graf-Vlachy, JA Griffith, D Grigoryev, J Gu, R H, AL Hadida, AC Hafenbrack, S Hafenbrädl, JJ Hammersley, H Han, JL Harman, A Hartanto, Jan K. Woike

*Corresponding author for this work

Research output: Contribution to journalArticlepeer-review

12 Downloads (Pure)

Abstract

<jats:p>This initiative examined systematically the extent to which a large set of archival research findings generalizes across contexts. We repeated the key analyses for 29 original strategic management effects in the same context (direct reproduction) as well as in 52 novel time periods and geographies; 45% of the reproductions returned results matching the original reports together with 55% of tests in different spans of years and 40% of tests in novel geographies. Some original findings were associated with multiple new tests. Reproducibility was the best predictor of generalizability—for the findings that proved directly reproducible, 84% emerged in other available time periods and 57% emerged in other geographies. Overall, only limited empirical evidence emerged for context sensitivity. In a forecasting survey, independent scientists were able to anticipate which effects would find support in tests in new samples.</jats:p>
Original languageEnglish
Number of pages0
JournalProceedings of the National Academy of Sciences
Volume119
Issue number30
Early online date19 Jul 2022
DOIs
Publication statusPublished - 19 Jul 2022

Fingerprint

Dive into the research topics of 'Examining the generalizability of research findings from archival data'. Together they form a unique fingerprint.

Cite this