Examining the generalizability of research findings from archival data
File(s)
Author(s)
Type
Journal Article
Abstract
This initiative examined systematically the extent to which a large set of archival research findings generalizes across contexts. We repeated the key analyses for 29 original strategic management effects in the same context (direct reproduction) as well as in 52 novel time periods and geographies; 45% of the reproductions returned results matching the original reports together with 55% of tests in different spans of years and 40% of tests in novel geographies. Some original findings were associated with multiple new tests. Reproducibility was the best predictor of generalizability-for the findings that proved directly reproducible, 84% emerged in other available time periods and 57% emerged in other geographies. Overall, only limited empirical evidence emerged for context sensitivity. In a forecasting survey, independent scientists were able to anticipate which effects would find support in tests in new samples.
Date Issued
2022-07-26
Date Acceptance
2022-06-08
Citation
Proceedings of the National Academy of Sciences of USA, 2022, 119 (30)
ISSN
0027-8424
Publisher
National Academy of Sciences
Journal / Book Title
Proceedings of the National Academy of Sciences of USA
Volume
119
Issue
30
Copyright Statement
© 2022 the Author(s). Published by PNAS.
This open access article is distributed under Creative
Commons Attribution-NonCommercial-NoDerivatives
License 4.0 (CC BY-NC-ND).
This open access article is distributed under Creative
Commons Attribution-NonCommercial-NoDerivatives
License 4.0 (CC BY-NC-ND).
Identifier
https://www.ncbi.nlm.nih.gov/pubmed/35858443
Subjects
archival data
context sensitivity
generalizability
reproducibility
research reliability
Publication Status
Published
Coverage Spatial
United States
Article Number
e2120377119
Date Publish Online
2022-07-19