Quantifying and addressing the prevalence and bias of study designs in the environmental and social sciences
File(s)
Published version
Author(s)
Type
Journal Article
Abstract
Building trust in science and evidence-based decision-making depends heavily on the credibility of studies and their findings. Researchers employ many different study designs that vary in their risk of bias to evaluate the true effect of interventions or impacts. Here, we empirically quantify, on a large scale, the prevalence of different study designs and the magnitude of bias in their estimates. Randomised designs and controlled observational designs with pre-intervention sampling were used by just 23% of intervention studies in biodiversity conservation, and 36% of intervention studies in social science. We demonstrate, through pairwise within-study comparisons across 49 environmental datasets, that these types of designs usually give less biased estimates than simpler observational designs. We propose a model-based approach to combine study estimates that may suffer from different levels of study design bias, discuss the implications for evidence synthesis, and how to facilitate the use of more credible study designs.
Date Issued
2020-12-11
Date Acceptance
2020-11-13
Citation
Nature Communications, 2020, 11
ISSN
2041-1723
Publisher
Nature Portfolio
Journal / Book Title
Nature Communications
Volume
11
Copyright Statement
© The Author(s) 2020 Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made. The images or other third party material in this article are included in the article’s Creative Commons license, unless indicated otherwise in a credit line to the material. If material is not included in the article’s Creative Commons license and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this license, visit http://creativecommons.org/licenses/by/4.0/.
License URL
Identifier
https://www.webofscience.com/api/gateway?GWVersion=2&SrcApp=PARTNER_APP&SrcAuth=LinksAMR&KeyUT=WOS:000600150800017&DestLinkType=FullRecord&DestApp=ALL_WOS&UsrCustomerID=a2bf6146997ec60c407a63945d4e92bb
PII: 10.1038/s41467-020-20142-y
Subjects
BIODIVERSITY
CAUSAL INFERENCE
DIFFERENCE-IN-DIFFERENCES
ECOLOGY
GUIDE
HEALTH
IMPACTS
METAANALYSIS
MODELS
Multidisciplinary Sciences
Science & Technology
Science & Technology - Other Topics
Publication Status
Published
Coverage Spatial
England
Article Number
6377
Date Publish Online
2020-12-11