Directing scientists away from potentially biased publications: the role of systematic reviews in health care

Abstract

Despite increasing concerns about the validity of published research, the issue of how the scientific community can maintain a high-quality body of research is not well understood. We consider the case of systematic reviews in health care, and explore whether risk of bias ratings communicated within these reviews may help shift scientists’ attention towards published research that is at a low risk of bias. We focus on publications deemed at risk of bias due to selective reporting; that is, scientific articles with high chances of systematic errors in the published research findings due to flaws in the reporting. Using a matched-sample control group we find that, after potential bias is signalled in systematic reviews, publications at high risk of bias attract less attention – as indicated by fewer follow-on citations – when compared to a control group of low risk of bias publications. We extend our analysis by considering those cases where risk of bias is unclear, and by examining how different features of the rating system may affect the magnitude of the main effect. The findings provide evidence about whether systematic reviews can play a role in signalling biases in the scientific literature, over and above their established role of synthesising prior research.