Are large randomized controlled trials in severe sepsis and septic shock statistically disadvantaged by repeated inadvertent underestimates of required sample size
File(s)
Published version
Author(s)
Wong, JLC
Mason, Alexina
Gordon, Anthony
Brett, Stephen
Type
Journal Article
Abstract
Objectives: We sought to understand why randomized controlled trials in septic shock have failed to
demonstrate effectiveness in the face of improving overall outcomes for patients and seemingly promising
results of early phase trials of interventions.
Design: We performed a retrospective analysis of large critical care trials of severe sepsis and septic shock.
Data were collected from the primary trial manuscripts, pre-published statistical plans or by direct
communication with corresponding authors.
Setting: Critical care randomized control trials in severe sepsis and septic shock.
Participants: 14619 patients randomized in 13 trials published between 2005 to 2015, enrolling greater than
500 patients and powered to a primary outcome of mortality.
Intervention: Multiple interventions including the evaluation of treatment strategies and novel therapeutics.
Primary and secondary outcome measures: Our primary outcome measure was the difference between the
anticipated and actual control arm mortality. Secondary analysis examined the actual effect size and the
anticipated effect size employed in sample size calculation.
Results: In this post-hoc analysis of 13 trials with 14 619 patients randomised, we highlight a global
tendency to overestimate control arm mortality in estimating sample size (absolute difference 9.8%, 95%
confidence interval, -14.7% to -5%, p<0.001). When we compared anticipated and actual effect size of a treatment there was also a substantial overestimation in proposed values (absolute difference 7.4%, 95%
confidence interval -9.0% to -5.8%, p<0.0001).
Conclusions: An interpretation of our results is that trials are consistently underpowered in the planning
phase by employing erroneous variables to calculate a satisfactory sample size. Our analysis cannot
establish if, given a larger sample size, a trial would have had a positive result. It is disappointing so many
promising phase II results have not translated into durable phase III outcomes. It is possible that our current
framework has biased us towards discounting potentially life-saving treatments.
demonstrate effectiveness in the face of improving overall outcomes for patients and seemingly promising
results of early phase trials of interventions.
Design: We performed a retrospective analysis of large critical care trials of severe sepsis and septic shock.
Data were collected from the primary trial manuscripts, pre-published statistical plans or by direct
communication with corresponding authors.
Setting: Critical care randomized control trials in severe sepsis and septic shock.
Participants: 14619 patients randomized in 13 trials published between 2005 to 2015, enrolling greater than
500 patients and powered to a primary outcome of mortality.
Intervention: Multiple interventions including the evaluation of treatment strategies and novel therapeutics.
Primary and secondary outcome measures: Our primary outcome measure was the difference between the
anticipated and actual control arm mortality. Secondary analysis examined the actual effect size and the
anticipated effect size employed in sample size calculation.
Results: In this post-hoc analysis of 13 trials with 14 619 patients randomised, we highlight a global
tendency to overestimate control arm mortality in estimating sample size (absolute difference 9.8%, 95%
confidence interval, -14.7% to -5%, p<0.001). When we compared anticipated and actual effect size of a treatment there was also a substantial overestimation in proposed values (absolute difference 7.4%, 95%
confidence interval -9.0% to -5.8%, p<0.0001).
Conclusions: An interpretation of our results is that trials are consistently underpowered in the planning
phase by employing erroneous variables to calculate a satisfactory sample size. Our analysis cannot
establish if, given a larger sample size, a trial would have had a positive result. It is disappointing so many
promising phase II results have not translated into durable phase III outcomes. It is possible that our current
framework has biased us towards discounting potentially life-saving treatments.
Date Issued
2018-08-29
Date Acceptance
2018-06-08
Citation
BMJ Open, 2018, 8
ISSN
2044-6055
Publisher
BMJ Journals
Journal / Book Title
BMJ Open
Volume
8
Copyright Statement
© Author(s) (or their employer(s)) 2018. Re-use permitted under CC BY-NC. No commercial re-use. See rights and permissions. Published by BMJ. This is an open access article distributed in accordance with the Creative Commons Attribution Non Commercial (CC BY-NC 4.0) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited, appropriate credit is given, any changes made indicated, and the use is non-commercial. See: http://creativecommons.org/licenses/by-nc/4.0/.
Subjects
Science & Technology
Life Sciences & Biomedicine
Medicine, General & Internal
General & Internal Medicine
clinical trials
sepsis
septic shock
randomized control trials
sample size calculation
GOAL-DIRECTED RESUSCITATION
INTENSIVE INSULIN THERAPY
ADULTS
MORTALITY
DESIGNS
CARE
clinical trials
randomized control trials
sample size calculation
sepsis
septic shock
Publication Status
Published
Article Number
e020068
Date Publish Online
2018-08-29