A systematic review of sample size estimation accuracy on power in malaria cluster randomised trials measuring epidemiological outcomes
File(s)s12874-024-02361-9.pdf (3.97 MB)
Published version
Author(s)
Biggs, Joseph
Challenger, Joseph
Hellewell, Joel
Churcher, Thomas Simon
Cook, Jackie
Type
Journal Article
Abstract
Introduction
Cluster randomised trials (CRTs) are the gold standard for measuring the community-wide impacts of malaria control tools. CRTs rely on well-defined sample size estimations to detect statistically significant effects of trialled interventions, however these are often predicted poorly by triallists. Here, we review the accuracy of predicted parameters used in sample size calculations for malaria CRTs with epidemiological outcomes.
Methods
We searched for published malaria CRTs using four online databases in March 2022. Eligible trials included those with malaria-specific epidemiological outcomes which randomised at least six geographical clusters to study arms. Predicted and observed sample size parameters were extracted by reviewers for each trial. Pair-wise Spearman’s correlation coefficients (rs) were calculated to assess the correlation between predicted and observed control-arm outcome measures and effect sizes (relative percentage reductions) between arms. Among trials which retrospectively calculated an estimate of heterogeneity in cluster outcomes, we recalculated study power according to observed trial estimates.
Results
Of the 1889 records identified and screened, 108 articles were eligible and comprised of 71 malaria CRTs. Among 91.5% (65/71) of trials that included sample size calculations, most estimated cluster heterogeneity using the coefficient of variation (k) (80%, 52/65) which were often predicted without using prior data (67.7%, 44/65). Predicted control-arm prevalence moderately correlated with observed control-arm prevalence (rs: 0.44, [95%CI: 0.12,0.68], p-value < 0.05], with 61.2% (19/31) of prevalence estimates overestimated. Among the minority of trials that retrospectively calculated cluster heterogeneity (20%, 13/65), empirical values contrasted with those used in sample size estimations and often compromised study power. Observed effect sizes were often smaller than had been predicted at the sample size stage (72.9%, 51/70) and were typically higher in the first, compared to the second, year of trials. Overall, effect sizes achieved by malaria interventions tested in trials decreased between 1995 and 2021.
Conclusions
Study findings reveal sample size parameters in malaria CRTs were often inaccurate and resulted in underpowered studies. Future trials must strive to obtain more representative epidemiological sample size inputs to ensure interventions against malaria are adequately evaluated.
Registration
This review is registered with PROSPERO (CRD42022315741).
Cluster randomised trials (CRTs) are the gold standard for measuring the community-wide impacts of malaria control tools. CRTs rely on well-defined sample size estimations to detect statistically significant effects of trialled interventions, however these are often predicted poorly by triallists. Here, we review the accuracy of predicted parameters used in sample size calculations for malaria CRTs with epidemiological outcomes.
Methods
We searched for published malaria CRTs using four online databases in March 2022. Eligible trials included those with malaria-specific epidemiological outcomes which randomised at least six geographical clusters to study arms. Predicted and observed sample size parameters were extracted by reviewers for each trial. Pair-wise Spearman’s correlation coefficients (rs) were calculated to assess the correlation between predicted and observed control-arm outcome measures and effect sizes (relative percentage reductions) between arms. Among trials which retrospectively calculated an estimate of heterogeneity in cluster outcomes, we recalculated study power according to observed trial estimates.
Results
Of the 1889 records identified and screened, 108 articles were eligible and comprised of 71 malaria CRTs. Among 91.5% (65/71) of trials that included sample size calculations, most estimated cluster heterogeneity using the coefficient of variation (k) (80%, 52/65) which were often predicted without using prior data (67.7%, 44/65). Predicted control-arm prevalence moderately correlated with observed control-arm prevalence (rs: 0.44, [95%CI: 0.12,0.68], p-value < 0.05], with 61.2% (19/31) of prevalence estimates overestimated. Among the minority of trials that retrospectively calculated cluster heterogeneity (20%, 13/65), empirical values contrasted with those used in sample size estimations and often compromised study power. Observed effect sizes were often smaller than had been predicted at the sample size stage (72.9%, 51/70) and were typically higher in the first, compared to the second, year of trials. Overall, effect sizes achieved by malaria interventions tested in trials decreased between 1995 and 2021.
Conclusions
Study findings reveal sample size parameters in malaria CRTs were often inaccurate and resulted in underpowered studies. Future trials must strive to obtain more representative epidemiological sample size inputs to ensure interventions against malaria are adequately evaluated.
Registration
This review is registered with PROSPERO (CRD42022315741).
Date Issued
2024-10-15
Date Acceptance
2024-10-01
Citation
BMC Medical Research Methodology, 2024, 24
ISSN
1471-2288
Publisher
BMC
Journal / Book Title
BMC Medical Research Methodology
Volume
24
Copyright Statement
© The Author(s) 2024. Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article’s Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article’s Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/.
License URL
Identifier
https://bmcmedresmethodol.biomedcentral.com/articles/10.1186/s12874-024-02361-9
Publication Status
Published
Article Number
238
Date Publish Online
2024-10-15