1
IRUS Total
Downloads
  Altmetric

Measuring vaccine efficacy against infection and disease in clinical trials: sources and magnitude of bias in COVID-19 vaccine efficacy estimates

File Description SizeFormat 
Williams et al - Main text - CID 2nd submission (clean).docxAccepted version1.06 MBMicrosoft WordView/Open
Williams et al - Appendix - CID 2nd Submission (clean).docxSupporting information810.45 kBMicrosoft WordView/Open
Title: Measuring vaccine efficacy against infection and disease in clinical trials: sources and magnitude of bias in COVID-19 vaccine efficacy estimates
Authors: Williams, LR
Ferguson, NM
Donnelly, CA
Grassly, NC
Item Type: Journal Article
Abstract: BACKGROUND: Phase III trials have estimated COVID-19 vaccine efficacy (VE) against symptomatic and asymptomatic infection. We explore the direction and magnitude of potential biases in these estimates and their implications for vaccine protection against infection and against disease in breakthrough infections. METHODS: We developed a mathematical model that accounts for natural and vaccine-induced immunity, changes in serostatus and imperfect sensitivity and specificity of tests for infection and antibodies. We estimated expected biases in VE against symptomatic, asymptomatic and any SARS͏CoV2 infections and against disease following infection for a range of vaccine characteristics and measurement approaches, and the likely overall biases for published trial results that included asymptomatic infections. RESULTS: VE against asymptomatic infection measured by PCR or serology is expected to be low or negative for vaccines that prevent disease but not infection. VE against any infection is overestimated when asymptomatic infections are less likely to be detected than symptomatic infections and the vaccine protects against symptom development. A competing bias towards underestimation arises for estimates based on tests with imperfect specificity, especially when testing is performed frequently. Our model indicates considerable uncertainty in Oxford-AstraZeneca ChAdOx1 and Janssen Ad26.COV2.S VE against any infection, with slightly higher than published, bias-adjusted values of 59.0% (95% uncertainty interval [UI] 38.4 to 77.1) and 70.9% (95% UI 49.8 to 80.7) respectively. CONCLUSIONS: Multiple biases are likely to influence COVID-19 VE estimates, potentially explaining the observed difference between ChAdOx1 and Ad26.COV2.S vaccines. These biases should be considered when interpreting both efficacy and effectiveness study results.
Issue Date: 26-Oct-2021
Date of Acceptance: 1-Oct-2021
URI: http://hdl.handle.net/10044/1/92427
DOI: 10.1093/cid/ciab914
ISSN: 1058-4838
Publisher: Oxford University Press
Journal / Book Title: Clinical Infectious Diseases
Copyright Statement: © The Author(s) 2021. Published by Oxford University Press for the Infectious Diseases Society of America. This is an Open Access article distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0/), which permits unrestricted reuse, distribution, and reproduction in any medium, provided the original work is properly cited.
Sponsor/Funder: Medical Research Council (MRC)
Funder's Grant Number: MR/R015600/1
Keywords: Bias
COVID-19
Clinical Trial
SARS-CoV-2
Vaccine efficacy
Bias
COVID-19
Clinical Trial
SARS-CoV-2
Vaccine efficacy
Microbiology
06 Biological Sciences
11 Medical and Health Sciences
Notes: BACKGROUND: Phase III trials have estimated COVID-19 vaccine efficacy (VE) against symptomatic and asymptomatic infection. We explore the direction and magnitude of potential biases in these estimates and their implications for vaccine protection against infection and against disease in breakthrough infections. METHODS: We developed a mathematical model that accounts for natural and vaccine-induced immunity, changes in serostatus and imperfect sensitivity and specificity of tests for infection and antibodies. We estimated expected biases in VE against symptomatic, asymptomatic and any SARS͏CoV2 infections and against disease following infection for a range of vaccine characteristics and measurement approaches, and the likely overall biases for published trial results that included asymptomatic infections. RESULTS: VE against asymptomatic infection measured by PCR or serology is expected to be low or negative for vaccines that prevent disease but not infection. VE against any infection is overestimated when asymptomatic infections are less likely to be detected than symptomatic infections and the vaccine protects against symptom development. A competing bias towards underestimation arises for estimates based on tests with imperfect specificity, especially when testing is performed frequently. Our model indicates considerable uncertainty in Oxford-AstraZeneca ChAdOx1 and Janssen Ad26.COV2.S VE against any infection, with slightly higher than published, bias-adjusted values of 59.0% (95% uncertainty interval [UI] 38.4 to 77.1) and 70.9% (95% UI 49.8 to 80.7) respectively. CONCLUSIONS: Multiple biases are likely to influence COVID-19 VE estimates, potentially explaining the observed difference between ChAdOx1 and Ad26.COV2.S vaccines. These biases should be considered when interpreting both efficacy and effectiveness study results.
Publication Status: Published online
Online Publication Date: 2021-10-26
Appears in Collections:Department of Infectious Diseases
Imperial College London COVID-19
School of Public Health



This item is licensed under a Creative Commons License Creative Commons