Optimal significance levels and sample sizes for signal detection methods based on non-constant hazards
File(s)
Published version
Author(s)
Sauzet, Odile
Dyck, Julia
Cornelius, Victoria
Type
Journal Article
Abstract
Background and Objectives
Statistical methods for signal detection of adverse drug reactions (ADRs) in electronic health records (EHRs) need information about optimal significance levels and sample sizes to achieve sufficient power. Sauzet and Cornelius proposed tests for signal detection based on the hazard functions of Weibull type distributions (WSP tests) which use the time-to-event information available in EHRs. Optimal significance levels and sample sizes for the application of the WPS tests are derived.
Method
A simulation study was performed with a range of scenarios for sample size, rate of event due (ADRs), and not due to the drug and random time to ADR occurrence. Based on the area under the curve of the receiver operating characteristic graph, we obtain optimal significance levels of the different WSP tests for the implementation in a hypothesis free signal detection setting and approximate sample sizes required to reach a power of 80% or 90%.
Results
The dWSP–pPWSP (combination of double WSP and power WSP) test with a significance level of 0.004 was recommended. Sample sizes needed for a power of 80% were found to start at 60 events for an ADR rate equal to the background rate of 0.1. The number of events required for a background rate of 0.05 and an ADR rate equal to a 20% increase of the background rate was 900.
Conclusion
Based on this study, it is recommended to use the dWSP–pWSP test combination for signal detection with a significance level of 0.004 when the same test is applied to all adverse events not depending on rates.
Statistical methods for signal detection of adverse drug reactions (ADRs) in electronic health records (EHRs) need information about optimal significance levels and sample sizes to achieve sufficient power. Sauzet and Cornelius proposed tests for signal detection based on the hazard functions of Weibull type distributions (WSP tests) which use the time-to-event information available in EHRs. Optimal significance levels and sample sizes for the application of the WPS tests are derived.
Method
A simulation study was performed with a range of scenarios for sample size, rate of event due (ADRs), and not due to the drug and random time to ADR occurrence. Based on the area under the curve of the receiver operating characteristic graph, we obtain optimal significance levels of the different WSP tests for the implementation in a hypothesis free signal detection setting and approximate sample sizes required to reach a power of 80% or 90%.
Results
The dWSP–pPWSP (combination of double WSP and power WSP) test with a significance level of 0.004 was recommended. Sample sizes needed for a power of 80% were found to start at 60 events for an ADR rate equal to the background rate of 0.1. The number of events required for a background rate of 0.05 and an ADR rate equal to a 20% increase of the background rate was 900.
Conclusion
Based on this study, it is recommended to use the dWSP–pWSP test combination for signal detection with a significance level of 0.004 when the same test is applied to all adverse events not depending on rates.
Date Issued
2024-11-01
Date Acceptance
2024-06-11
Citation
Drug Safety, 2024, 47 (11), pp.1149-1156
ISSN
0114-5916
Publisher
Springer
Start Page
1149
End Page
1156
Journal / Book Title
Drug Safety
Volume
47
Issue
11
Copyright Statement
© The Author(s) 2024 Open Access This article is licensed under a Creative Commons Attribution-NonCommercial 4.0 International License, which permits any non-commercial use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by-nc/4.0/.
License URL
Identifier
https://www.ncbi.nlm.nih.gov/pubmed/38982034
PII: 10.1007/s40264-024-01460-2
Subjects
DATABASES
Life Sciences & Biomedicine
Pharmacology & Pharmacy
Public, Environmental & Occupational Health
Science & Technology
Toxicology
Publication Status
Published
Coverage Spatial
New Zealand
Date Publish Online
2024-07-09