Diagnosing early-onset neonatal sepsis in low-resource settings: development of a multivariable prediction model

Objective To develop a clinical prediction model to diagnose neonatal sepsis in low-resource settings. Design Secondary analysis of data collected by the Neotree digital health system from 01/02/2019 to 31/03/2020. We used multivariable logistic regression with candidate predictors identified from expert opinion and literature review. Missing data were imputed using multivariate imputation and model performance was evaluated in the derivation cohort. Setting A tertiary neonatal unit at Sally Mugabe Central Hospital, Zimbabwe. Patients We included 2628 neonates aged <72 hours, gestation [≥]32+0 weeks and birth weight [≥]1500 grams. Interventions Participants received standard care as no specific interventions were dictated by the study protocol. Main outcome measures Clinical early-onset neonatal sepsis (within the first 72 hours of life), defined by the treating consultant neonatologist. Results Clinical early-onset sepsis was diagnosed in 297 neonates (11.3%). The optimal model included eight predictors: maternal fever, offensive liquor, prolonged rupture of membranes, neonatal temperature, respiratory rate, activity, chest retractions and grunting. Receiver operating characteristic analysis gave an area under the curve of 0.736 (95% confidence interval 0.701-0.772). For a sensitivity of 95% (92-97%), corresponding specificity was 11% (10-13%), positive predictive value 12% (11-13%), negative predictive value 95% (92-97%), positive likelihood ratio 1.1 (95% CI 1.0-1.1), and negative likelihood ratio 0.4 (95% CI 0.3-0.6). Conclusions Our clinical prediction model achieved high sensitivity with modest specificity, suggesting it may be suited to excluding early-onset sepsis. Future work will validate and refine this model before considering it for clinical use within the Neotree.

Neonatal sepsis caused 15% of the 2.5 million neonatal deaths worldwide in 2018 and has a 94 mortality rate of 110-190 per 1000 livebirths. [1,2] It can be difficult to diagnose as the 95 clinical features overlap with non-infectious diseases. [3] Failing to treat sepsis with timely 96 antimicrobials increases the risk of death or disability, but empirical antimicrobial therapy in 97 non-infected neonates contributes to antimicrobial resistance and adverse outcomes. [4,5] 98 99 Isolating a pathogenic organism from a normally sterile site is the gold standard diagnostic 100 method, [6] but has limitations. In low-resource settings (LRS), cultures and blood counts are 101 often unavailable, [7] or turnaround times are too long to usefully inform management. [8,9] 102 Blood cultures have high sensitivity provided sufficient inoculate volume is obtained, but 103 sampling can be difficult in unwell neonates. [10] Therefore, clinicians may diagnose sepsis 104 and initiate empirical antimicrobial therapy despite negative cultures, based on clinical 105 presentation, risk factors and/or raised inflammatory markers. This is often called 'culture-106 negative' sepsis and up to 16 times more neonates receive antibiotics for culture-negative 107 sepsis than for sepsis with a positive culture. [11] Diagnostic challenges are increased in LRS 108 where early neonatal care may be led by less experienced healthcare professionals (HCPs) 109 without immediate local senior support. [8] 110 111 Clinical prediction models combine patient or disease characteristics to estimate the 112 probability of a diagnosis or outcome. [12] Models to diagnose neonatal sepsis may improve 113 diagnostic accuracy and rationalise antibiotic use. In LRS, they could provide clinical 114 decision support for less experienced HCPs, especially if models do not require laboratory 115 tests. Several existing models estimate the probability of neonatal sepsis, [13] for example, the 116 . CC-BY 4.0 International license It is made available under a is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. (which was not certified by peer review) preprint We evaluated performance of the optimal model in the derivation cohort. Discrimination was 204 quantified by plotting a receiver operating characteristic (ROC) curve in each imputed 205 dataset. We pooled the area under the curve (AUC) and variance across imputed datasets 206 using Rubin's rules. [24] Discrimination was visualised with a box plot and density plot of the 207 distributions of predicted probabilities for each observed outcome group (in the single dataset 208 used for model selection). We calculated Yates' discrimination slope as the absolute 209 difference in mean predicted probabilities between the two observed outcome groups. [25] 210 Sensitivity, specificity, predictive values, and likelihood ratios of the optimal model were 211 estimated in the single dataset used for model selection. These metrics are presented for the 212 'optimal' probability threshold according to Youden's J statistic, [26] and for thresholds 213 corresponding to sensitivities of 80, 85, 90 and 95%. Confidence intervals for likelihood 214 ratios and Yates' discrimination slope were obtained using bootstrap with 10,000 resamples 215 (basic method or normal approximation). [ shown in Additional File 2 were generated at the time of data import and, thus, were not 223 known to anyone outside of the research group. 224 . CC-BY 4.0 International license It is made available under a is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity.  Fisher's exact test indicated a significant difference in the distribution between the two 248 . CC-BY 4.0 International license It is made available under a is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. (which was not certified by peer review) preprint The optimal model included eight predictors: temperature at admission, respiratory rate, 265 maternal fever during labour, offensive liquor, premature rupture of membranes, activity, 266 chest retractions, and grunting (Table 4). It can be written as 267

269
. CC-BY 4.0 International license It is made available under a is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity.

279
The 'optimal' classification threshold was 0.121 (i.e. 12.1% predicted probability of clinical 280 EOS) yielding sensitivity 65% (95% CI 59-70%) and specificity 74% (95% CI 72-75%) 281 (Table 5). For a sensitivity of 95%, the corresponding classification threshold was 0.034 282 giving sensitivity 95% (95% CI 92-97%) and specificity 11% (95% CI 10-13%) ( Table 5). 283 . CC-BY 4.0 International license It is made available under a is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. (which was not certified by peer review) preprint  [29] Comparison of performance between models is 308 . CC-BY 4.0 International license It is made available under a is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. . CC-BY 4.0 International license It is made available under a is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. (which was not certified by peer review) preprint An open-source, anonymised research database is planned as part of the wider Neotree 439 project. Currently, sharing of deidentified individual participant data will be considered on a 440 case-by-case basis. 441 . CC-BY 4.0 International license It is made available under a is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. (which was not certified by peer review) preprint The copyright holder for this this version posted November 20, 2022. ;    Analyses were performed on the complete data after multiple imputation of missing values. CI = confidence interval; OR = odds ratio; SE = standard error.
. CC-BY 4.0 International license It is made available under a is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. (which was not certified by peer review) preprint The copyright holder for this this version posted November 20, 2022. ; https://doi.org/10.1101/2022.11.19.22282335 doi: medRxiv preprint *The 'optimal' threshold according to Youden's J statistic. Data are presented for the single dataset used for model selection. Numbers in brackets represent the 95% confidence intervals. LR+ = positive likelihood ratio; LR-= negative likelihood ratio; PPV = positive predictive value; NPV = negative predictive value; sens = sensitivity; spec = specificity.
. CC-BY 4.0 International license It is made available under a is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. (which was not certified by peer review) preprint The copyright holder for this this version posted November 20, 2022. ; https://doi.org/10.1101/2022.11.19.22282335 doi: medRxiv preprint . CC-BY 4.0 International license It is made available under a is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. . CC-BY 4.0 International license It is made available under a is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. (which was not certified by peer review) preprint The copyright holder for this this version posted November 20, 2022. ; . CC-BY 4.0 International license It is made available under a is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. (which was not certified by peer review) preprint