Using text mining to track outbreak trends in global surveillance of emerging diseases: ProMED-mail
File(s)
Published version
Author(s)
You, Jingxian
Expert, Paul
Costelloe, Ceire
Type
Journal Article
Abstract
ProMED-mail (Program for Monitoring Emerging Disease) is an international disease outbreak monitoring and early warning system. Every year, users contribute thousands of reports that include reference to infectious diseases and toxins. However, due to the uneven distribution of the reports for each disease, traditional statistics-based text mining techniques, represented by term frequency-related algorithm, are not suitable. Thus, we conducted a study in three steps (i) report filtering, (ii) keyword extraction from reports and finally (iii) word co-occurrence network analysis to fill the gap between ProMED and its utilization. The keyword extraction was performed with the TextRank algorithm, keywords co-occurrence networks were then produced using the top keywords from each document and multiple network centrality measures were computed to analyse the co-occurrence networks. We used two major outbreaks in recent years, Ebola, 2014 and Zika 2015, as cases to illustrate and validate the process. We found that the extracted information structures are consistent with World Health Organisation description of the timeline and phases of the epidemics. Our research presents a pipeline that can extract and organize the information to characterize the evolution of epidemic outbreaks. It also highlights the potential for ProMED to be utilized in monitoring, evaluating and improving responses to outbreaks.
Date Issued
2021-07-01
Date Acceptance
2021-05-14
Citation
Journal of the Royal Statistical Society Series A: Statistics in Society, 2021, 184 (4), pp.1245-1259
ISSN
0964-1998
Publisher
Royal Statistical Society
Start Page
1245
End Page
1259
Journal / Book Title
Journal of the Royal Statistical Society Series A: Statistics in Society
Volume
184
Issue
4
Copyright Statement
© 2021 The Authors. Journal of the Royal Statistical Society: Series A (Statistics in Society) published by John Wiley & Sons Ltd on behalf of Royal Statistical Society. This is an open access article under the terms of the Creative Commons Attribution-NonCommercial License, which permits use, distribution and reproduction in any medium, provided the original work is properly cited and is not used for commercial purposes.
License URL
Sponsor
Imperial College Healthcare NHS Trust- BRC Funding
Identifier
http://gateway.webofknowledge.com/gateway/Gateway.cgi?GWVersion=2&SrcApp=PARTNER_APP&SrcAuth=LinksAMR&KeyUT=WOS:000668499300001&DestLinkType=FullRecord&DestApp=ALL_WOS&UsrCustomerID=1ba7043ffcc86c417c072aa74d649202
Grant Number
RDA02
Subjects
Social Sciences
Science & Technology
Physical Sciences
Social Sciences, Mathematical Methods
Statistics & Probability
Mathematical Methods In Social Sciences
Mathematics
communicable disease
data mining
emerging
EARLY WARNING SYSTEM
Publication Status
Published online
Date Publish Online
2021-07-01