Assessing the epidemiological utility of public discourse on Twitter during the COVID-19 pandemic
File(s)
Author(s)
Naidoo, Tristan
Type
Thesis
Abstract
In the context of epidemiology, accurately evaluating adherence and protective behaviours remains a substantial challenge. To explore whether social media data can provide behavioural insight during disease outbreaks, I examined whether Twitter discussions related to COVID-19, and their associated sentiment, were linked to epidemiological outcomes.
I began by presenting a novel multilingual dataset of 5.15 million tweets collected between January 2020 and January 2022 across eight countries and languages, using 843 disease-, policy-, and vaccine-related search terms.
Using an annotated subset of this dataset, I conducted an empirical evaluation to identify the most effective multilingual sentiment classification approach. I compared lexicon-based models, encoder language models (encoder LMs), and large language models (LLMs). Despite advances in natural language processing (NLP), many public health studies continue to rely on traditional lexicon-based methods. To support the adoption of more accurate, modern approaches, I have provided practical recommendations for model selection.
I applied the best-performing model from the multilingual sentiment classification to the curated dataset. Using the result, I described the temporal evolution of sentiment-stratified tweet volumes and identified the topics that dominated Twitter discussion.
Next, I assessed associations between tweet variables and epidemiological and policy variables using two approaches. First, I used Kendall’s tau to test in-sample correlations at zero lag and cross-correlations across a 14-day window. Second, I employed time series modelling to capture non-linear relationships and longer temporal dependencies, using four classical machine learning and seven deep learning models.
Looking ahead, my thesis has demonstrated that integrating social media analytics into outbreak response holds considerable promise for generating timely behavioural insights. This potential is reinforced by advances in NLP that have greatly enhanced our ability to analyse large-scale text data, and my work provides a foundation for future research to build on.
I began by presenting a novel multilingual dataset of 5.15 million tweets collected between January 2020 and January 2022 across eight countries and languages, using 843 disease-, policy-, and vaccine-related search terms.
Using an annotated subset of this dataset, I conducted an empirical evaluation to identify the most effective multilingual sentiment classification approach. I compared lexicon-based models, encoder language models (encoder LMs), and large language models (LLMs). Despite advances in natural language processing (NLP), many public health studies continue to rely on traditional lexicon-based methods. To support the adoption of more accurate, modern approaches, I have provided practical recommendations for model selection.
I applied the best-performing model from the multilingual sentiment classification to the curated dataset. Using the result, I described the temporal evolution of sentiment-stratified tweet volumes and identified the topics that dominated Twitter discussion.
Next, I assessed associations between tweet variables and epidemiological and policy variables using two approaches. First, I used Kendall’s tau to test in-sample correlations at zero lag and cross-correlations across a 14-day window. Second, I employed time series modelling to capture non-linear relationships and longer temporal dependencies, using four classical machine learning and seven deep learning models.
Looking ahead, my thesis has demonstrated that integrating social media analytics into outbreak response holds considerable promise for generating timely behavioural insights. This potential is reinforced by advances in NLP that have greatly enhanced our ability to analyse large-scale text data, and my work provides a foundation for future research to build on.
Version
Open Access
Date Issued
2025-10-08
Date Awarded
2025-12-01
Copyright Statement
Attribution-NonCommercial 4.0 International Licence (CC BY-NC)
License URL
Advisor
Ferguson, Neil
Camacho-Collados, Jose
Sponsor
Community Jameel
Publisher Department
School of Public Health
Publisher Institution
Imperial College London
Qualification Level
Doctoral
Qualification Name
Doctor of Philosophy (PhD)