Application of machine learning to equity indices
File(s)
Author(s)
Mostafavi, Seyed Mostafa
Type
Thesis or dissertation
Abstract
This thesis presents a comprehensive investigation of Machine Learning (ML) applications for equity market forecasting,
focusing on the S&P 500 and Dubai Financial Market (DFM) indices. Addressing the limitations of traditional forecasting
methods under volatile, nonlinear, and globally interconnected market conditions, it proposes hybrid frameworks combining
ensemble machine learning techniques with deep learning architectures to improve predictive accuracy, robustness, and
interpretability across developed and emerging markets.
The thesis reviews index forecasting methods and motivates the adoption of ML techniques capable of modelling complex
nonlinear relationships. It introduces Random Forest (RF), Support Vector Regression (SVR), XGBoost, and Long Short-Term
Memory (LSTM) networks, together with Principal Component Analysis (PCA), permutation importance, and SHAP for feature
selection and model interpretation.
Two complementary forecasting strategies are developed for the S&P 500. The first employs global equity indices within a
hybrid RF-LSTM framework to capture international market spillovers, while the second applies feature selection to 88 technical
indicators and compares ensemble and deep learning models. Both approaches consistently outperform conventional methods
in forecasting accuracy and robustness.
For the DFM Index, a novel two-stage hybrid framework integrates SVR, RF, and XGBoost with LSTM-based residual
correction, effectively modelling both cross-sectional and temporal market dynamics. Extensive empirical evaluation
demonstrates statistically significant improvements over standalone machine learning models.
The findings emphasise the value of hybrid ML frameworks, model interpretability, and international market
interdependencies. The proposed methodologies provide practical insights for investors, asset managers, and policymakers while
offering a flexible foundation for future research, including transformer-based architectures and real-time sentiment analysis.
Overall, this thesis advances the application of Machine Learning to financial forecasting in both developed and emerging equity
markets.
focusing on the S&P 500 and Dubai Financial Market (DFM) indices. Addressing the limitations of traditional forecasting
methods under volatile, nonlinear, and globally interconnected market conditions, it proposes hybrid frameworks combining
ensemble machine learning techniques with deep learning architectures to improve predictive accuracy, robustness, and
interpretability across developed and emerging markets.
The thesis reviews index forecasting methods and motivates the adoption of ML techniques capable of modelling complex
nonlinear relationships. It introduces Random Forest (RF), Support Vector Regression (SVR), XGBoost, and Long Short-Term
Memory (LSTM) networks, together with Principal Component Analysis (PCA), permutation importance, and SHAP for feature
selection and model interpretation.
Two complementary forecasting strategies are developed for the S&P 500. The first employs global equity indices within a
hybrid RF-LSTM framework to capture international market spillovers, while the second applies feature selection to 88 technical
indicators and compares ensemble and deep learning models. Both approaches consistently outperform conventional methods
in forecasting accuracy and robustness.
For the DFM Index, a novel two-stage hybrid framework integrates SVR, RF, and XGBoost with LSTM-based residual
correction, effectively modelling both cross-sectional and temporal market dynamics. Extensive empirical evaluation
demonstrates statistically significant improvements over standalone machine learning models.
The findings emphasise the value of hybrid ML frameworks, model interpretability, and international market
interdependencies. The proposed methodologies provide practical insights for investors, asset managers, and policymakers while
offering a flexible foundation for future research, including transformer-based architectures and real-time sentiment analysis.
Overall, this thesis advances the application of Machine Learning to financial forecasting in both developed and emerging equity
markets.
Version
Open Access
Date Issued
2025-08-07
Date Awarded
2026-07-01
Copyright Statement
Attribution-NonCommercial 4.0 International Licence (CC BY-NC)
License URL
Advisor
Edalat, Abbas
Publisher Department
Department of Computing
Publisher Institution
Imperial College London
Qualification Level
Doctoral
Qualification Name
Doctor of Philosophy (PhD)