Long-term operational spatiotemporal forecasting by combining machine learning and data assimilation approaches
File(s)
Author(s)
Cai, Shengjuan
Type
Thesis
Abstract
Spatiotemporal forecasting is the key problem of many real-world applications, such as weather and air quality forecasting. Traditional physics-based forecasting processes for complex dynamic systems, which require high spatial and temporal resolutions, can be computationally intensive and time-consuming. The advent of machine learning (ML) techniques, particularly neural networks, offers a promising alternative by automatically learning and effectively modelling complex and non-linear relationships from large datasets. Although advanced neural networks have shown exciting performance in various forecasting tasks, they still struggle with long-term forecasting due to uncertainty and error accumulations.
Data assimilation (DA) is a critical technique for reducing model uncertainty and improving forecasting accuracy by optimally integrating observations into model forecasts. Although DA is a crucial concept in atmospheric science and oceanography, it remains underexplored in ML-based forecasting tasks. Meanwhile, commonly used ensemble-based and variational DA methods face challenges, such as reliance on linear and Gaussian assumptions, high computational demand in large-scale problems, and complexity in adjoint calculations. In this study, I primarily focus on four challenges: 1. Error and uncertainty accumulations in ML-based long-term forecasting; 2. Improving the accuracy and efficiency of DA in large-scale systems with sparse observations; 3. Online operational forecasting in complex spatiotemporal systems; 4. Realistic application of ML-based operational forecasting, such as in regional and global PM2.5 (Particulate Matter with a diameter of 2.5 micrometres or smaller) forecasting.
The key contributions include: 1. Development of a hybrid data-driven and data assimilation method to mitigate error and uncertainty accumulations in ML-based long-term forecasting; 2. Development of a Neural Network-based Data Assimilation (DANet) for large-scale systems with sparse observations; 3. Development of Dual Deep Neural Network (D-DNet) for online operational forecasting in complex spatiotemporal systems; 4. Application of these developed ML-based methods in realistic scenarios, such as operational PM2.5 forecasting on both regional and global scales.
Data assimilation (DA) is a critical technique for reducing model uncertainty and improving forecasting accuracy by optimally integrating observations into model forecasts. Although DA is a crucial concept in atmospheric science and oceanography, it remains underexplored in ML-based forecasting tasks. Meanwhile, commonly used ensemble-based and variational DA methods face challenges, such as reliance on linear and Gaussian assumptions, high computational demand in large-scale problems, and complexity in adjoint calculations. In this study, I primarily focus on four challenges: 1. Error and uncertainty accumulations in ML-based long-term forecasting; 2. Improving the accuracy and efficiency of DA in large-scale systems with sparse observations; 3. Online operational forecasting in complex spatiotemporal systems; 4. Realistic application of ML-based operational forecasting, such as in regional and global PM2.5 (Particulate Matter with a diameter of 2.5 micrometres or smaller) forecasting.
The key contributions include: 1. Development of a hybrid data-driven and data assimilation method to mitigate error and uncertainty accumulations in ML-based long-term forecasting; 2. Development of a Neural Network-based Data Assimilation (DANet) for large-scale systems with sparse observations; 3. Development of Dual Deep Neural Network (D-DNet) for online operational forecasting in complex spatiotemporal systems; 4. Application of these developed ML-based methods in realistic scenarios, such as operational PM2.5 forecasting on both regional and global scales.
Version
Open Access
Date Issued
2024-02-25
Date Awarded
2024-07-01
Copyright Statement
Creative Commons Attribution NonCommercial Licence
License URL
Advisor
Fang, Fangxin
Wang, Yanghua
Sponsor
Resource Geophysics Academy
Publisher Department
Earth Science & Engineering
Publisher Institution
Imperial College London
Qualification Level
Doctoral
Qualification Name
Doctor of Philosophy (PhD)