Data representation is crucial for machine learning (ML) tasks. The underlying challenge when analysing temporal data such as time series and data streams is identifying a compact set of representations that encapsulate their complex characteristics. A suitable feature representation should capture the discriminative and statistical properties of the original data for classification and anomaly detection while considering the underlying nature and size of the data.

In this thesis, a novel temporal data classification framework is developed to discover and analyse data representations in the probability domain using Wavelet Density Estimators (WDEs), which constitute key components in constructing a novel Wavelet Probabilistic Neural Network (WPNN). The proposed WPNN can operate alongside various deep neural networks (DNNs), enabling automatic feature discovery and analysis of temporal data within the probability and temporal domains. Three algorithms are devised to perform classification and anomaly detection.

The first algorithm focuses on data stream classification, employing a novel WPNN to generate data representations in the probability domain. This algorithm has a bounded number of basis functions and operates with constant time and space complexities regardless of the original sample space size, making it well-suited for analysing online stationary and non-stationary data streams.

The second algorithm addresses time series with imbalanced distributions. It constructs a low-dimensional feature space employing Recurrent Neural Networks (RNNs) and models this space with a WPNN ensemble. It effectively handles high-dimensional data and addresses challenges of data scarcity, noise perturbation, and non-stationarity, making it particularly suitable for anomaly detection.

The third algorithm focuses on multivariate time series classification. It uses RNNs and Convolutional Neural Networks (CNNs), along with novel Wavelet Probabilistic Pattern Generator and Analyser modules, to create a comprehensive feature space. These modules enhance class boundary delineation by capturing unique data characteristics, thus improving robustness to data scarcity, noise perturbation, and non-stationarity.