Sampling, testing and estimation with stein reproducing kernels
File(s)
Author(s)
Liu, Xing
Type
Thesis
Abstract
Statistical models have grown increasingly complex to better capture the diverse data types present in modern applications. However, this complexity has created significant challenges for statistical inference. A key obstacle is that many of these models are unnormalized, meaning that the model likelihood can only be evaluated up to an intractable normalizing constant. In such cases, traditional methods that rely on the exact model likelihood—such as likelihood-ratio tests for model evaluation or maximum likelihood estimation for model fitting—are no longer feasible.
To address the challenges posed by unnormalized models, one promising tool is the kernel Stein discrepancy (KSD). KSD is a statistical divergence that leverages the Stein reproducing kernel—a reproducing kernel based on Stein’s method—to allow tractable computation without requiring the normalizing constant of the model likelihood. Consequently, KSD has facilitated the development of novel algorithms that can be efficiently applied to unnormalized models across various statistical tasks, including sample approximation, goodness-of-fit testing, and parameter estimation.
This thesis focuses on identifying the limitations of existing KSD-based methods and proposing novel solutions. First, we focus on KSD-based methods for sample approximation. We address the curse of dimensionality inherent in a widely used KSD-based sample approximation method, and propose an extension that mitigates this issue by using linear projections. Second, we turn our attention to goodness-of-fit tests based on KSD. We identify two key limitations of such tests: their lack of test power against multi-modal alternatives, and their lack of robustness under data contamination. For each limitation, we introduce new solutions. Finally, we explore the use of KSD in parameter estimation. We compare the relative statistical efficiency of KSD-based estimators to alternative methods, providing insights into how the kernel choice can affect their efficiency.
To address the challenges posed by unnormalized models, one promising tool is the kernel Stein discrepancy (KSD). KSD is a statistical divergence that leverages the Stein reproducing kernel—a reproducing kernel based on Stein’s method—to allow tractable computation without requiring the normalizing constant of the model likelihood. Consequently, KSD has facilitated the development of novel algorithms that can be efficiently applied to unnormalized models across various statistical tasks, including sample approximation, goodness-of-fit testing, and parameter estimation.
This thesis focuses on identifying the limitations of existing KSD-based methods and proposing novel solutions. First, we focus on KSD-based methods for sample approximation. We address the curse of dimensionality inherent in a widely used KSD-based sample approximation method, and propose an extension that mitigates this issue by using linear projections. Second, we turn our attention to goodness-of-fit tests based on KSD. We identify two key limitations of such tests: their lack of test power against multi-modal alternatives, and their lack of robustness under data contamination. For each limitation, we introduce new solutions. Finally, we explore the use of KSD in parameter estimation. We compare the relative statistical efficiency of KSD-based estimators to alternative methods, providing insights into how the kernel choice can affect their efficiency.
Date Issued
2024-09-30
Date Awarded
2025-03-01
Copyright Statement
Attribution-NonCommercial 4.0 International Licence (CC BY-NC)
License URL
Advisor
Gandy, Axel
Duncan, Andrew
Sponsor
Imperial College London President's PhD scholarship
Alan Turing Enrichment Scheme Award
Publisher Department
Department of Mathematics
Publisher Institution
Imperial College London
Qualification Level
Doctoral
Qualification Name
Doctor of Philosophy (PhD)