Theoretical and methodological advances in Bayesian semiparametrics with variational inference
File(s)
Author(s)
Travis, Luke Maxwell Morgan
Type
Thesis
Abstract
In this thesis we broadly consider the topic of Bayesian nonparametric methods. In particular we concern ourselves with methods for semiparametric inference, where one is interested in a finite-dimensional property of the infinite-dimensional parameter of interest.
We begin by introducing the Bayesian method for inference in general, and describe desirable theoretical properties that we would like the posterior distribution to satisfy.
In Chapter 2, we consider the theoretical properties of a generalised Bayesian posterior distribution known as the fractional posterior. We obtain a precise dependence of the contraction rate on the fractional exponent which defines the fractional posterior and establish a general Bernstein-von Mises result for approximately linear functionals of the parameter of interest. We illustrate the general result with a Gaussian process prior distribution in the context of the Gaussian white noise and density estimation models.
In Chapter 3, we consider the pointwise uncertainty quantification properties of a commonly used, computationally scalable, variational approximation to the Gaussian process posterior in the regression setting. With a specific prior, we obtain precise expressions for the frequentist coverage of the Bayesian credible intervals depending on the smoothness of the prior and the true underlying regression function.
In Chapter 4, we introduce a scalable Bayesian method for high-dimensional sparse linear regression. We demonstrate a semiparametric Bernstein-von Mises result for our method and show that practically it offers excellent performance, in particular overcoming the common problem with mean-field variational inference of underestimation of the posterior variance.
In Chapter 5, we introduce a Bayesian method for regression in the presence of unobservable confounding variables. We demonstrate conditions under which our method can recover the true parameter at the same rate as the analogous method in the presence of no confounding. We also demonstrate that the method offers excellent performance in a wide variety of settings.
We begin by introducing the Bayesian method for inference in general, and describe desirable theoretical properties that we would like the posterior distribution to satisfy.
In Chapter 2, we consider the theoretical properties of a generalised Bayesian posterior distribution known as the fractional posterior. We obtain a precise dependence of the contraction rate on the fractional exponent which defines the fractional posterior and establish a general Bernstein-von Mises result for approximately linear functionals of the parameter of interest. We illustrate the general result with a Gaussian process prior distribution in the context of the Gaussian white noise and density estimation models.
In Chapter 3, we consider the pointwise uncertainty quantification properties of a commonly used, computationally scalable, variational approximation to the Gaussian process posterior in the regression setting. With a specific prior, we obtain precise expressions for the frequentist coverage of the Bayesian credible intervals depending on the smoothness of the prior and the true underlying regression function.
In Chapter 4, we introduce a scalable Bayesian method for high-dimensional sparse linear regression. We demonstrate a semiparametric Bernstein-von Mises result for our method and show that practically it offers excellent performance, in particular overcoming the common problem with mean-field variational inference of underestimation of the posterior variance.
In Chapter 5, we introduce a Bayesian method for regression in the presence of unobservable confounding variables. We demonstrate conditions under which our method can recover the true parameter at the same rate as the analogous method in the presence of no confounding. We also demonstrate that the method offers excellent performance in a wide variety of settings.
Version
Open Access
Date Issued
2024-12-20
Date Awarded
2025-11-01
Copyright Statement
Attribution-NonCommercial 4.0 International Licence (CC BY-NC)
License URL
Advisor
Ray, Kolyan
Sponsor
Imperial College London
Grant Number
G44156
Publisher Department
Department of Mathematics
Publisher Institution
Imperial College London
Qualification Level
Doctoral
Qualification Name
Doctor of Philosophy (PhD)