Some aspects of approximation and computation for Bayesian inference

Abstract

This thesis studies efficient integration techniques for implementation in Bayesian inference. Specifically, analytic, computational and hybrid methods are considered, in order to obtain accurate approximations of posterior moments or marginal distributions of parameters of interest, when nuisance parameters are present. Analytic techniques play an important role in this area and a large part of this work is concerned with their application. The main methods considered are Laplace’s method for the approximation of marginal posterior densities and posterior expectations and approximations based on asymptotic expansions of signed roots of log-density ratios. A method for deriving a likelihood for a single parameter of interest, in the presence of nuisance parameters, based on inversion of confidence intervals, is also explored. Sufficient conditions under which Laplace approximations to marginal posterior densities yield exact answers are identified and these are applied in the case of natural exponential families with cuts. Computational methods and in particular the importance sampling and sampling-importance resampling algorithms are considered in conjunction with analytic methods, the results from the latter being used as starting points for the former. The aim of these hybrid schemes is to improve the accuracy of the analytic approximations on one hand and the efficiency of the sampling algorithm on the other. A further part of the thesis is concerned with the development of methods for the numerical evaluation of reference distributions. The theoretical background of reference analysis is outlined and the difficulties in its implementation illustrated. A sampling-based approach is adopted for the numerical approximation of reference distributions. The effect of reparameterizations in the accuracy and efficiency of these approximation methods is investigated throughout this work.